2024-09-14T05:18:42.867 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-14T05:18:42.883 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-14T05:18:42.975 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900319 branch: main description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} email: ceph-qa@ceph.com first_in_suite: false job_id: '7900319' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client: rbd cache: true rbd cache policy: writearound global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd memory target: 1610612736 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 0d61375b8ab53be4c233f5b93e463edfdff9316a owner: scheduled_teuthology@teuthology priority: 950 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 5839 sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 sleep_before_teardown: 0 subset: 99/128 suite: rbd suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 0d61375b8ab53be4c233f5b93e463edfdff9316a targets: smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJAs2acobITiT2tDphg8y2qH2yrlISSYq5XeAOkDe/tq9gSWjDYGhMlT8HS7dJfTqg3ICR74VygRpwO8EbaneGQ= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBzm2iJBTdRQUrsd7xUwblriZV0viHx41Tw2uJxIklbZjcH1IOPZbAw3L9XvL31D7cPQYFspIZUfKfLtGJSFUQg= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_20:16:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-14T05:18:42.975 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa; will attempt to use it 2024-09-14T05:18:42.976 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa/tasks 2024-09-14T05:18:42.976 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-14T05:18:42.977 INFO:teuthology.task.internal:Checking packages... 2024-09-14T05:18:42.999 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '7f24e589328ae9e639c0009d93340e5b235d0a47' 2024-09-14T05:18:42.999 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-14T05:18:42.999 INFO:teuthology.packaging:ref: None 2024-09-14T05:18:42.999 INFO:teuthology.packaging:tag: None 2024-09-14T05:18:42.999 INFO:teuthology.packaging:branch: main 2024-09-14T05:18:42.999 INFO:teuthology.packaging:sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T05:18:42.999 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-09-14T05:18:43.253 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4926.g1c941a66 2024-09-14T05:18:43.254 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-14T05:18:43.261 INFO:teuthology.task.internal:no buildpackages task found 2024-09-14T05:18:43.261 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-14T05:18:43.267 INFO:teuthology.task.internal:Saving configuration 2024-09-14T05:18:43.282 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-14T05:18:43.289 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-14T05:18:43.316 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900319', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 05:13:34.391325', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJAs2acobITiT2tDphg8y2qH2yrlISSYq5XeAOkDe/tq9gSWjDYGhMlT8HS7dJfTqg3ICR74VygRpwO8EbaneGQ='} 2024-09-14T05:18:43.337 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900319', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 05:13:34.392472', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBzm2iJBTdRQUrsd7xUwblriZV0viHx41Tw2uJxIklbZjcH1IOPZbAw3L9XvL31D7cPQYFspIZUfKfLtGJSFUQg='} 2024-09-14T05:18:43.337 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-14T05:18:43.344 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-14T05:18:43.344 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-14T05:18:43.344 INFO:teuthology.run_tasks:Running task console_log... 2024-09-14T05:18:43.405 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f903881f400>, signals=[15]) 2024-09-14T05:18:43.405 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-14T05:18:43.411 INFO:teuthology.task.internal:Opening connections... 2024-09-14T05:18:43.411 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2024-09-14T05:18:43.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T05:18:43.495 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-09-14T05:18:43.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T05:18:43.572 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-14T05:18:43.579 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2024-09-14T05:18:43.597 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2024-09-14T05:18:43.597 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2024-09-14T05:18:43.654 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Stream" 2024-09-14T05:18:43.654 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="9" 2024-09-14T05:18:43.654 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2024-09-14T05:18:43.654 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2024-09-14T05:18:43.655 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="9" 2024-09-14T05:18:43.655 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el9" 2024-09-14T05:18:43.655 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-14T05:18:43.655 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2024-09-14T05:18:43.655 INFO:teuthology.orchestra.run.smithi071.stdout:LOGO="fedora-logo-icon" 2024-09-14T05:18:43.655 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-14T05:18:43.655 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2024-09-14T05:18:43.655 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-14T05:18:43.655 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-14T05:18:43.655 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-14T05:18:43.656 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2024-09-14T05:18:43.677 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-09-14T05:18:43.696 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-09-14T05:18:43.696 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-09-14T05:18:43.754 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-09-14T05:18:43.754 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-09-14T05:18:43.754 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-09-14T05:18:43.754 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-09-14T05:18:43.754 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-09-14T05:18:43.754 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-09-14T05:18:43.754 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-14T05:18:43.754 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-09-14T05:18:43.754 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-09-14T05:18:43.755 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-14T05:18:43.755 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-09-14T05:18:43.755 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-14T05:18:43.755 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-14T05:18:43.755 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-14T05:18:43.755 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-09-14T05:18:43.777 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-14T05:18:43.787 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-14T05:18:43.793 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-14T05:18:43.793 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2024-09-14T05:18:43.796 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-09-14T05:18:43.814 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-14T05:18:43.820 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-14T05:18:43.821 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2024-09-14T05:18:43.853 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-09-14T05:18:43.892 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-14T05:18:43.947 INFO:teuthology.run_tasks:Running task kernel... 2024-09-14T05:18:43.964 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-14T05:18:43.964 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-14T05:18:43.964 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T05:18:43.965 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T05:18:43.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T05:18:43.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T05:18:43.981 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-09-14T05:18:43.981 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-09-14T05:18:44.037 INFO:teuthology.orchestra.run.smithi071.stdout:5.14.0-503.el9.x86_64 2024-09-14T05:18:44.037 INFO:teuthology.task.kernel:Running kernel on smithi071: 5.14.0-503.el9.x86_64 2024-09-14T05:18:44.038 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum install -y kernel 2024-09-14T05:18:44.039 INFO:teuthology.orchestra.run.smithi150.stdout:5.14.0-503.el9.x86_64 2024-09-14T05:18:44.039 INFO:teuthology.task.kernel:Running kernel on smithi150: 5.14.0-503.el9.x86_64 2024-09-14T05:18:44.039 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum install -y kernel 2024-09-14T05:18:46.346 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-14T05:18:47.486 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-09-14T05:18:48.097 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 35 MB/s | 20 MB 00:00 2024-09-14T05:18:52.448 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-14T05:18:53.237 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 4.4 MB/s | 20 MB 00:04 2024-09-14T05:18:53.965 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 19 kB 00:00 2024-09-14T05:18:54.845 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-14T05:18:58.704 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-14T05:19:00.259 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-09-14T05:19:00.371 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-14T05:19:01.234 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-14T05:19:02.274 INFO:teuthology.orchestra.run.smithi150.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-14T05:19:02.274 INFO:teuthology.orchestra.run.smithi150.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-14T05:19:02.370 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:19:02.378 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:19:02.378 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-09-14T05:19:02.378 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:19:02.379 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-14T05:19:02.379 INFO:teuthology.orchestra.run.smithi150.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-14T05:19:02.379 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-14T05:19:02.379 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-14T05:19:02.379 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-14T05:19:02.379 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-14T05:19:02.379 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:19:02.379 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-14T05:19:02.379 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:19:02.379 INFO:teuthology.orchestra.run.smithi150.stdout:Install 4 Packages 2024-09-14T05:19:02.379 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:19:02.380 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 79 M 2024-09-14T05:19:02.380 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 126 M 2024-09-14T05:19:02.380 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-14T05:19:02.760 INFO:teuthology.orchestra.run.smithi150.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 219 kB/s | 23 kB 00:00 2024-09-14T05:19:03.452 INFO:teuthology.orchestra.run.smithi150.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-09-14T05:19:03.953 INFO:teuthology.orchestra.run.smithi150.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-14T05:19:04.311 INFO:teuthology.orchestra.run.smithi150.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 21 MB/s | 35 MB 00:01 2024-09-14T05:19:04.311 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:19:04.311 INFO:teuthology.orchestra.run.smithi150.stdout:Total 41 MB/s | 79 MB 00:01 2024-09-14T05:19:04.994 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-14T05:19:05.349 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-14T05:19:05.350 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-14T05:19:06.293 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-14T05:19:06.293 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-14T05:19:06.622 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-14T05:19:08.654 INFO:teuthology.orchestra.run.smithi071.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-14T05:19:08.654 INFO:teuthology.orchestra.run.smithi071.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-14T05:19:08.749 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-14T05:19:08.759 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:19:08.763 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:19:08.763 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-09-14T05:19:08.763 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:19:08.764 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-14T05:19:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-14T05:19:08.764 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-09-14T05:19:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-14T05:19:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-14T05:19:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-14T05:19:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:19:08.764 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-14T05:19:08.764 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:19:08.764 INFO:teuthology.orchestra.run.smithi071.stdout:Install 4 Packages 2024-09-14T05:19:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:19:08.764 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 79 M 2024-09-14T05:19:08.764 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 126 M 2024-09-14T05:19:08.765 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-14T05:19:08.890 INFO:teuthology.orchestra.run.smithi071.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 291 kB/s | 23 kB 00:00 2024-09-14T05:19:09.473 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-14T05:19:09.570 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T05:19:11.590 INFO:teuthology.orchestra.run.smithi071.stdout:(2/4): kernel-modules-core-5.14.0-505.el9.x86_6 11 MB/s | 29 MB 00:02 2024-09-14T05:19:12.365 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T05:19:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T05:19:13.041 INFO:teuthology.orchestra.run.smithi071.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 3.7 MB/s | 16 MB 00:04 2024-09-14T05:19:17.317 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T05:19:17.432 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T05:19:17.790 INFO:teuthology.orchestra.run.smithi071.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 3.8 MB/s | 35 MB 00:08 2024-09-14T05:19:17.790 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:19:17.790 INFO:teuthology.orchestra.run.smithi071.stdout:Total 8.7 MB/s | 79 MB 00:09 2024-09-14T05:19:18.473 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-14T05:19:18.825 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-14T05:19:18.825 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-14T05:19:19.831 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-14T05:19:19.831 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-14T05:19:22.265 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T05:19:22.290 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-14T05:19:23.218 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-14T05:19:23.308 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T05:19:26.101 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T05:19:26.238 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T05:19:31.077 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T05:19:31.121 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T05:19:35.986 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T05:20:00.457 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T05:20:00.505 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-14T05:20:00.838 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T05:20:00.838 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-14T05:20:00.838 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T05:20:00.838 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T05:20:01.715 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T05:20:01.715 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:20:01.716 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-14T05:20:01.716 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-14T05:20:01.716 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-14T05:20:01.716 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-14T05:20:01.716 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-14T05:20:01.716 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:20:01.716 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:20:02.078 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T05:20:02.165 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T05:20:02.166 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T05:20:02.166 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-14T05:20:02.166 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T05:20:02.166 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T05:20:02.167 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-14T05:20:02.167 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi150.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T05:20:02.167 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T05:20:02.225 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T05:20:02.225 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T05:20:02.225 DEBUG:teuthology.orchestra.run.smithi150:> sudo rpm -qi grub2-tools 2024-09-14T05:20:02.256 INFO:teuthology.orchestra.run.smithi150.stdout:Name : grub2-tools 2024-09-14T05:20:02.257 INFO:teuthology.orchestra.run.smithi150.stdout:Epoch : 1 2024-09-14T05:20:02.257 INFO:teuthology.orchestra.run.smithi150.stdout:Version : 2.06 2024-09-14T05:20:02.257 INFO:teuthology.orchestra.run.smithi150.stdout:Release : 82.el9 2024-09-14T05:20:02.257 INFO:teuthology.orchestra.run.smithi150.stdout:Architecture: x86_64 2024-09-14T05:20:02.257 INFO:teuthology.orchestra.run.smithi150.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-14T05:20:02.257 INFO:teuthology.orchestra.run.smithi150.stdout:Group : Unspecified 2024-09-14T05:20:02.257 INFO:teuthology.orchestra.run.smithi150.stdout:Size : 8274670 2024-09-14T05:20:02.257 INFO:teuthology.orchestra.run.smithi150.stdout:License : GPLv3+ 2024-09-14T05:20:02.257 INFO:teuthology.orchestra.run.smithi150.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-14T05:20:02.257 INFO:teuthology.orchestra.run.smithi150.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-14T05:20:02.257 INFO:teuthology.orchestra.run.smithi150.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-14T05:20:02.258 INFO:teuthology.orchestra.run.smithi150.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-14T05:20:02.258 INFO:teuthology.orchestra.run.smithi150.stdout:Packager : builder@centos.org 2024-09-14T05:20:02.258 INFO:teuthology.orchestra.run.smithi150.stdout:Vendor : CentOS 2024-09-14T05:20:02.258 INFO:teuthology.orchestra.run.smithi150.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-14T05:20:02.258 INFO:teuthology.orchestra.run.smithi150.stdout:Summary : Support tools for GRUB. 2024-09-14T05:20:02.258 INFO:teuthology.orchestra.run.smithi150.stdout:Description : 2024-09-14T05:20:02.258 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:20:02.258 INFO:teuthology.orchestra.run.smithi150.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-14T05:20:02.258 INFO:teuthology.orchestra.run.smithi150.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-14T05:20:02.258 INFO:teuthology.orchestra.run.smithi150.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-14T05:20:02.258 INFO:teuthology.orchestra.run.smithi150.stdout:hardware devices. 2024-09-14T05:20:02.258 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:20:02.258 INFO:teuthology.orchestra.run.smithi150.stdout:This subpackage provides tools for support of all platforms. 2024-09-14T05:20:02.259 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-14T05:20:02.259 INFO:teuthology.task.kernel:Updating grub on smithi150 to boot 5.14.0-505.el9.x86_64 2024-09-14T05:20:02.259 DEBUG:teuthology.orchestra.run.smithi150:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-14T05:20:02.896 INFO:teuthology.orchestra.run.smithi150.stderr:Generating grub configuration file ... 2024-09-14T05:20:04.271 INFO:teuthology.orchestra.run.smithi150.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-14T05:20:04.304 INFO:teuthology.orchestra.run.smithi150.stderr:done 2024-09-14T05:20:04.308 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-09-14T05:20:04.325 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.Wb22uMv4Ka 2024-09-14T05:20:04.325 DEBUG:teuthology.orchestra.run.smithi150:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Wb22uMv4Ka 2024-09-14T05:20:04.392 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.Wb22uMv4Ka 2024-09-14T05:20:04.526 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.Wb22uMv4Ka is 6KB 2024-09-14T05:20:04.575 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.Wb22uMv4Ka 2024-09-14T05:20:04.590 DEBUG:teuthology.orchestra.run.smithi150:> sudo /bin/ls /boot/loader/entries || true 2024-09-14T05:20:04.653 INFO:teuthology.orchestra.run.smithi150.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-14T05:20:04.653 INFO:teuthology.orchestra.run.smithi150.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-14T05:20:04.653 INFO:teuthology.orchestra.run.smithi150.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-14T05:20:04.653 INFO:teuthology.orchestra.run.smithi150.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-14T05:20:04.654 DEBUG:teuthology.orchestra.run.smithi150:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-14T05:20:04.810 DEBUG:teuthology.orchestra.run.smithi150:> sudo shutdown -r now 2024-09-14T05:20:14.236 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T05:20:14.275 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-14T05:20:14.670 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T05:20:14.670 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-14T05:20:14.670 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T05:20:14.670 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T05:20:15.495 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T05:20:15.495 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:20:15.495 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-14T05:20:15.495 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-14T05:20:15.496 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-14T05:20:15.496 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-14T05:20:15.496 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-14T05:20:15.496 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:20:15.496 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:20:15.879 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T05:20:15.965 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T05:20:15.966 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T05:20:15.968 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-14T05:20:15.968 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T05:20:15.968 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T05:20:15.968 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-14T05:20:15.969 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi071.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T05:20:15.969 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T05:20:16.028 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T05:20:16.028 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T05:20:16.028 DEBUG:teuthology.orchestra.run.smithi071:> sudo rpm -qi grub2-tools 2024-09-14T05:20:16.068 INFO:teuthology.orchestra.run.smithi071.stdout:Name : grub2-tools 2024-09-14T05:20:16.068 INFO:teuthology.orchestra.run.smithi071.stdout:Epoch : 1 2024-09-14T05:20:16.068 INFO:teuthology.orchestra.run.smithi071.stdout:Version : 2.06 2024-09-14T05:20:16.068 INFO:teuthology.orchestra.run.smithi071.stdout:Release : 82.el9 2024-09-14T05:20:16.068 INFO:teuthology.orchestra.run.smithi071.stdout:Architecture: x86_64 2024-09-14T05:20:16.068 INFO:teuthology.orchestra.run.smithi071.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-14T05:20:16.068 INFO:teuthology.orchestra.run.smithi071.stdout:Group : Unspecified 2024-09-14T05:20:16.068 INFO:teuthology.orchestra.run.smithi071.stdout:Size : 8274670 2024-09-14T05:20:16.069 INFO:teuthology.orchestra.run.smithi071.stdout:License : GPLv3+ 2024-09-14T05:20:16.069 INFO:teuthology.orchestra.run.smithi071.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-14T05:20:16.069 INFO:teuthology.orchestra.run.smithi071.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-14T05:20:16.069 INFO:teuthology.orchestra.run.smithi071.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-14T05:20:16.069 INFO:teuthology.orchestra.run.smithi071.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-14T05:20:16.069 INFO:teuthology.orchestra.run.smithi071.stdout:Packager : builder@centos.org 2024-09-14T05:20:16.069 INFO:teuthology.orchestra.run.smithi071.stdout:Vendor : CentOS 2024-09-14T05:20:16.069 INFO:teuthology.orchestra.run.smithi071.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-14T05:20:16.069 INFO:teuthology.orchestra.run.smithi071.stdout:Summary : Support tools for GRUB. 2024-09-14T05:20:16.069 INFO:teuthology.orchestra.run.smithi071.stdout:Description : 2024-09-14T05:20:16.070 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:20:16.070 INFO:teuthology.orchestra.run.smithi071.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-14T05:20:16.070 INFO:teuthology.orchestra.run.smithi071.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-14T05:20:16.070 INFO:teuthology.orchestra.run.smithi071.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-14T05:20:16.070 INFO:teuthology.orchestra.run.smithi071.stdout:hardware devices. 2024-09-14T05:20:16.070 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:20:16.070 INFO:teuthology.orchestra.run.smithi071.stdout:This subpackage provides tools for support of all platforms. 2024-09-14T05:20:16.071 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-14T05:20:16.071 INFO:teuthology.task.kernel:Updating grub on smithi071 to boot 5.14.0-505.el9.x86_64 2024-09-14T05:20:16.071 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-14T05:20:16.739 INFO:teuthology.orchestra.run.smithi071.stderr:Generating grub configuration file ... 2024-09-14T05:20:18.041 INFO:teuthology.orchestra.run.smithi071.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-14T05:20:18.071 INFO:teuthology.orchestra.run.smithi071.stderr:done 2024-09-14T05:20:18.073 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-09-14T05:20:18.089 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.YASBgIgT5J 2024-09-14T05:20:18.089 DEBUG:teuthology.orchestra.run.smithi071:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.YASBgIgT5J 2024-09-14T05:20:18.157 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.YASBgIgT5J 2024-09-14T05:20:18.305 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.YASBgIgT5J is 6KB 2024-09-14T05:20:18.326 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.YASBgIgT5J 2024-09-14T05:20:18.341 DEBUG:teuthology.orchestra.run.smithi071:> sudo /bin/ls /boot/loader/entries || true 2024-09-14T05:20:18.403 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-14T05:20:18.403 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-14T05:20:18.403 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-14T05:20:18.404 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-14T05:20:18.405 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-14T05:20:18.560 DEBUG:teuthology.orchestra.run.smithi071:> sudo shutdown -r now 2024-09-14T05:20:34.828 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-14T05:20:34.829 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-14T05:20:34.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T05:20:48.576 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-14T05:20:48.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-14T05:20:48.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T05:20:53.309 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-09-14T05:21:02.317 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-14T05:21:02.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T05:21:05.373 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-09-14T05:21:07.133 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-09-14T05:21:16.141 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-14T05:21:16.141 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T05:21:17.375 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-14T05:21:17.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T05:21:19.197 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-09-14T05:21:31.209 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-14T05:21:31.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T05:21:35.809 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-09-14T05:21:49.629 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-09-14T05:21:50.811 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-14T05:21:50.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T05:21:51.466 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-09-14T05:21:51.921 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-14T05:21:51.921 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-14T05:21:51.921 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-09-14T05:21:51.936 INFO:teuthology.orchestra.run.smithi150.stdout:5.14.0-505.el9.x86_64 2024-09-14T05:21:51.936 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-14T05:21:51.936 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T05:21:51.936 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-14T05:21:52.937 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-14T05:21:52.938 DEBUG:teuthology.orchestra.run.smithi150:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T05:21:53.022 INFO:teuthology.orchestra.run.smithi150.stdout:ttyS1 2024-09-14T05:21:53.049 DEBUG:teuthology.parallel:result is None 2024-09-14T05:22:04.630 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-14T05:22:04.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T05:22:04.951 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-09-14T05:22:05.309 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-14T05:22:05.309 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-14T05:22:05.309 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-09-14T05:22:05.324 INFO:teuthology.orchestra.run.smithi071.stdout:5.14.0-505.el9.x86_64 2024-09-14T05:22:05.324 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-14T05:22:05.324 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T05:22:05.324 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-14T05:22:06.324 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-14T05:22:06.325 DEBUG:teuthology.orchestra.run.smithi071:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T05:22:06.446 INFO:teuthology.orchestra.run.smithi071.stdout:ttyS1 2024-09-14T05:22:06.470 DEBUG:teuthology.parallel:result is None 2024-09-14T05:22:06.470 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-14T05:22:06.478 INFO:teuthology.task.internal:Creating test directory... 2024-09-14T05:22:06.478 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T05:22:06.480 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T05:22:06.527 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-14T05:22:06.533 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-14T05:22:06.543 INFO:teuthology.task.internal:Creating archive directory... 2024-09-14T05:22:06.543 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T05:22:06.546 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T05:22:06.669 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-14T05:22:06.675 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-14T05:22:06.675 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T05:22:06.678 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T05:22:06.709 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T05:22:06.722 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T05:22:06.742 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T05:22:06.759 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T05:22:06.761 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-14T05:22:06.769 INFO:teuthology.task.internal:Configuring sudo... 2024-09-14T05:22:06.769 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T05:22:06.772 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T05:22:06.829 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-14T05:22:06.838 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-14T05:22:06.838 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T05:22:06.842 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T05:22:06.888 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T05:22:06.956 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T05:22:07.039 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T05:22:07.085 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T05:22:07.125 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:22:07.125 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T05:22:07.189 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T05:22:07.238 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T05:22:07.290 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T05:22:07.377 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T05:22:07.420 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:22:07.421 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T05:22:07.490 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2024-09-14T05:22:07.492 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-09-14T05:22:07.542 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-14T05:22:07.598 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-14T05:22:08.121 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-14T05:22:08.128 INFO:teuthology.task.internal:Starting timer... 2024-09-14T05:22:08.128 INFO:teuthology.run_tasks:Running task pcp... 2024-09-14T05:22:08.138 INFO:teuthology.run_tasks:Running task selinux... 2024-09-14T05:22:08.148 DEBUG:teuthology.orchestra.run.smithi071:> sudo service auditd rotate 2024-09-14T05:22:08.182 INFO:teuthology.orchestra.run.smithi071.stdout:Rotating logs: 2024-09-14T05:22:08.184 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-09-14T05:22:08.236 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-09-14T05:22:08.238 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-14T05:22:08.238 DEBUG:teuthology.orchestra.run.smithi071:> /usr/sbin/getenforce 2024-09-14T05:22:08.270 INFO:teuthology.orchestra.run.smithi071.stdout:Permissive 2024-09-14T05:22:08.271 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-09-14T05:22:08.298 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-09-14T05:22:08.299 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi071.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi150.front.sepia.ceph.com': 'permissive'} 2024-09-14T05:22:08.299 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-14T05:22:08.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T05:22:08.334 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-14T05:22:08.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T05:22:08.366 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-14T05:22:08.367 DEBUG:teuthology.orchestra.run.smithi071:> sudo /usr/sbin/setenforce permissive 2024-09-14T05:22:08.398 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-09-14T05:22:08.437 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-14T05:22:08.451 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-14T05:22:08.455 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-14T05:22:08.565 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-14T05:22:08.576 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-14T05:22:08.577 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi071.front.sepia.ceph.com,smithi150.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-14T05:25:55.365 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi071.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com')] 2024-09-14T05:25:55.366 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-14T05:25:55.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T05:25:55.447 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-09-14T05:25:55.531 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-14T05:25:55.531 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-14T05:25:55.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T05:25:55.608 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-09-14T05:25:55.690 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-14T05:25:55.690 INFO:teuthology.run_tasks:Running task clock... 2024-09-14T05:25:55.701 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-14T05:25:55.701 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T05:25:55.702 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T05:25:55.704 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T05:25:55.704 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T05:25:55.730 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-14T05:25:55.744 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-14T05:25:55.761 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-14T05:25:55.768 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: ntpd: command not found 2024-09-14T05:25:55.775 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-14T05:25:55.779 INFO:teuthology.orchestra.run.smithi071.stdout:506 Cannot talk to daemon 2024-09-14T05:25:55.793 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-14T05:25:55.800 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-09-14T05:25:55.807 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-14T05:25:55.811 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-09-14T05:25:55.826 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-14T05:25:55.840 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-14T05:25:55.861 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-09-14T05:25:55.864 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-14T05:25:55.864 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-09-14T05:25:55.864 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T05:25:55.864 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T05:25:55.864 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T05:25:55.864 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T05:25:55.894 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-09-14T05:25:55.896 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-14T05:25:55.897 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-09-14T05:25:55.897 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T05:25:55.897 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T05:25:55.897 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T05:25:55.897 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T05:25:55.898 INFO:teuthology.run_tasks:Running task install... 2024-09-14T05:25:55.906 DEBUG:teuthology.task.install:project ceph 2024-09-14T05:25:55.906 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}} 2024-09-14T05:25:55.906 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'} 2024-09-14T05:25:55.906 INFO:teuthology.task.install:Using flavor: default 2024-09-14T05:25:55.912 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-14T05:25:55.912 INFO:teuthology.task.install:extra packages: [] 2024-09-14T05:25:55.913 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47', 'tag': None, 'wait_for_package': False} 2024-09-14T05:25:55.913 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T05:25:55.916 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47', 'tag': None, 'wait_for_package': False} 2024-09-14T05:25:55.916 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T05:25:56.062 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/ 2024-09-14T05:25:56.062 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4775.g7f24e589 2024-09-14T05:25:56.076 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/ 2024-09-14T05:25:56.077 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4775.g7f24e589 2024-09-14T05:25:56.176 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-14T05:25:56.177 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:25:56.177 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-14T05:25:56.187 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-14T05:25:56.187 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:25:56.187 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-14T05:25:56.202 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-14T05:25:56.202 DEBUG:teuthology.orchestra.run.smithi071:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7f24e589328ae9e639c0009d93340e5b235d0a47/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-14T05:25:56.213 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-14T05:25:56.214 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7f24e589328ae9e639c0009d93340e5b235d0a47/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-14T05:25:56.268 DEBUG:teuthology.orchestra.run.smithi071:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-14T05:25:56.281 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-14T05:25:56.343 DEBUG:teuthology.orchestra.run.smithi071:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-14T05:25:56.357 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-14T05:25:56.408 INFO:teuthology.orchestra.run.smithi071.stdout:check_obsoletes = 1 2024-09-14T05:25:56.410 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum clean all 2024-09-14T05:25:56.421 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-09-14T05:25:56.423 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-09-14T05:25:56.637 INFO:teuthology.orchestra.run.smithi071.stdout:45 files removed 2024-09-14T05:25:56.644 INFO:teuthology.orchestra.run.smithi150.stdout:45 files removed 2024-09-14T05:25:56.664 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-radosgw 2024-09-14T05:25:56.672 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw 2024-09-14T05:25:57.209 INFO:teuthology.orchestra.run.smithi071.stdout:ceph packages for x86_64 320 kB/s | 84 kB 00:00 2024-09-14T05:25:57.216 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 326 kB/s | 84 kB 00:00 2024-09-14T05:25:57.504 INFO:teuthology.orchestra.run.smithi071.stdout:ceph noarch packages 96 kB/s | 24 kB 00:00 2024-09-14T05:25:57.517 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-14T05:25:57.776 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 8.7 kB/s | 1.9 kB 00:00 2024-09-14T05:25:57.863 INFO:teuthology.orchestra.run.smithi071.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-09-14T05:25:58.301 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-14T05:25:59.529 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.2 MB 00:01 2024-09-14T05:26:01.594 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 9.3 MB/s | 20 MB 00:02 2024-09-14T05:26:01.689 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 19 MB/s | 20 MB 00:01 2024-09-14T05:26:06.049 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-14T05:26:07.848 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-14T05:26:08.861 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-14T05:26:09.453 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 6.5 MB 00:04 2024-09-14T05:26:11.064 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 19 kB 00:00 2024-09-14T05:26:12.301 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-14T05:26:14.266 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-14T05:26:16.625 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:26:16.637 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:26:16.637 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-09-14T05:26:16.637 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:26:16.637 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-14T05:26:16.638 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 11 M 2024-09-14T05:26:16.638 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrading: 2024-09-14T05:26:16.638 INFO:teuthology.orchestra.run.smithi071.stdout: librados2 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 3.5 M 2024-09-14T05:26:16.638 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 3.2 M 2024-09-14T05:26:16.638 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-09-14T05:26:16.638 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-14T05:26:16.638 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 5.5 M 2024-09-14T05:26:16.638 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 22 M 2024-09-14T05:26:16.638 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 25 k 2024-09-14T05:26:16.638 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-14T05:26:16.638 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-14T05:26:16.638 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-14T05:26:16.639 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 755 k 2024-09-14T05:26:16.639 INFO:teuthology.orchestra.run.smithi071.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-14T05:26:16.639 INFO:teuthology.orchestra.run.smithi071.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-14T05:26:16.639 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-14T05:26:16.639 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-14T05:26:16.639 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 508 k 2024-09-14T05:26:16.639 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-14T05:26:16.639 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 5.5 M 2024-09-14T05:26:16.639 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-14T05:26:16.639 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-14T05:26:16.639 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-14T05:26:16.639 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 45 k 2024-09-14T05:26:16.640 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 143 k 2024-09-14T05:26:16.640 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 161 k 2024-09-14T05:26:16.640 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-14T05:26:16.640 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 321 k 2024-09-14T05:26:16.640 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 306 k 2024-09-14T05:26:16.640 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 99 k 2024-09-14T05:26:16.640 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-14T05:26:16.640 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-14T05:26:16.640 INFO:teuthology.orchestra.run.smithi071.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-14T05:26:16.640 INFO:teuthology.orchestra.run.smithi071.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-14T05:26:16.640 INFO:teuthology.orchestra.run.smithi071.stdout:Installing weak dependencies: 2024-09-14T05:26:16.640 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-14T05:26:16.641 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:26:16.641 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-14T05:26:16.641 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:26:16.641 INFO:teuthology.orchestra.run.smithi071.stdout:Install 31 Packages 2024-09-14T05:26:16.641 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrade 2 Packages 2024-09-14T05:26:16.641 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:26:16.641 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 62 M 2024-09-14T05:26:16.641 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-14T05:26:17.742 INFO:teuthology.orchestra.run.smithi071.stdout:(1/33): ceph-common-19.3.0-4775.g7f24e589.el9.x 22 MB/s | 22 MB 00:00 2024-09-14T05:26:17.817 INFO:teuthology.orchestra.run.smithi071.stdout:(2/33): ceph-base-19.3.0-4775.g7f24e589.el9.x86 5.1 MB/s | 5.5 MB 00:01 2024-09-14T05:26:17.843 INFO:teuthology.orchestra.run.smithi071.stdout:(3/33): ceph-selinux-19.3.0-4775.g7f24e589.el9. 251 kB/s | 25 kB 00:00 2024-09-14T05:26:17.849 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-14T05:26:17.959 INFO:teuthology.orchestra.run.smithi071.stdout:(4/33): ceph-radosgw-19.3.0-4775.g7f24e589.el9. 9.0 MB/s | 11 MB 00:01 2024-09-14T05:26:17.993 INFO:teuthology.orchestra.run.smithi071.stdout:(5/33): libcephfs2-19.3.0-4775.g7f24e589.el9.x8 4.2 MB/s | 755 kB 00:00 2024-09-14T05:26:18.027 INFO:teuthology.orchestra.run.smithi071.stdout:(6/33): libradosstriper1-19.3.0-4775.g7f24e589. 2.7 MB/s | 508 kB 00:00 2024-09-14T05:26:18.052 INFO:teuthology.orchestra.run.smithi071.stdout:(7/33): python3-ceph-argparse-19.3.0-4775.g7f24 771 kB/s | 45 kB 00:00 2024-09-14T05:26:18.077 INFO:teuthology.orchestra.run.smithi071.stdout:(8/33): python3-ceph-common-19.3.0-4775.g7f24e5 2.8 MB/s | 143 kB 00:00 2024-09-14T05:26:18.102 INFO:teuthology.orchestra.run.smithi071.stdout:(9/33): python3-cephfs-19.3.0-4775.g7f24e589.el 3.1 MB/s | 161 kB 00:00 2024-09-14T05:26:18.144 INFO:teuthology.orchestra.run.smithi071.stdout:(10/33): python3-rados-19.3.0-4775.g7f24e589.el 4.7 MB/s | 321 kB 00:00 2024-09-14T05:26:18.170 INFO:teuthology.orchestra.run.smithi071.stdout:(11/33): python3-rbd-19.3.0-4775.g7f24e589.el9. 4.5 MB/s | 306 kB 00:00 2024-09-14T05:26:18.195 INFO:teuthology.orchestra.run.smithi071.stdout:(12/33): python3-rgw-19.3.0-4775.g7f24e589.el9. 1.9 MB/s | 99 kB 00:00 2024-09-14T05:26:18.270 INFO:teuthology.orchestra.run.smithi071.stdout:(13/33): librgw2-19.3.0-4775.g7f24e589.el9.x86_ 18 MB/s | 5.5 MB 00:00 2024-09-14T05:26:18.645 INFO:teuthology.orchestra.run.smithi071.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 225 kB/s | 107 kB 00:00 2024-09-14T05:26:18.754 INFO:teuthology.orchestra.run.smithi071.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 305 kB/s | 171 kB 00:00 2024-09-14T05:26:18.779 INFO:teuthology.orchestra.run.smithi071.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 339 kB/s | 45 kB 00:00 2024-09-14T05:26:18.830 INFO:teuthology.orchestra.run.smithi071.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 286 kB/s | 160 kB 00:00 2024-09-14T05:26:18.938 INFO:teuthology.orchestra.run.smithi071.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 390 kB/s | 42 kB 00:00 2024-09-14T05:26:19.063 INFO:teuthology.orchestra.run.smithi071.stdout:(19/33): librdkafka-1.6.1-102.el9.x86_64.rpm 2.1 MB/s | 662 kB 00:00 2024-09-14T05:26:19.089 INFO:teuthology.orchestra.run.smithi071.stdout:(20/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 946 kB/s | 292 kB 00:00 2024-09-14T05:26:19.139 INFO:teuthology.orchestra.run.smithi071.stdout:(21/33): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 221 kB 00:00 2024-09-14T05:26:19.164 INFO:teuthology.orchestra.run.smithi071.stdout:(22/33): qatlib-service-24.02.0-1.el9.x86_64.rp 357 kB/s | 36 kB 00:00 2024-09-14T05:26:19.189 INFO:teuthology.orchestra.run.smithi071.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 472 kB/s | 47 kB 00:00 2024-09-14T05:26:19.231 INFO:teuthology.orchestra.run.smithi071.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.3 MB/s | 308 kB 00:00 2024-09-14T05:26:19.257 INFO:teuthology.orchestra.run.smithi071.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2024-09-14T05:26:19.282 INFO:teuthology.orchestra.run.smithi071.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 287 kB/s | 26 kB 00:00 2024-09-14T05:26:19.307 INFO:teuthology.orchestra.run.smithi071.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-14T05:26:19.341 INFO:teuthology.orchestra.run.smithi071.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.7 MB/s | 191 kB 00:00 2024-09-14T05:26:19.408 INFO:teuthology.orchestra.run.smithi071.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-09-14T05:26:19.475 INFO:teuthology.orchestra.run.smithi071.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-14T05:26:19.617 INFO:teuthology.orchestra.run.smithi071.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-09-14T05:26:19.717 INFO:teuthology.orchestra.run.smithi071.stdout:(32/33): librbd1-19.3.0-4775.g7f24e589.el9.x86_ 13 MB/s | 3.2 MB 00:00 2024-09-14T05:26:19.767 INFO:teuthology.orchestra.run.smithi071.stdout:(33/33): librados2-19.3.0-4775.g7f24e589.el9.x8 9.7 MB/s | 3.5 MB 00:00 2024-09-14T05:26:19.772 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:26:19.772 INFO:teuthology.orchestra.run.smithi071.stdout:Total 20 MB/s | 62 MB 00:03 2024-09-14T05:26:19.892 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-14T05:26:19.930 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-14T05:26:19.931 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-14T05:26:20.225 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:26:20.255 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:26:20.255 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-14T05:26:20.255 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:26:20.255 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-14T05:26:20.255 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 11 M 2024-09-14T05:26:20.256 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-09-14T05:26:20.256 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 3.5 M 2024-09-14T05:26:20.256 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 3.2 M 2024-09-14T05:26:20.256 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-14T05:26:20.256 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-14T05:26:20.256 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 5.5 M 2024-09-14T05:26:20.256 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 22 M 2024-09-14T05:26:20.256 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 25 k 2024-09-14T05:26:20.256 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-14T05:26:20.256 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-14T05:26:20.256 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-14T05:26:20.256 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 755 k 2024-09-14T05:26:20.257 INFO:teuthology.orchestra.run.smithi150.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-14T05:26:20.257 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-14T05:26:20.257 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-14T05:26:20.257 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-14T05:26:20.257 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 508 k 2024-09-14T05:26:20.257 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-14T05:26:20.257 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 5.5 M 2024-09-14T05:26:20.257 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-14T05:26:20.257 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-14T05:26:20.257 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-14T05:26:20.257 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 45 k 2024-09-14T05:26:20.257 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 143 k 2024-09-14T05:26:20.257 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 161 k 2024-09-14T05:26:20.258 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-14T05:26:20.258 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 321 k 2024-09-14T05:26:20.258 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 306 k 2024-09-14T05:26:20.258 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 99 k 2024-09-14T05:26:20.258 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-14T05:26:20.258 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-14T05:26:20.258 INFO:teuthology.orchestra.run.smithi150.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-14T05:26:20.258 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-14T05:26:20.258 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-09-14T05:26:20.258 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-14T05:26:20.258 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:26:20.258 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-14T05:26:20.258 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:26:20.258 INFO:teuthology.orchestra.run.smithi150.stdout:Install 31 Packages 2024-09-14T05:26:20.259 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 2 Packages 2024-09-14T05:26:20.259 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:26:20.259 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 62 M 2024-09-14T05:26:20.259 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-14T05:26:20.579 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-14T05:26:20.580 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-14T05:26:20.992 INFO:teuthology.orchestra.run.smithi150.stdout:(1/33): ceph-base-19.3.0-4775.g7f24e589.el9.x86 10 MB/s | 5.5 MB 00:00 2024-09-14T05:26:21.059 INFO:teuthology.orchestra.run.smithi150.stdout:(2/33): ceph-selinux-19.3.0-4775.g7f24e589.el9. 378 kB/s | 25 kB 00:00 2024-09-14T05:26:21.175 INFO:teuthology.orchestra.run.smithi150.stdout:(3/33): ceph-radosgw-19.3.0-4775.g7f24e589.el9. 15 MB/s | 11 MB 00:00 2024-09-14T05:26:21.217 INFO:teuthology.orchestra.run.smithi150.stdout:(4/33): libcephfs2-19.3.0-4775.g7f24e589.el9.x8 4.7 MB/s | 755 kB 00:00 2024-09-14T05:26:21.242 INFO:teuthology.orchestra.run.smithi150.stdout:(5/33): libradosstriper1-19.3.0-4775.g7f24e589. 7.5 MB/s | 508 kB 00:00 2024-09-14T05:26:21.301 INFO:teuthology.orchestra.run.smithi150.stdout:(6/33): python3-ceph-argparse-19.3.0-4775.g7f24 774 kB/s | 45 kB 00:00 2024-09-14T05:26:21.393 INFO:teuthology.orchestra.run.smithi150.stdout:(7/33): python3-ceph-common-19.3.0-4775.g7f24e5 1.5 MB/s | 143 kB 00:00 2024-09-14T05:26:21.468 INFO:teuthology.orchestra.run.smithi150.stdout:(8/33): librgw2-19.3.0-4775.g7f24e589.el9.x86_6 22 MB/s | 5.5 MB 00:00 2024-09-14T05:26:21.493 INFO:teuthology.orchestra.run.smithi150.stdout:(9/33): python3-cephfs-19.3.0-4775.g7f24e589.el 1.6 MB/s | 161 kB 00:00 2024-09-14T05:26:21.519 INFO:teuthology.orchestra.run.smithi150.stdout:(10/33): python3-rados-19.3.0-4775.g7f24e589.el 6.2 MB/s | 321 kB 00:00 2024-09-14T05:26:21.544 INFO:teuthology.orchestra.run.smithi150.stdout:(11/33): python3-rbd-19.3.0-4775.g7f24e589.el9. 5.9 MB/s | 306 kB 00:00 2024-09-14T05:26:21.569 INFO:teuthology.orchestra.run.smithi150.stdout:(12/33): python3-rgw-19.3.0-4775.g7f24e589.el9. 1.9 MB/s | 99 kB 00:00 2024-09-14T05:26:21.778 INFO:teuthology.orchestra.run.smithi150.stdout:(13/33): ceph-common-19.3.0-4775.g7f24e589.el9. 16 MB/s | 22 MB 00:01 2024-09-14T05:26:21.886 INFO:teuthology.orchestra.run.smithi150.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 312 kB/s | 107 kB 00:00 2024-09-14T05:26:21.937 INFO:teuthology.orchestra.run.smithi150.stdout:(15/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-09-14T05:26:21.987 INFO:teuthology.orchestra.run.smithi150.stdout:(16/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 768 kB/s | 160 kB 00:00 2024-09-14T05:26:22.012 INFO:teuthology.orchestra.run.smithi150.stdout:(17/33): libnbd-1.20.2-2.el9.x86_64.rpm 385 kB/s | 171 kB 00:00 2024-09-14T05:26:22.063 INFO:teuthology.orchestra.run.smithi150.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 847 kB/s | 42 kB 00:00 2024-09-14T05:26:22.121 INFO:teuthology.orchestra.run.smithi150.stdout:(19/33): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-09-14T05:26:22.134 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-14T05:26:22.147 INFO:teuthology.orchestra.run.smithi150.stdout:(20/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-14T05:26:22.172 INFO:teuthology.orchestra.run.smithi150.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 713 kB/s | 36 kB 00:00 2024-09-14T05:26:22.197 INFO:teuthology.orchestra.run.smithi150.stdout:(22/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 938 kB/s | 47 kB 00:00 2024-09-14T05:26:22.217 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-14T05:26:22.239 INFO:teuthology.orchestra.run.smithi150.stdout:(23/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.1 MB/s | 292 kB 00:00 2024-09-14T05:26:22.247 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-14T05:26:22.273 INFO:teuthology.orchestra.run.smithi150.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.0 MB/s | 308 kB 00:00 2024-09-14T05:26:22.280 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-14T05:26:22.298 INFO:teuthology.orchestra.run.smithi150.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-14T05:26:22.311 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-14T05:26:22.316 INFO:teuthology.orchestra.run.smithi150.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 354 kB/s | 26 kB 00:00 2024-09-14T05:26:22.331 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-14T05:26:22.340 INFO:teuthology.orchestra.run.smithi150.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-14T05:26:22.374 INFO:teuthology.orchestra.run.smithi150.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-14T05:26:22.449 INFO:teuthology.orchestra.run.smithi150.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-09-14T05:26:22.491 INFO:teuthology.orchestra.run.smithi150.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-14T05:26:22.521 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-14T05:26:22.583 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-14T05:26:22.583 INFO:teuthology.orchestra.run.smithi150.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-14T05:26:22.708 INFO:teuthology.orchestra.run.smithi150.stdout:(32/33): librados2-19.3.0-4775.g7f24e589.el9.x8 13 MB/s | 3.5 MB 00:00 2024-09-14T05:26:22.758 INFO:teuthology.orchestra.run.smithi150.stdout:(33/33): librbd1-19.3.0-4775.g7f24e589.el9.x86_ 12 MB/s | 3.2 MB 00:00 2024-09-14T05:26:22.763 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:26:22.763 INFO:teuthology.orchestra.run.smithi150.stdout:Total 25 MB/s | 62 MB 00:02 2024-09-14T05:26:22.873 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-14T05:26:22.910 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-14T05:26:22.910 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-14T05:26:22.985 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-14T05:26:23.040 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 7/35 2024-09-14T05:26:23.132 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 7/35 2024-09-14T05:26:23.223 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 8/35 2024-09-14T05:26:23.250 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 9/35 2024-09-14T05:26:23.313 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 9/35 2024-09-14T05:26:23.350 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-14T05:26:23.394 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-argparse-2:19.3.0-4775.g7f24e589.el9. 11/35 2024-09-14T05:26:23.475 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 12/35 2024-09-14T05:26:23.502 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_6 13/35 2024-09-14T05:26:23.566 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_6 13/35 2024-09-14T05:26:23.596 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-14T05:26:23.596 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-14T05:26:23.600 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-14T05:26:23.718 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-14T05:26:23.752 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-14T05:26:23.792 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-14T05:26:23.833 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-14T05:26:23.862 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-14T05:26:23.905 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-14T05:26:23.941 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-14T05:26:24.260 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-14T05:26:24.317 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 23/35 2024-09-14T05:26:24.416 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 23/35 2024-09-14T05:26:24.455 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 24/35 2024-09-14T05:26:24.499 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-14T05:26:25.000 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-14T05:26:25.027 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 27/35 2024-09-14T05:26:25.097 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-14T05:26:25.137 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 27/35 2024-09-14T05:26:25.187 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-14T05:26:25.201 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 28/35 2024-09-14T05:26:25.222 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-14T05:26:25.229 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-common-2:19.3.0-4775.g7f24e589.el9.x8 29/35 2024-09-14T05:26:25.255 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-14T05:26:25.277 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-14T05:26:25.301 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-14T05:26:25.505 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-14T05:26:25.558 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-14T05:26:25.973 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-14T05:26:26.032 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 7/35 2024-09-14T05:26:26.124 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 7/35 2024-09-14T05:26:26.223 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 8/35 2024-09-14T05:26:26.250 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 9/35 2024-09-14T05:26:26.313 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 9/35 2024-09-14T05:26:26.350 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-14T05:26:26.394 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:19.3.0-4775.g7f24e589.el9. 11/35 2024-09-14T05:26:26.464 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 12/35 2024-09-14T05:26:26.494 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_6 13/35 2024-09-14T05:26:26.558 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_6 13/35 2024-09-14T05:26:26.592 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-14T05:26:26.712 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-14T05:26:26.743 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-14T05:26:26.784 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-14T05:26:26.817 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-14T05:26:26.846 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-14T05:26:26.879 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-14T05:26:26.916 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-14T05:26:27.238 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-14T05:26:27.276 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 23/35 2024-09-14T05:26:27.358 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 23/35 2024-09-14T05:26:27.389 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 24/35 2024-09-14T05:26:27.425 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-14T05:26:27.534 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 30/35 2024-09-14T05:26:27.576 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 30/35 2024-09-14T05:26:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-14T05:26:27.953 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 27/35 2024-09-14T05:26:28.016 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 27/35 2024-09-14T05:26:28.093 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 28/35 2024-09-14T05:26:28.122 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:19.3.0-4775.g7f24e589.el9.x8 29/35 2024-09-14T05:26:28.135 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 30/35 2024-09-14T05:26:28.187 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 31/35 2024-09-14T05:26:28.278 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 31/35 2024-09-14T05:26:28.279 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-14T05:26:28.279 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-14T05:26:28.279 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:26:28.329 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 32/35 2024-09-14T05:26:30.421 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 30/35 2024-09-14T05:26:30.452 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 30/35 2024-09-14T05:26:31.013 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 30/35 2024-09-14T05:26:31.046 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 31/35 2024-09-14T05:26:31.145 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 31/35 2024-09-14T05:26:31.145 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-14T05:26:31.145 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-14T05:26:31.146 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:26:31.173 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 32/35 2024-09-14T05:26:38.315 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 32/35 2024-09-14T05:26:38.316 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /sys 2024-09-14T05:26:38.316 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /proc 2024-09-14T05:26:38.316 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /mnt 2024-09-14T05:26:38.316 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /var/tmp 2024-09-14T05:26:38.316 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /home 2024-09-14T05:26:38.316 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /root 2024-09-14T05:26:38.316 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /tmp 2024-09-14T05:26:38.316 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:26:39.320 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 33/35 2024-09-14T05:26:39.345 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 33/35 2024-09-14T05:26:39.346 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T05:26:39.346 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-14T05:26:39.346 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T05:26:39.346 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T05:26:39.346 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:26:39.348 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-14T05:26:39.430 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-14T05:26:39.431 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-14T05:26:40.587 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-14T05:26:40.587 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 1/35 2024-09-14T05:26:40.587 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 2/35 2024-09-14T05:26:40.587 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 3/35 2024-09-14T05:26:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 4/35 2024-09-14T05:26:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 5/35 2024-09-14T05:26:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_6 6/35 2024-09-14T05:26:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 7/35 2024-09-14T05:26:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4775.g7f24e589.el9. 8/35 2024-09-14T05:26:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-common-2:19.3.0-4775.g7f24e589.el9.x8 9/35 2024-09-14T05:26:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 10/35 2024-09-14T05:26:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 11/35 2024-09-14T05:26:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 12/35 2024-09-14T05:26:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 13/35 2024-09-14T05:26:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-14T05:26:40.588 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-14T05:26:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-14T05:26:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-14T05:26:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-14T05:26:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-14T05:26:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-14T05:26:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-14T05:26:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-14T05:26:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-14T05:26:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-14T05:26:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-14T05:26:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-14T05:26:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-14T05:26:40.589 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-14T05:26:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-14T05:26:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-14T05:26:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-14T05:26:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 32/35 2024-09-14T05:26:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-14T05:26:40.590 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 34/35 2024-09-14T05:26:41.403 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 32/35 2024-09-14T05:26:41.403 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-09-14T05:26:41.403 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-09-14T05:26:41.403 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-09-14T05:26:41.403 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-09-14T05:26:41.403 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-09-14T05:26:41.403 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /root 2024-09-14T05:26:41.404 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-09-14T05:26:41.404 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:26:41.562 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-14T05:26:41.562 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:26:41.562 INFO:teuthology.orchestra.run.smithi071.stdout:Upgraded: 2024-09-14T05:26:41.562 INFO:teuthology.orchestra.run.smithi071.stdout: librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:41.562 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:41.562 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-14T05:26:41.562 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-14T05:26:41.562 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:41.563 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:41.563 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:41.563 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:41.563 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-14T05:26:41.563 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-14T05:26:41.563 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-14T05:26:41.563 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:41.563 INFO:teuthology.orchestra.run.smithi071.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-14T05:26:41.563 INFO:teuthology.orchestra.run.smithi071.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-14T05:26:41.563 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-14T05:26:41.563 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-14T05:26:41.563 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:41.563 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-14T05:26:41.564 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:41.564 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-14T05:26:41.564 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-14T05:26:41.564 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-14T05:26:41.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:41.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:41.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:41.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-14T05:26:41.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:41.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:41.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:41.564 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-14T05:26:41.564 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-14T05:26:41.564 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-14T05:26:41.565 INFO:teuthology.orchestra.run.smithi071.stdout: re2-1:20211101-20.el9.x86_64 2024-09-14T05:26:41.565 INFO:teuthology.orchestra.run.smithi071.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-14T05:26:41.565 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:26:41.565 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:26:41.929 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-test 2024-09-14T05:26:42.423 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 33/35 2024-09-14T05:26:42.449 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 33/35 2024-09-14T05:26:42.449 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T05:26:42.449 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-14T05:26:42.449 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T05:26:42.449 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T05:26:42.449 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:26:42.451 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-14T05:26:42.506 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:00:28 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:26:42.551 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-14T05:26:42.551 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-14T05:26:42.639 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:26:42.640 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:26:42.640 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-09-14T05:26:42.640 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:26:42.640 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-14T05:26:42.640 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 50 M 2024-09-14T05:26:42.640 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-09-14T05:26:42.640 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 162 k 2024-09-14T05:26:42.640 INFO:teuthology.orchestra.run.smithi071.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-14T05:26:42.640 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-14T05:26:42.641 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:26:42.641 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-14T05:26:42.641 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:26:42.641 INFO:teuthology.orchestra.run.smithi071.stdout:Install 4 Packages 2024-09-14T05:26:42.641 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:26:42.641 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 51 M 2024-09-14T05:26:42.641 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 217 M 2024-09-14T05:26:42.642 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-14T05:26:43.102 INFO:teuthology.orchestra.run.smithi071.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-14T05:26:43.127 INFO:teuthology.orchestra.run.smithi071.stdout:(2/4): libcephsqlite-19.3.0-4775.g7f24e589.el9. 686 kB/s | 162 kB 00:00 2024-09-14T05:26:43.152 INFO:teuthology.orchestra.run.smithi071.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-14T05:26:43.728 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-14T05:26:43.728 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 1/35 2024-09-14T05:26:43.728 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 2/35 2024-09-14T05:26:43.728 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 3/35 2024-09-14T05:26:43.728 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 4/35 2024-09-14T05:26:43.729 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 5/35 2024-09-14T05:26:43.729 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_6 6/35 2024-09-14T05:26:43.729 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 7/35 2024-09-14T05:26:43.729 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4775.g7f24e589.el9. 8/35 2024-09-14T05:26:43.729 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:19.3.0-4775.g7f24e589.el9.x8 9/35 2024-09-14T05:26:43.729 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 10/35 2024-09-14T05:26:43.729 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 11/35 2024-09-14T05:26:43.729 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 12/35 2024-09-14T05:26:43.729 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 13/35 2024-09-14T05:26:43.729 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-14T05:26:43.730 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-14T05:26:43.730 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-14T05:26:43.730 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-14T05:26:43.730 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-14T05:26:43.730 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-14T05:26:43.730 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-14T05:26:43.730 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-14T05:26:43.730 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-14T05:26:43.730 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-14T05:26:43.730 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-14T05:26:43.730 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-14T05:26:43.730 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-14T05:26:43.731 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-14T05:26:43.731 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-14T05:26:43.731 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-14T05:26:43.731 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-14T05:26:43.731 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-14T05:26:43.731 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 32/35 2024-09-14T05:26:43.731 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-14T05:26:43.731 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 34/35 2024-09-14T05:26:44.469 INFO:teuthology.orchestra.run.smithi071.stdout:(4/4): ceph-test-19.3.0-4775.g7f24e589.el9.x86_ 32 MB/s | 50 MB 00:01 2024-09-14T05:26:44.471 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:26:44.471 INFO:teuthology.orchestra.run.smithi071.stdout:Total 28 MB/s | 51 MB 00:01 2024-09-14T05:26:44.499 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-14T05:26:44.519 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-14T05:26:44.520 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-14T05:26:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-14T05:26:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:26:44.582 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-09-14T05:26:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:44.582 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-14T05:26:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-14T05:26:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-14T05:26:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-14T05:26:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-14T05:26:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-14T05:26:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-14T05:26:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-14T05:26:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-14T05:26:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-14T05:26:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-14T05:26:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-14T05:26:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-14T05:26:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-14T05:26:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-14T05:26:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-14T05:26:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-14T05:26:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: re2-1:20211101-20.el9.x86_64 2024-09-14T05:26:44.585 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-14T05:26:44.585 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:26:44.585 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:26:44.921 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-14T05:26:44.921 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-14T05:26:44.978 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-test 2024-09-14T05:26:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-14T05:26:45.549 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-14T05:26:45.552 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:28 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:26:45.601 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-14T05:26:45.629 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 3/4 2024-09-14T05:26:45.685 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:26:45.685 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:26:45.685 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-14T05:26:45.685 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:26:45.685 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-14T05:26:45.686 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 50 M 2024-09-14T05:26:45.686 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-14T05:26:45.686 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 162 k 2024-09-14T05:26:45.686 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-14T05:26:45.686 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-14T05:26:45.686 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:26:45.686 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-14T05:26:45.686 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:26:45.686 INFO:teuthology.orchestra.run.smithi150.stdout:Install 4 Packages 2024-09-14T05:26:45.686 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:26:45.687 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 51 M 2024-09-14T05:26:45.687 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 217 M 2024-09-14T05:26:45.687 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-14T05:26:46.105 INFO:teuthology.orchestra.run.smithi150.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.2 MB/s | 304 kB 00:00 2024-09-14T05:26:46.172 INFO:teuthology.orchestra.run.smithi150.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 955 kB/s | 64 kB 00:00 2024-09-14T05:26:46.197 INFO:teuthology.orchestra.run.smithi150.stdout:(3/4): libcephsqlite-19.3.0-4775.g7f24e589.el9. 704 kB/s | 162 kB 00:00 2024-09-14T05:26:47.531 INFO:teuthology.orchestra.run.smithi150.stdout:(4/4): ceph-test-19.3.0-4775.g7f24e589.el9.x86_ 32 MB/s | 50 MB 00:01 2024-09-14T05:26:47.533 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:26:47.533 INFO:teuthology.orchestra.run.smithi150.stdout:Total 27 MB/s | 51 MB 00:01 2024-09-14T05:26:47.561 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-14T05:26:47.582 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-14T05:26:47.582 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-14T05:26:47.981 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-14T05:26:47.982 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-14T05:26:48.524 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-14T05:26:48.623 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-14T05:26:48.663 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-14T05:26:48.691 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 3/4 2024-09-14T05:26:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 3/4 2024-09-14T05:26:50.477 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 4/4 2024-09-14T05:26:53.048 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 4/4 2024-09-14T05:26:53.048 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 1/4 2024-09-14T05:26:53.048 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 2/4 2024-09-14T05:26:53.048 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-14T05:26:53.332 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-14T05:26:53.332 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:26:53.332 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-14T05:26:53.332 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:53.333 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:53.333 INFO:teuthology.orchestra.run.smithi071.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-14T05:26:53.333 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-14T05:26:53.333 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:26:53.333 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:26:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 3/4 2024-09-14T05:26:53.499 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 4/4 2024-09-14T05:26:53.634 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph 2024-09-14T05:26:54.216 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:00:40 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:26:54.369 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:26:54.371 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:26:54.371 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-09-14T05:26:54.371 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:26:54.372 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-14T05:26:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: ceph x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 6.6 k 2024-09-14T05:26:54.372 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-09-14T05:26:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 2.4 M 2024-09-14T05:26:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 1.5 M 2024-09-14T05:26:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 291 k 2024-09-14T05:26:54.372 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 4.8 M 2024-09-14T05:26:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 17 M 2024-09-14T05:26:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-14T05:26:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-14T05:26:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-14T05:26:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-14T05:26:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-14T05:26:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-14T05:26:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-14T05:26:54.373 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-14T05:26:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-14T05:26:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-14T05:26:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections 2024-09-14T05:26:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-14T05:26:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-14T05:26:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-14T05:26:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-14T05:26:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-14T05:26:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-14T05:26:54.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-14T05:26:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-14T05:26:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-14T05:26:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-14T05:26:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-14T05:26:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-14T05:26:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-14T05:26:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-14T05:26:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-14T05:26:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-14T05:26:54.375 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-14T05:26:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-14T05:26:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-14T05:26:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-14T05:26:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-14T05:26:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:26:54.376 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-14T05:26:54.376 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:26:54.376 INFO:teuthology.orchestra.run.smithi071.stdout:Install 37 Packages 2024-09-14T05:26:54.376 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:26:54.377 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 31 M 2024-09-14T05:26:54.377 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 109 M 2024-09-14T05:26:54.377 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-14T05:26:55.156 INFO:teuthology.orchestra.run.smithi071.stdout:(1/37): ceph-19.3.0-4775.g7f24e589.el9.x86_64.r 52 kB/s | 6.6 kB 00:00 2024-09-14T05:26:55.372 INFO:teuthology.orchestra.run.smithi071.stdout:(2/37): ceph-mds-19.3.0-4775.g7f24e589.el9.x86_ 6.9 MB/s | 2.4 MB 00:00 2024-09-14T05:26:55.406 INFO:teuthology.orchestra.run.smithi071.stdout:(3/37): ceph-mgr-19.3.0-4775.g7f24e589.el9.x86_ 4.0 MB/s | 1.5 MB 00:00 2024-09-14T05:26:55.490 INFO:teuthology.orchestra.run.smithi071.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4775.g7f24 3.4 MB/s | 291 kB 00:00 2024-09-14T05:26:55.657 INFO:teuthology.orchestra.run.smithi071.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-09-14T05:26:55.807 INFO:teuthology.orchestra.run.smithi071.stdout:(6/37): ceph-mon-19.3.0-4775.g7f24e589.el9.x86_ 7.4 MB/s | 4.8 MB 00:00 2024-09-14T05:26:55.840 INFO:teuthology.orchestra.run.smithi071.stdout:(7/37): python3-cryptography-36.0.1-4.el9.x86_6 6.7 MB/s | 1.2 MB 00:00 2024-09-14T05:26:55.945 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 4/4 2024-09-14T05:26:55.945 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 1/4 2024-09-14T05:26:55.945 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 2/4 2024-09-14T05:26:55.946 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-14T05:26:56.008 INFO:teuthology.orchestra.run.smithi071.stdout:(8/37): ceph-osd-19.3.0-4775.g7f24e589.el9.x86_ 27 MB/s | 17 MB 00:00 2024-09-14T05:26:56.033 INFO:teuthology.orchestra.run.smithi071.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 704 kB/s | 135 kB 00:00 2024-09-14T05:26:56.058 INFO:teuthology.orchestra.run.smithi071.stdout:(10/37): python3-ply-3.11-14.el9.noarch.rpm 425 kB/s | 106 kB 00:00 2024-09-14T05:26:56.083 INFO:teuthology.orchestra.run.smithi071.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-09-14T05:26:56.150 INFO:teuthology.orchestra.run.smithi071.stdout:(12/37): python3-requests-2.25.1-8.el9.noarch.r 878 kB/s | 125 kB 00:00 2024-09-14T05:26:56.251 INFO:teuthology.orchestra.run.smithi071.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 981 kB/s | 188 kB 00:00 2024-09-14T05:26:56.260 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-14T05:26:56.260 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:26:56.260 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-14T05:26:56.260 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:56.260 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:26:56.260 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-14T05:26:56.260 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-14T05:26:56.261 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:26:56.261 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:26:56.276 INFO:teuthology.orchestra.run.smithi071.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 278 kB/s | 35 kB 00:00 2024-09-14T05:26:56.301 INFO:teuthology.orchestra.run.smithi071.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 792 kB/s | 172 kB 00:00 2024-09-14T05:26:56.326 INFO:teuthology.orchestra.run.smithi071.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 1.0 MB/s | 77 kB 00:00 2024-09-14T05:26:56.351 INFO:teuthology.orchestra.run.smithi071.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-09-14T05:26:56.385 INFO:teuthology.orchestra.run.smithi071.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 496 kB/s | 42 kB 00:00 2024-09-14T05:26:56.410 INFO:teuthology.orchestra.run.smithi071.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-09-14T05:26:56.452 INFO:teuthology.orchestra.run.smithi071.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-09-14T05:26:56.478 INFO:teuthology.orchestra.run.smithi071.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 481 kB/s | 43 kB 00:00 2024-09-14T05:26:56.503 INFO:teuthology.orchestra.run.smithi071.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 427 kB/s | 11 kB 00:00 2024-09-14T05:26:56.537 INFO:teuthology.orchestra.run.smithi071.stdout:(23/37): python3-cherrypy-18.6.1-2.el9.noarch.r 4.3 MB/s | 358 kB 00:00 2024-09-14T05:26:56.543 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph 2024-09-14T05:26:56.562 INFO:teuthology.orchestra.run.smithi071.stdout:(24/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-14T05:26:56.587 INFO:teuthology.orchestra.run.smithi071.stdout:(25/37): python3-jaraco-classes-3.2.1-5.el9.noa 211 kB/s | 18 kB 00:00 2024-09-14T05:26:56.612 INFO:teuthology.orchestra.run.smithi071.stdout:(26/37): python3-jaraco-collections-3.0.0-8.el9 307 kB/s | 23 kB 00:00 2024-09-14T05:26:56.637 INFO:teuthology.orchestra.run.smithi071.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-09-14T05:26:56.663 INFO:teuthology.orchestra.run.smithi071.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 260 kB/s | 20 kB 00:00 2024-09-14T05:26:56.688 INFO:teuthology.orchestra.run.smithi071.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-09-14T05:26:56.713 INFO:teuthology.orchestra.run.smithi071.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-14T05:26:56.739 INFO:teuthology.orchestra.run.smithi071.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-14T05:26:56.764 INFO:teuthology.orchestra.run.smithi071.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-14T05:26:56.789 INFO:teuthology.orchestra.run.smithi071.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 714 kB/s | 36 kB 00:00 2024-09-14T05:26:56.814 INFO:teuthology.orchestra.run.smithi071.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-14T05:26:56.840 INFO:teuthology.orchestra.run.smithi071.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 802 kB/s | 20 kB 00:00 2024-09-14T05:26:56.865 INFO:teuthology.orchestra.run.smithi071.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-14T05:26:56.898 INFO:teuthology.orchestra.run.smithi071.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-14T05:26:56.905 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:26:56.905 INFO:teuthology.orchestra.run.smithi071.stdout:Total 12 MB/s | 31 MB 00:02 2024-09-14T05:26:57.056 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-14T05:26:57.095 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-14T05:26:57.095 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-14T05:26:57.139 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:40 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:26:57.282 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:26:57.285 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:26:57.285 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-14T05:26:57.285 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:26:57.285 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-14T05:26:57.285 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 6.6 k 2024-09-14T05:26:57.285 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-14T05:26:57.285 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 2.4 M 2024-09-14T05:26:57.285 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 1.5 M 2024-09-14T05:26:57.285 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 291 k 2024-09-14T05:26:57.285 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 4.8 M 2024-09-14T05:26:57.285 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 17 M 2024-09-14T05:26:57.285 INFO:teuthology.orchestra.run.smithi150.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-14T05:26:57.286 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-14T05:26:57.286 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-14T05:26:57.286 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-14T05:26:57.286 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-14T05:26:57.286 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-14T05:26:57.288 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-14T05:26:57.288 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-14T05:26:57.288 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-14T05:26:57.288 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-14T05:26:57.288 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections 2024-09-14T05:26:57.288 INFO:teuthology.orchestra.run.smithi150.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-14T05:26:57.289 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-14T05:26:57.289 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-14T05:26:57.289 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-14T05:26:57.289 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-14T05:26:57.289 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-14T05:26:57.289 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-14T05:26:57.289 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-14T05:26:57.289 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-14T05:26:57.289 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-14T05:26:57.289 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-14T05:26:57.289 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-14T05:26:57.289 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-14T05:26:57.289 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-14T05:26:57.289 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-14T05:26:57.290 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-14T05:26:57.290 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-14T05:26:57.290 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-14T05:26:57.290 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-14T05:26:57.290 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-14T05:26:57.290 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-14T05:26:57.290 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:26:57.290 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-14T05:26:57.290 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:26:57.290 INFO:teuthology.orchestra.run.smithi150.stdout:Install 37 Packages 2024-09-14T05:26:57.290 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:26:57.291 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 31 M 2024-09-14T05:26:57.291 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 109 M 2024-09-14T05:26:57.291 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-14T05:26:57.573 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-14T05:26:57.573 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-14T05:26:57.715 INFO:teuthology.orchestra.run.smithi150.stdout:(1/37): ceph-19.3.0-4775.g7f24e589.el9.x86_64.r 45 kB/s | 6.6 kB 00:00 2024-09-14T05:26:57.890 INFO:teuthology.orchestra.run.smithi150.stdout:(2/37): ceph-mgr-19.3.0-4775.g7f24e589.el9.x86_ 4.8 MB/s | 1.5 MB 00:00 2024-09-14T05:26:57.932 INFO:teuthology.orchestra.run.smithi150.stdout:(3/37): ceph-mds-19.3.0-4775.g7f24e589.el9.x86_ 6.6 MB/s | 2.4 MB 00:00 2024-09-14T05:26:57.999 INFO:teuthology.orchestra.run.smithi150.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4775.g7f24 4.3 MB/s | 291 kB 00:00 2024-09-14T05:26:58.133 INFO:teuthology.orchestra.run.smithi150.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-09-14T05:26:58.408 INFO:teuthology.orchestra.run.smithi150.stdout:(6/37): ceph-osd-19.3.0-4775.g7f24e589.el9.x86_ 33 MB/s | 17 MB 00:00 2024-09-14T05:26:58.450 INFO:teuthology.orchestra.run.smithi150.stdout:(7/37): python3-cryptography-36.0.1-4.el9.x86_6 3.9 MB/s | 1.2 MB 00:00 2024-09-14T05:26:58.517 INFO:teuthology.orchestra.run.smithi150.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-09-14T05:26:58.542 INFO:teuthology.orchestra.run.smithi150.stdout:(9/37): python3-ply-3.11-14.el9.noarch.rpm 797 kB/s | 106 kB 00:00 2024-09-14T05:26:58.567 INFO:teuthology.orchestra.run.smithi150.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-09-14T05:26:58.610 INFO:teuthology.orchestra.run.smithi150.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.1 MB/s | 215 kB 00:00 2024-09-14T05:26:58.668 INFO:teuthology.orchestra.run.smithi150.stdout:(12/37): ceph-mon-19.3.0-4775.g7f24e589.el9.x86 5.0 MB/s | 4.8 MB 00:00 2024-09-14T05:26:58.718 INFO:teuthology.orchestra.run.smithi150.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-09-14T05:26:58.760 INFO:teuthology.orchestra.run.smithi150.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 378 kB/s | 35 kB 00:00 2024-09-14T05:26:58.794 INFO:teuthology.orchestra.run.smithi150.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 936 kB/s | 172 kB 00:00 2024-09-14T05:26:58.819 INFO:teuthology.orchestra.run.smithi150.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 766 kB/s | 77 kB 00:00 2024-09-14T05:26:58.844 INFO:teuthology.orchestra.run.smithi150.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 607 kB/s | 51 kB 00:00 2024-09-14T05:26:58.870 INFO:teuthology.orchestra.run.smithi150.stdout:(18/37): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-09-14T05:26:58.895 INFO:teuthology.orchestra.run.smithi150.stdout:(19/37): python3-toml-0.10.2-6.el9.noarch.rpm 413 kB/s | 42 kB 00:00 2024-09-14T05:26:58.941 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-14T05:26:58.954 INFO:teuthology.orchestra.run.smithi150.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 520 kB/s | 43 kB 00:00 2024-09-14T05:26:58.979 INFO:teuthology.orchestra.run.smithi150.stdout:(21/37): luarocks-3.9.2-5.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-09-14T05:26:59.004 INFO:teuthology.orchestra.run.smithi150.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 430 kB/s | 11 kB 00:00 2024-09-14T05:26:59.018 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-14T05:26:59.029 INFO:teuthology.orchestra.run.smithi150.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 712 kB/s | 18 kB 00:00 2024-09-14T05:26:59.055 INFO:teuthology.orchestra.run.smithi150.stdout:(24/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-14T05:26:59.059 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-14T05:26:59.080 INFO:teuthology.orchestra.run.smithi150.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 457 kB/s | 23 kB 00:00 2024-09-14T05:26:59.104 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-14T05:26:59.105 INFO:teuthology.orchestra.run.smithi150.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-14T05:26:59.131 INFO:teuthology.orchestra.run.smithi150.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-09-14T05:26:59.156 INFO:teuthology.orchestra.run.smithi150.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-09-14T05:26:59.181 INFO:teuthology.orchestra.run.smithi150.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.5 MB/s | 358 kB 00:00 2024-09-14T05:26:59.182 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-14T05:26:59.206 INFO:teuthology.orchestra.run.smithi150.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-14T05:26:59.231 INFO:teuthology.orchestra.run.smithi150.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-14T05:26:59.235 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-14T05:26:59.257 INFO:teuthology.orchestra.run.smithi150.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-14T05:26:59.276 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-14T05:26:59.277 INFO:teuthology.orchestra.run.smithi150.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 796 kB/s | 36 kB 00:00 2024-09-14T05:26:59.302 INFO:teuthology.orchestra.run.smithi150.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-14T05:26:59.319 INFO:teuthology.orchestra.run.smithi150.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 1.2 MB/s | 20 kB 00:00 2024-09-14T05:26:59.353 INFO:teuthology.orchestra.run.smithi150.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-14T05:26:59.376 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-14T05:26:59.386 INFO:teuthology.orchestra.run.smithi150.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-14T05:26:59.391 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:26:59.391 INFO:teuthology.orchestra.run.smithi150.stdout:Total 15 MB/s | 31 MB 00:02 2024-09-14T05:26:59.463 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-14T05:26:59.504 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-14T05:26:59.539 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-14T05:26:59.543 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-14T05:26:59.569 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-14T05:26:59.583 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-14T05:26:59.583 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-14T05:26:59.606 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-14T05:26:59.637 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-14T05:26:59.677 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-14T05:26:59.712 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-14T05:26:59.747 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-14T05:26:59.795 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-14T05:26:59.835 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-14T05:26:59.907 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-14T05:27:00.046 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-14T05:27:00.059 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-14T05:27:00.059 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-14T05:27:00.127 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-14T05:27:00.188 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-14T05:27:00.226 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-14T05:27:00.275 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-14T05:27:00.332 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-14T05:27:00.526 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-14T05:27:00.580 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-14T05:27:00.659 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-14T05:27:00.746 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-14T05:27:00.779 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-14T05:27:00.906 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-14T05:27:01.095 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4775.g7f24e589.el9. 32/37 2024-09-14T05:27:01.123 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 33/37 2024-09-14T05:27:01.150 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 33/37 2024-09-14T05:27:01.150 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T05:27:01.151 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-14T05:27:01.151 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T05:27:01.151 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T05:27:01.151 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:01.504 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-14T05:27:01.573 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-14T05:27:01.605 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-14T05:27:01.633 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-14T05:27:01.707 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-14T05:27:01.782 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-14T05:27:01.815 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-14T05:27:01.906 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-14T05:27:01.981 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-14T05:27:02.017 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-14T05:27:02.053 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-14T05:27:02.082 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-14T05:27:02.117 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-14T05:27:02.150 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-14T05:27:02.191 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-14T05:27:02.226 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-14T05:27:02.260 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-14T05:27:02.309 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-14T05:27:02.340 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-14T05:27:02.414 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-14T05:27:02.554 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-14T05:27:02.616 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 34/37 2024-09-14T05:27:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-14T05:27:02.643 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 34/37 2024-09-14T05:27:02.643 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T05:27:02.644 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-14T05:27:02.644 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T05:27:02.644 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T05:27:02.644 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:02.693 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-14T05:27:02.740 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-14T05:27:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-14T05:27:02.852 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-14T05:27:03.049 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-14T05:27:03.100 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 35/37 2024-09-14T05:27:03.102 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-14T05:27:03.123 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 35/37 2024-09-14T05:27:03.123 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T05:27:03.123 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-14T05:27:03.123 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T05:27:03.123 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T05:27:03.123 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:03.180 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-14T05:27:03.269 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-14T05:27:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-14T05:27:03.350 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 36/37 2024-09-14T05:27:03.373 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 36/37 2024-09-14T05:27:03.373 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T05:27:03.373 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-14T05:27:03.373 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T05:27:03.374 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T05:27:03.374 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:03.438 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 37/37 2024-09-14T05:27:03.440 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-14T05:27:03.600 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4775.g7f24e589.el9. 32/37 2024-09-14T05:27:03.659 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 33/37 2024-09-14T05:27:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 33/37 2024-09-14T05:27:03.686 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T05:27:03.686 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-14T05:27:03.686 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T05:27:03.686 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T05:27:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:05.154 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 34/37 2024-09-14T05:27:05.181 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 34/37 2024-09-14T05:27:05.181 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T05:27:05.182 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-14T05:27:05.182 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T05:27:05.182 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T05:27:05.182 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:05.360 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 37/37 2024-09-14T05:27:05.361 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 1/37 2024-09-14T05:27:05.361 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 2/37 2024-09-14T05:27:05.361 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 3/37 2024-09-14T05:27:05.361 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 4/37 2024-09-14T05:27:05.361 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 5/37 2024-09-14T05:27:05.361 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4775.g7f24e589.el9. 6/37 2024-09-14T05:27:05.361 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-14T05:27:05.361 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-14T05:27:05.361 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-14T05:27:05.361 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-14T05:27:05.361 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-14T05:27:05.361 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-14T05:27:05.361 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-14T05:27:05.362 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-14T05:27:05.362 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-14T05:27:05.362 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-14T05:27:05.362 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-14T05:27:05.362 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-14T05:27:05.362 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-14T05:27:05.362 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-14T05:27:05.362 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-14T05:27:05.362 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-14T05:27:05.362 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-14T05:27:05.362 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-14T05:27:05.362 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-14T05:27:05.362 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-14T05:27:05.362 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-14T05:27:05.363 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-14T05:27:05.363 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-14T05:27:05.363 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-14T05:27:05.363 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-14T05:27:05.363 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-14T05:27:05.363 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-14T05:27:05.363 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-14T05:27:05.363 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-14T05:27:05.363 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-14T05:27:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 35/37 2024-09-14T05:27:05.812 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 35/37 2024-09-14T05:27:05.812 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T05:27:05.812 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-14T05:27:05.813 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T05:27:05.813 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T05:27:05.813 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:06.056 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 36/37 2024-09-14T05:27:06.079 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 36/37 2024-09-14T05:27:06.079 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T05:27:06.079 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-14T05:27:06.079 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T05:27:06.079 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T05:27:06.079 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:06.153 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 37/37 2024-09-14T05:27:06.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-14T05:27:06.469 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:06.469 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-14T05:27:06.469 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:27:06.469 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:27:06.469 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:27:06.469 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T05:27:06.469 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:27:06.470 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:27:06.470 INFO:teuthology.orchestra.run.smithi071.stdout: lua-5.4.4-4.el9.x86_64 2024-09-14T05:27:06.470 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-14T05:27:06.470 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-14T05:27:06.470 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-14T05:27:06.470 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-14T05:27:06.470 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-14T05:27:06.470 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-14T05:27:06.470 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-14T05:27:06.470 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-14T05:27:06.470 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-14T05:27:06.470 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-14T05:27:06.471 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-14T05:27:06.471 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-14T05:27:06.471 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-14T05:27:06.471 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-14T05:27:06.471 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-14T05:27:06.471 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-14T05:27:06.471 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-14T05:27:06.471 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-14T05:27:06.471 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply-3.11-14.el9.noarch 2024-09-14T05:27:06.471 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-14T05:27:06.471 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-14T05:27:06.471 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-14T05:27:06.471 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-14T05:27:06.471 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-14T05:27:06.472 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-14T05:27:06.472 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-14T05:27:06.472 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-14T05:27:06.472 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-14T05:27:06.472 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-14T05:27:06.472 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-14T05:27:06.472 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:06.472 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:27:06.837 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-base 2024-09-14T05:27:07.409 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:00:53 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:27:07.499 INFO:teuthology.orchestra.run.smithi071.stdout:Package ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T05:27:07.542 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:27:07.543 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-09-14T05:27:07.543 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:27:07.614 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install cephadm 2024-09-14T05:27:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 37/37 2024-09-14T05:27:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 1/37 2024-09-14T05:27:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 2/37 2024-09-14T05:27:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 3/37 2024-09-14T05:27:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 4/37 2024-09-14T05:27:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 5/37 2024-09-14T05:27:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4775.g7f24e589.el9. 6/37 2024-09-14T05:27:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-14T05:27:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-14T05:27:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-14T05:27:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-14T05:27:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-14T05:27:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-14T05:27:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-14T05:27:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-14T05:27:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-14T05:27:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-14T05:27:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-14T05:27:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-14T05:27:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-14T05:27:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-14T05:27:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-14T05:27:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-14T05:27:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-14T05:27:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-14T05:27:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-14T05:27:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-14T05:27:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-14T05:27:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-14T05:27:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-14T05:27:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-14T05:27:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-14T05:27:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-14T05:27:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-14T05:27:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-14T05:27:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-14T05:27:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-14T05:27:08.173 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:00:54 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:27:08.305 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:27:08.306 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:08.306 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-09-14T05:27:08.307 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:08.307 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-14T05:27:08.307 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 783 k 2024-09-14T05:27:08.307 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:08.307 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-14T05:27:08.307 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:08.307 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-09-14T05:27:08.307 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:08.307 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 783 k 2024-09-14T05:27:08.307 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 789 k 2024-09-14T05:27:08.307 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-14T05:27:08.573 INFO:teuthology.orchestra.run.smithi071.stdout:cephadm-19.3.0-4775.g7f24e589.el9.noarch.rpm 2.9 MB/s | 783 kB 00:00 2024-09-14T05:27:08.573 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:08.574 INFO:teuthology.orchestra.run.smithi071.stdout:Total 2.9 MB/s | 783 kB 00:00 2024-09-14T05:27:08.574 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-14T05:27:08.579 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-14T05:27:08.579 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-14T05:27:08.585 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-14T05:27:08.585 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-14T05:27:08.655 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-14T05:27:08.707 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-14T05:27:08.707 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:08.707 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-14T05:27:08.707 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:27:08.708 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:27:08.708 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:27:08.708 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T05:27:08.708 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:27:08.708 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:27:08.708 INFO:teuthology.orchestra.run.smithi150.stdout: lua-5.4.4-4.el9.x86_64 2024-09-14T05:27:08.709 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-14T05:27:08.713 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-14T05:27:08.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-14T05:27:08.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-14T05:27:08.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-14T05:27:08.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-14T05:27:08.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-14T05:27:08.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-14T05:27:08.715 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-14T05:27:08.715 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-14T05:27:08.715 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-14T05:27:08.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-14T05:27:08.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-14T05:27:08.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-14T05:27:08.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-14T05:27:08.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-14T05:27:08.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-14T05:27:08.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-14T05:27:08.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply-3.11-14.el9.noarch 2024-09-14T05:27:08.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-14T05:27:08.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-14T05:27:08.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-14T05:27:08.718 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-14T05:27:08.718 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-14T05:27:08.718 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-14T05:27:08.718 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-14T05:27:08.718 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-14T05:27:08.718 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-14T05:27:08.718 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-14T05:27:08.719 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-14T05:27:08.719 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:08.719 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:27:09.089 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T05:27:09.117 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-base 2024-09-14T05:27:09.184 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T05:27:09.537 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T05:27:09.690 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:52 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:27:09.781 INFO:teuthology.orchestra.run.smithi150.stdout:Package ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T05:27:09.782 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T05:27:09.782 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:09.782 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-14T05:27:09.782 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T05:27:09.782 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:09.782 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:27:09.823 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:27:09.824 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-14T05:27:09.824 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:27:09.899 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install cephadm 2024-09-14T05:27:09.962 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-immutable-object-cache 2024-09-14T05:27:10.457 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:53 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:27:10.529 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:00:56 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:27:10.589 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:27:10.590 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:10.590 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-14T05:27:10.590 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:10.591 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-14T05:27:10.591 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 783 k 2024-09-14T05:27:10.591 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:10.591 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-14T05:27:10.591 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:10.591 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-14T05:27:10.591 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:10.591 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 783 k 2024-09-14T05:27:10.592 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 789 k 2024-09-14T05:27:10.592 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-14T05:27:10.664 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:27:10.664 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:10.664 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repo Size 2024-09-14T05:27:10.665 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:10.665 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-14T05:27:10.665 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 145 k 2024-09-14T05:27:10.665 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:10.665 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-14T05:27:10.665 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:10.665 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-09-14T05:27:10.665 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:10.665 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 145 k 2024-09-14T05:27:10.665 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 432 k 2024-09-14T05:27:10.665 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-14T05:27:10.854 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-immutable-object-cache-19.3.0-4775.g7f24e5 766 kB/s | 145 kB 00:00 2024-09-14T05:27:10.855 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:10.855 INFO:teuthology.orchestra.run.smithi071.stdout:Total 761 kB/s | 145 kB 00:00 2024-09-14T05:27:10.855 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-14T05:27:10.862 INFO:teuthology.orchestra.run.smithi150.stdout:cephadm-19.3.0-4775.g7f24e589.el9.noarch.rpm 2.8 MB/s | 783 kB 00:00 2024-09-14T05:27:10.863 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-14T05:27:10.863 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-14T05:27:10.863 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:10.863 INFO:teuthology.orchestra.run.smithi150.stdout:Total 2.8 MB/s | 783 kB 00:00 2024-09-14T05:27:10.863 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-14T05:27:10.868 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-14T05:27:10.868 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-14T05:27:10.874 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-14T05:27:10.875 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-14T05:27:10.919 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-14T05:27:10.920 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-14T05:27:10.950 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-14T05:27:11.073 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-14T05:27:11.148 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589. 1/1 2024-09-14T05:27:11.172 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589. 1/1 2024-09-14T05:27:11.188 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T05:27:11.188 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-14T05:27:11.188 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T05:27:11.524 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T05:27:11.883 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T05:27:12.143 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T05:27:12.143 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:12.143 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-14T05:27:12.143 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T05:27:12.143 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:12.143 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:27:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589. 1/1 2024-09-14T05:27:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:12.155 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-14T05:27:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:27:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:12.155 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:27:12.340 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-immutable-object-cache 2024-09-14T05:27:12.407 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr 2024-09-14T05:27:12.921 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:55 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:27:12.970 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:00:58 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:27:13.058 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:27:13.058 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:13.058 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repo Size 2024-09-14T05:27:13.058 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:13.058 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-14T05:27:13.058 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 145 k 2024-09-14T05:27:13.058 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:13.059 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-14T05:27:13.059 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:13.059 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-14T05:27:13.059 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:13.059 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 145 k 2024-09-14T05:27:13.059 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 432 k 2024-09-14T05:27:13.059 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-14T05:27:13.063 INFO:teuthology.orchestra.run.smithi071.stdout:Package ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T05:27:13.107 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:27:13.108 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-09-14T05:27:13.108 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:27:13.181 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr-dashboard 2024-09-14T05:27:13.251 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-immutable-object-cache-19.3.0-4775.g7f24e5 757 kB/s | 145 kB 00:00 2024-09-14T05:27:13.251 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:13.252 INFO:teuthology.orchestra.run.smithi150.stdout:Total 751 kB/s | 145 kB 00:00 2024-09-14T05:27:13.252 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-14T05:27:13.259 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-14T05:27:13.259 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-14T05:27:13.316 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-14T05:27:13.316 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-14T05:27:13.471 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-14T05:27:13.552 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589. 1/1 2024-09-14T05:27:13.577 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589. 1/1 2024-09-14T05:27:13.578 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T05:27:13.578 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-14T05:27:13.578 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:13.744 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:00:59 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:27:13.877 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:27:13.878 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:13.878 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-09-14T05:27:13.878 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:13.878 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-14T05:27:13.879 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 14 M 2024-09-14T05:27:13.879 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-09-14T05:27:13.879 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-14T05:27:13.879 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 34 k 2024-09-14T05:27:13.879 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 16 k 2024-09-14T05:27:13.879 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-14T05:27:13.879 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-14T05:27:13.879 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-14T05:27:13.879 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-14T05:27:13.879 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-14T05:27:13.879 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-14T05:27:13.880 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-14T05:27:13.880 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-14T05:27:13.880 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:13.880 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-14T05:27:13.880 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:13.880 INFO:teuthology.orchestra.run.smithi071.stdout:Install 12 Packages 2024-09-14T05:27:13.880 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:13.881 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 19 M 2024-09-14T05:27:13.881 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 159 M 2024-09-14T05:27:13.881 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-14T05:27:14.174 INFO:teuthology.orchestra.run.smithi071.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4775.g7f 246 kB/s | 34 kB 00:00 2024-09-14T05:27:14.199 INFO:teuthology.orchestra.run.smithi071.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4775.g7f2 100 kB/s | 16 kB 00:00 2024-09-14T05:27:14.325 INFO:teuthology.orchestra.run.smithi071.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.1 MB/s | 269 kB 00:00 2024-09-14T05:27:14.433 INFO:teuthology.orchestra.run.smithi071.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-09-14T05:27:14.484 INFO:teuthology.orchestra.run.smithi071.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-14T05:27:14.500 INFO:teuthology.orchestra.run.smithi071.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-09-14T05:27:14.507 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589. 1/1 2024-09-14T05:27:14.508 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:14.508 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-14T05:27:14.508 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:27:14.508 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:14.508 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:27:14.593 INFO:teuthology.orchestra.run.smithi071.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 22 MB/s | 2.0 MB 00:00 2024-09-14T05:27:14.617 INFO:teuthology.orchestra.run.smithi071.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-14T05:27:14.635 INFO:teuthology.orchestra.run.smithi071.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.8 MB/s | 31 kB 00:00 2024-09-14T05:27:14.660 INFO:teuthology.orchestra.run.smithi071.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.4 MB/s | 188 kB 00:00 2024-09-14T05:27:14.721 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr 2024-09-14T05:27:15.077 INFO:teuthology.orchestra.run.smithi071.stdout:(11/12): protobuf-compiler-3.14.0-13.el9.x86_64 1.1 MB/s | 863 kB 00:00 2024-09-14T05:27:15.302 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:58 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:27:15.395 INFO:teuthology.orchestra.run.smithi150.stdout:Package ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T05:27:15.437 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:27:15.438 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-14T05:27:15.438 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:27:15.509 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-dashboard 2024-09-14T05:27:16.071 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:59 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:27:16.203 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:27:16.205 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:16.205 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-14T05:27:16.205 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:16.205 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-14T05:27:16.205 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 14 M 2024-09-14T05:27:16.205 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-14T05:27:16.205 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-14T05:27:16.205 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 34 k 2024-09-14T05:27:16.205 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 16 k 2024-09-14T05:27:16.206 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-14T05:27:16.206 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-14T05:27:16.206 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-14T05:27:16.206 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-14T05:27:16.206 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-14T05:27:16.206 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-14T05:27:16.206 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-14T05:27:16.206 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-14T05:27:16.206 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:16.206 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-14T05:27:16.206 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:16.206 INFO:teuthology.orchestra.run.smithi150.stdout:Install 12 Packages 2024-09-14T05:27:16.206 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:16.207 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 19 M 2024-09-14T05:27:16.207 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 159 M 2024-09-14T05:27:16.207 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-14T05:27:16.573 INFO:teuthology.orchestra.run.smithi150.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4775.g7f 214 kB/s | 34 kB 00:00 2024-09-14T05:27:16.598 INFO:teuthology.orchestra.run.smithi150.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4775.g7f2 89 kB/s | 16 kB 00:00 2024-09-14T05:27:16.799 INFO:teuthology.orchestra.run.smithi150.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.3 MB/s | 269 kB 00:00 2024-09-14T05:27:16.857 INFO:teuthology.orchestra.run.smithi150.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.6 MB/s | 1.0 MB 00:00 2024-09-14T05:27:16.907 INFO:teuthology.orchestra.run.smithi150.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-14T05:27:16.941 INFO:teuthology.orchestra.run.smithi150.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 578 kB/s | 19 kB 00:00 2024-09-14T05:27:17.008 INFO:teuthology.orchestra.run.smithi150.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 31 MB/s | 2.0 MB 00:00 2024-09-14T05:27:17.033 INFO:teuthology.orchestra.run.smithi150.stdout:(8/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.6 MB/s | 863 kB 00:00 2024-09-14T05:27:17.067 INFO:teuthology.orchestra.run.smithi150.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.4 MB/s | 144 kB 00:00 2024-09-14T05:27:17.077 INFO:teuthology.orchestra.run.smithi071.stdout:(12/12): ceph-mgr-dashboard-19.3.0-4775.g7f24e5 4.7 MB/s | 14 MB 00:03 2024-09-14T05:27:17.081 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:17.081 INFO:teuthology.orchestra.run.smithi071.stdout:Total 6.0 MB/s | 19 MB 00:03 2024-09-14T05:27:17.092 INFO:teuthology.orchestra.run.smithi150.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 524 kB/s | 31 kB 00:00 2024-09-14T05:27:17.117 INFO:teuthology.orchestra.run.smithi150.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.8 MB/s | 188 kB 00:00 2024-09-14T05:27:17.134 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-14T05:27:17.154 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-14T05:27:17.154 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-14T05:27:17.268 INFO:teuthology.orchestra.run.smithi150.stdout:(12/12): ceph-mgr-dashboard-19.3.0-4775.g7f24e5 17 MB/s | 14 MB 00:00 2024-09-14T05:27:17.272 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:17.272 INFO:teuthology.orchestra.run.smithi150.stdout:Total 18 MB/s | 19 MB 00:01 2024-09-14T05:27:17.320 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-14T05:27:17.339 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-14T05:27:17.339 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-14T05:27:17.427 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-14T05:27:17.427 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-14T05:27:17.606 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-14T05:27:17.607 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-14T05:27:18.026 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-14T05:27:18.102 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-14T05:27:18.145 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-14T05:27:18.191 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-14T05:27:18.206 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-14T05:27:18.231 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-14T05:27:18.285 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-14T05:27:18.320 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-14T05:27:18.328 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-14T05:27:18.367 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-14T05:27:18.397 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-14T05:27:18.457 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-14T05:27:18.503 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-14T05:27:18.551 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-14T05:27:18.602 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-14T05:27:18.631 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-14T05:27:18.631 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-14T05:27:18.663 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4775.g7f24e589.el9 10/12 2024-09-14T05:27:18.716 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-14T05:27:18.777 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-14T05:27:18.814 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-14T05:27:18.846 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4775.g7f24e589.el9 10/12 2024-09-14T05:27:21.854 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4775.g7f24e589.el 11/12 2024-09-14T05:27:21.917 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noa 12/12 2024-09-14T05:27:22.076 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4775.g7f24e589.el 11/12 2024-09-14T05:27:22.125 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noa 12/12 2024-09-14T05:27:23.905 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noa 12/12 2024-09-14T05:27:23.905 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4775.g7f24e589.el 1/12 2024-09-14T05:27:23.905 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noa 2/12 2024-09-14T05:27:23.905 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4775.g7f24e589.el9 3/12 2024-09-14T05:27:23.906 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-14T05:27:23.906 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-14T05:27:23.906 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-14T05:27:23.906 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-14T05:27:23.906 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-14T05:27:23.906 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-14T05:27:23.906 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-14T05:27:23.906 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-14T05:27:24.082 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noa 12/12 2024-09-14T05:27:24.082 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4775.g7f24e589.el 1/12 2024-09-14T05:27:24.082 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noa 2/12 2024-09-14T05:27:24.082 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4775.g7f24e589.el9 3/12 2024-09-14T05:27:24.082 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-14T05:27:24.082 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-14T05:27:24.082 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-14T05:27:24.082 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-14T05:27:24.082 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-14T05:27:24.082 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-14T05:27:24.082 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-14T05:27:24.083 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-14T05:27:24.227 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-14T05:27:24.227 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:24.227 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-14T05:27:24.227 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-14T05:27:24.227 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T05:27:24.227 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T05:27:24.227 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T05:27:24.228 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-14T05:27:24.228 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-14T05:27:24.228 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-14T05:27:24.228 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-14T05:27:24.228 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-14T05:27:24.228 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-14T05:27:24.228 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-14T05:27:24.228 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-14T05:27:24.228 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:24.228 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:27:24.417 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-14T05:27:24.417 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:24.417 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-14T05:27:24.417 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-14T05:27:24.417 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T05:27:24.417 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T05:27:24.417 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T05:27:24.417 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-14T05:27:24.417 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-14T05:27:24.417 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-14T05:27:24.417 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-14T05:27:24.418 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-14T05:27:24.418 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-14T05:27:24.418 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-14T05:27:24.418 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-14T05:27:24.418 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:24.418 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:27:24.495 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-14T05:27:24.659 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-14T05:27:25.059 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:11 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:27:25.195 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:27:25.196 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-09-14T05:27:25.196 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-09-14T05:27:25.197 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-09-14T05:27:25.197 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-14T05:27:25.197 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 7.4 M 2024-09-14T05:27:25.197 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-09-14T05:27:25.197 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-14T05:27:25.197 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-14T05:27:25.197 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-14T05:27:25.197 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-14T05:27:25.197 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-14T05:27:25.197 INFO:teuthology.orchestra.run.smithi071.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-14T05:27:25.197 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-14T05:27:25.197 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-14T05:27:25.197 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-14T05:27:25.198 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-14T05:27:25.198 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-14T05:27:25.198 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:25.198 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-14T05:27:25.198 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-09-14T05:27:25.198 INFO:teuthology.orchestra.run.smithi071.stdout:Install 12 Packages 2024-09-14T05:27:25.198 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:25.199 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 38 M 2024-09-14T05:27:25.199 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 212 M 2024-09-14T05:27:25.199 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-14T05:27:25.230 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:08 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:27:25.373 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:27:25.374 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-09-14T05:27:25.374 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-14T05:27:25.374 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-09-14T05:27:25.374 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-14T05:27:25.374 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 7.4 M 2024-09-14T05:27:25.374 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-14T05:27:25.374 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-14T05:27:25.374 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-14T05:27:25.374 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-14T05:27:25.374 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-14T05:27:25.374 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-14T05:27:25.375 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-14T05:27:25.375 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-14T05:27:25.375 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-14T05:27:25.375 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-14T05:27:25.375 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-14T05:27:25.375 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-14T05:27:25.375 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:25.375 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-14T05:27:25.375 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-09-14T05:27:25.375 INFO:teuthology.orchestra.run.smithi150.stdout:Install 12 Packages 2024-09-14T05:27:25.375 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:25.376 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 38 M 2024-09-14T05:27:25.376 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 212 M 2024-09-14T05:27:25.376 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-14T05:27:25.466 INFO:teuthology.orchestra.run.smithi071.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-14T05:27:25.507 INFO:teuthology.orchestra.run.smithi071.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 801 kB 00:00 2024-09-14T05:27:25.717 INFO:teuthology.orchestra.run.smithi150.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-14T05:27:25.816 INFO:teuthology.orchestra.run.smithi071.stdout:(3/12): ceph-mgr-diskprediction-local-19.3.0-47 15 MB/s | 7.4 MB 00:00 2024-09-14T05:27:25.858 INFO:teuthology.orchestra.run.smithi150.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.4 MB/s | 801 kB 00:00 2024-09-14T05:27:26.050 INFO:teuthology.orchestra.run.smithi150.stdout:(3/12): ceph-mgr-diskprediction-local-19.3.0-47 14 MB/s | 7.4 MB 00:00 2024-09-14T05:27:26.067 INFO:teuthology.orchestra.run.smithi150.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 93 kB/s | 33 kB 00:00 2024-09-14T05:27:26.083 INFO:teuthology.orchestra.run.smithi071.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 53 kB/s | 33 kB 00:00 2024-09-14T05:27:26.267 INFO:teuthology.orchestra.run.smithi150.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 192 kB/s | 38 kB 00:00 2024-09-14T05:27:26.343 INFO:teuthology.orchestra.run.smithi150.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 62 kB/s | 18 kB 00:00 2024-09-14T05:27:26.392 INFO:teuthology.orchestra.run.smithi071.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 32 kB/s | 18 kB 00:00 2024-09-14T05:27:26.721 INFO:teuthology.orchestra.run.smithi150.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 653 kB/s | 245 kB 00:00 2024-09-14T05:27:26.746 INFO:teuthology.orchestra.run.smithi071.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.4 MB/s | 3.0 MB 00:01 2024-09-14T05:27:26.759 INFO:teuthology.orchestra.run.smithi071.stdout:(7/12): openblas-0.3.26-2.el9.x86_64.rpm 58 kB/s | 38 kB 00:00 2024-09-14T05:27:26.777 INFO:teuthology.orchestra.run.smithi150.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.3 MB/s | 3.0 MB 00:00 2024-09-14T05:27:26.897 INFO:teuthology.orchestra.run.smithi150.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.3 MB/s | 250 kB 00:00 2024-09-14T05:27:26.969 INFO:teuthology.orchestra.run.smithi150.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 7.0 MB/s | 4.9 MB 00:00 2024-09-14T05:27:27.261 INFO:teuthology.orchestra.run.smithi150.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.4 MB/s | 5.1 MB 00:00 2024-09-14T05:27:27.501 INFO:teuthology.orchestra.run.smithi071.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 319 kB/s | 245 kB 00:00 2024-09-14T05:27:27.634 INFO:teuthology.orchestra.run.smithi071.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 4.0 MB/s | 4.9 MB 00:01 2024-09-14T05:27:27.703 INFO:teuthology.orchestra.run.smithi150.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-09-14T05:27:27.703 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:27.703 INFO:teuthology.orchestra.run.smithi150.stdout:Total 16 MB/s | 38 MB 00:02 2024-09-14T05:27:27.959 INFO:teuthology.orchestra.run.smithi071.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.2 MB/s | 5.1 MB 00:01 2024-09-14T05:27:28.060 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-14T05:27:28.087 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-14T05:27:28.087 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-14T05:27:28.351 INFO:teuthology.orchestra.run.smithi071.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 295 kB/s | 250 kB 00:00 2024-09-14T05:27:28.542 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-14T05:27:28.542 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-14T05:27:29.101 INFO:teuthology.orchestra.run.smithi071.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-14T05:27:29.103 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:29.103 INFO:teuthology.orchestra.run.smithi071.stdout:Total 9.8 MB/s | 38 MB 00:03 2024-09-14T05:27:29.304 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-14T05:27:29.381 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-14T05:27:29.434 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-14T05:27:29.455 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-14T05:27:29.481 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-14T05:27:29.481 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-14T05:27:29.534 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-14T05:27:29.577 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-14T05:27:29.765 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-14T05:27:29.793 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-14T05:27:29.871 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-14T05:27:29.923 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-14T05:27:29.924 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-14T05:27:30.545 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-14T05:27:30.675 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-14T05:27:30.697 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-14T05:27:30.778 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-14T05:27:30.833 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-14T05:27:30.933 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-14T05:27:30.976 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-14T05:27:31.160 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-14T05:27:31.191 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-14T05:27:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-14T05:27:31.935 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-14T05:27:31.955 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T05:27:32.046 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-14T05:27:32.996 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-14T05:27:33.028 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e 12/12 2024-09-14T05:27:33.285 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T05:27:34.334 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-14T05:27:34.365 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e 12/12 2024-09-14T05:27:35.592 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e 12/12 2024-09-14T05:27:35.592 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e 1/12 2024-09-14T05:27:35.592 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-14T05:27:35.592 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-14T05:27:35.592 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-14T05:27:35.592 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-14T05:27:35.593 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-14T05:27:35.593 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-14T05:27:35.593 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-14T05:27:35.593 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-14T05:27:35.593 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T05:27:35.593 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-14T05:27:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-14T05:27:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:36.025 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-14T05:27:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T05:27:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-14T05:27:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-14T05:27:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-14T05:27:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-14T05:27:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-14T05:27:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-14T05:27:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-14T05:27:36.026 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-14T05:27:36.026 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-14T05:27:36.026 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-14T05:27:36.026 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-14T05:27:36.026 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:36.026 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:27:36.405 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-rook 2024-09-14T05:27:36.984 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:19 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:27:37.057 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e 12/12 2024-09-14T05:27:37.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e 1/12 2024-09-14T05:27:37.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-14T05:27:37.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-14T05:27:37.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-14T05:27:37.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-14T05:27:37.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-14T05:27:37.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-14T05:27:37.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-14T05:27:37.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-14T05:27:37.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T05:27:37.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-14T05:27:37.121 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:27:37.122 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:37.122 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-14T05:27:37.122 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:37.123 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-14T05:27:37.123 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 49 k 2024-09-14T05:27:37.123 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-14T05:27:37.123 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-14T05:27:37.123 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-14T05:27:37.123 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-14T05:27:37.123 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-14T05:27:37.123 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-14T05:27:37.123 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-14T05:27:37.123 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-14T05:27:37.123 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-14T05:27:37.123 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-14T05:27:37.123 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-14T05:27:37.124 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-14T05:27:37.124 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-14T05:27:37.124 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:37.124 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-14T05:27:37.124 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:37.124 INFO:teuthology.orchestra.run.smithi150.stdout:Install 13 Packages 2024-09-14T05:27:37.124 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:37.125 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 2.2 M 2024-09-14T05:27:37.125 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 26 M 2024-09-14T05:27:37.125 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-14T05:27:37.487 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-14T05:27:37.488 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:37.488 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-14T05:27:37.488 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T05:27:37.488 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-14T05:27:37.488 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-14T05:27:37.488 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-14T05:27:37.488 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-14T05:27:37.488 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-14T05:27:37.488 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-14T05:27:37.488 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-14T05:27:37.488 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-14T05:27:37.489 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-14T05:27:37.489 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-14T05:27:37.489 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-14T05:27:37.489 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:37.489 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:27:37.550 INFO:teuthology.orchestra.run.smithi150.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 219 kB/s | 26 kB 00:00 2024-09-14T05:27:37.575 INFO:teuthology.orchestra.run.smithi150.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 135 kB/s | 19 kB 00:00 2024-09-14T05:27:37.600 INFO:teuthology.orchestra.run.smithi150.stdout:(3/13): ceph-mgr-rook-19.3.0-4775.g7f24e589.el9 295 kB/s | 49 kB 00:00 2024-09-14T05:27:37.658 INFO:teuthology.orchestra.run.smithi150.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-14T05:27:37.684 INFO:teuthology.orchestra.run.smithi150.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-14T05:27:37.717 INFO:teuthology.orchestra.run.smithi150.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 914 kB/s | 54 kB 00:00 2024-09-14T05:27:37.751 INFO:teuthology.orchestra.run.smithi150.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 480 kB/s | 32 kB 00:00 2024-09-14T05:27:37.776 INFO:teuthology.orchestra.run.smithi150.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 248 kB/s | 14 kB 00:00 2024-09-14T05:27:37.827 INFO:teuthology.orchestra.run.smithi150.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 2.9 MB/s | 220 kB 00:00 2024-09-14T05:27:37.860 INFO:teuthology.orchestra.run.smithi150.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.0 MB/s | 279 kB 00:00 2024-09-14T05:27:37.885 INFO:teuthology.orchestra.run.smithi150.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-14T05:27:37.900 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr-rook 2024-09-14T05:27:37.927 INFO:teuthology.orchestra.run.smithi150.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-14T05:27:37.961 INFO:teuthology.orchestra.run.smithi150.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 895 kB/s | 90 kB 00:00 2024-09-14T05:27:37.963 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:37.964 INFO:teuthology.orchestra.run.smithi150.stdout:Total 2.6 MB/s | 2.2 MB 00:00 2024-09-14T05:27:38.029 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-14T05:27:38.045 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-14T05:27:38.045 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-14T05:27:38.137 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-14T05:27:38.137 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-14T05:27:38.478 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:24 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:27:38.615 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:27:38.616 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:38.616 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-09-14T05:27:38.616 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:38.616 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-14T05:27:38.616 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 49 k 2024-09-14T05:27:38.616 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-09-14T05:27:38.616 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-14T05:27:38.616 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-14T05:27:38.616 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-14T05:27:38.616 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-14T05:27:38.616 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-14T05:27:38.617 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-14T05:27:38.617 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-14T05:27:38.617 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-14T05:27:38.617 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-14T05:27:38.617 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-14T05:27:38.617 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-14T05:27:38.617 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-14T05:27:38.617 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:38.617 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-14T05:27:38.617 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:38.617 INFO:teuthology.orchestra.run.smithi071.stdout:Install 13 Packages 2024-09-14T05:27:38.617 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:38.618 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 2.2 M 2024-09-14T05:27:38.618 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 26 M 2024-09-14T05:27:38.618 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-14T05:27:38.663 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-14T05:27:38.836 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-14T05:27:38.888 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-14T05:27:38.943 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-14T05:27:38.979 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-14T05:27:39.021 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-14T05:27:39.039 INFO:teuthology.orchestra.run.smithi071.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 231 kB/s | 26 kB 00:00 2024-09-14T05:27:39.064 INFO:teuthology.orchestra.run.smithi071.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 141 kB/s | 19 kB 00:00 2024-09-14T05:27:39.090 INFO:teuthology.orchestra.run.smithi071.stdout:(3/13): ceph-mgr-rook-19.3.0-4775.g7f24e589.el9 305 kB/s | 49 kB 00:00 2024-09-14T05:27:39.125 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-14T05:27:39.173 INFO:teuthology.orchestra.run.smithi071.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-14T05:27:39.207 INFO:teuthology.orchestra.run.smithi071.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 222 kB 00:00 2024-09-14T05:27:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-14T05:27:39.232 INFO:teuthology.orchestra.run.smithi071.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 916 kB/s | 54 kB 00:00 2024-09-14T05:27:39.274 INFO:teuthology.orchestra.run.smithi071.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 481 kB/s | 32 kB 00:00 2024-09-14T05:27:39.289 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-14T05:27:39.291 INFO:teuthology.orchestra.run.smithi071.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 279 kB 00:00 2024-09-14T05:27:39.316 INFO:teuthology.orchestra.run.smithi071.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 168 kB/s | 14 kB 00:00 2024-09-14T05:27:39.350 INFO:teuthology.orchestra.run.smithi071.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.2 MB/s | 59 kB 00:00 2024-09-14T05:27:39.375 INFO:teuthology.orchestra.run.smithi071.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 2.2 MB/s | 220 kB 00:00 2024-09-14T05:27:39.400 INFO:teuthology.orchestra.run.smithi071.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-14T05:27:39.442 INFO:teuthology.orchestra.run.smithi071.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-14T05:27:39.444 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:39.445 INFO:teuthology.orchestra.run.smithi071.stdout:Total 2.7 MB/s | 2.2 MB 00:00 2024-09-14T05:27:39.508 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-14T05:27:39.523 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-14T05:27:39.523 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-14T05:27:39.611 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-14T05:27:39.612 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-14T05:27:40.026 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-14T05:27:40.086 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-14T05:27:40.113 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-14T05:27:40.120 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-14T05:27:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-14T05:27:40.198 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 13/13 2024-09-14T05:27:40.288 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-14T05:27:40.352 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-14T05:27:40.407 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-14T05:27:40.444 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-14T05:27:40.486 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-14T05:27:40.595 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-14T05:27:40.705 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-14T05:27:40.754 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-14T05:27:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 13/13 2024-09-14T05:27:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 1/13 2024-09-14T05:27:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-14T05:27:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-14T05:27:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-14T05:27:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-14T05:27:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-14T05:27:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-14T05:27:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-14T05:27:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-14T05:27:40.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-14T05:27:40.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-14T05:27:40.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-14T05:27:41.331 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-14T05:27:41.331 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:41.331 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-14T05:27:41.331 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T05:27:41.331 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-14T05:27:41.331 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-14T05:27:41.332 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-14T05:27:41.332 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-14T05:27:41.332 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-14T05:27:41.332 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-14T05:27:41.332 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-14T05:27:41.332 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-14T05:27:41.332 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-14T05:27:41.332 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-14T05:27:41.332 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-14T05:27:41.332 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-14T05:27:41.332 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:41.332 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:27:41.466 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-14T05:27:41.526 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-14T05:27:41.551 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-14T05:27:41.583 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-cephadm 2024-09-14T05:27:41.599 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-14T05:27:41.621 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 13/13 2024-09-14T05:27:42.155 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:25 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:27:42.292 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:27:42.292 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:42.292 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-14T05:27:42.292 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:42.292 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-14T05:27:42.292 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 154 k 2024-09-14T05:27:42.293 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-14T05:27:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-14T05:27:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-14T05:27:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-14T05:27:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-14T05:27:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-14T05:27:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:42.293 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-14T05:27:42.293 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:42.293 INFO:teuthology.orchestra.run.smithi150.stdout:Install 6 Packages 2024-09-14T05:27:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:42.293 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 7.0 M 2024-09-14T05:27:42.293 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 33 M 2024-09-14T05:27:42.294 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-14T05:27:42.406 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 13/13 2024-09-14T05:27:42.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 1/13 2024-09-14T05:27:42.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-14T05:27:42.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-14T05:27:42.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-14T05:27:42.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-14T05:27:42.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-14T05:27:42.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-14T05:27:42.407 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-14T05:27:42.407 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-14T05:27:42.407 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-14T05:27:42.407 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-14T05:27:42.407 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-14T05:27:42.750 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-14T05:27:42.750 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:42.750 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-14T05:27:42.750 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T05:27:42.750 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-14T05:27:42.751 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-14T05:27:42.751 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-14T05:27:42.751 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-14T05:27:42.751 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-14T05:27:42.751 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-14T05:27:42.751 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-14T05:27:42.751 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-14T05:27:42.751 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-14T05:27:42.751 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-14T05:27:42.751 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-14T05:27:42.751 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-14T05:27:42.751 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:42.751 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:27:42.837 INFO:teuthology.orchestra.run.smithi150.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.7 MB/s | 248 kB 00:00 2024-09-14T05:27:42.896 INFO:teuthology.orchestra.run.smithi150.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-14T05:27:42.912 INFO:teuthology.orchestra.run.smithi150.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 3.4 MB/s | 58 kB 00:00 2024-09-14T05:27:42.938 INFO:teuthology.orchestra.run.smithi150.stdout:(4/6): ceph-mgr-cephadm-19.3.0-4775.g7f24e589.e 640 kB/s | 154 kB 00:00 2024-09-14T05:27:42.954 INFO:teuthology.orchestra.run.smithi150.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.7 MB/s | 74 kB 00:00 2024-09-14T05:27:43.026 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr-cephadm 2024-09-14T05:27:43.055 INFO:teuthology.orchestra.run.smithi150.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-14T05:27:43.056 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:43.057 INFO:teuthology.orchestra.run.smithi150.stdout:Total 9.2 MB/s | 7.0 MB 00:00 2024-09-14T05:27:43.128 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-14T05:27:43.139 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-14T05:27:43.139 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-14T05:27:43.240 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-14T05:27:43.240 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-14T05:27:43.552 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-14T05:27:43.595 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:29 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:27:43.721 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-14T05:27:43.734 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:27:43.734 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:43.734 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-09-14T05:27:43.735 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:43.735 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-14T05:27:43.735 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 154 k 2024-09-14T05:27:43.735 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-09-14T05:27:43.735 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-14T05:27:43.735 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-14T05:27:43.735 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-14T05:27:43.735 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-14T05:27:43.735 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-14T05:27:43.735 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:43.735 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-14T05:27:43.736 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:43.736 INFO:teuthology.orchestra.run.smithi071.stdout:Install 6 Packages 2024-09-14T05:27:43.736 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:43.736 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 7.0 M 2024-09-14T05:27:43.736 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 33 M 2024-09-14T05:27:43.736 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-14T05:27:43.765 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-14T05:27:44.127 INFO:teuthology.orchestra.run.smithi071.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-14T05:27:44.152 INFO:teuthology.orchestra.run.smithi071.stdout:(2/6): ceph-mgr-cephadm-19.3.0-4775.g7f24e589.e 684 kB/s | 154 kB 00:00 2024-09-14T05:27:44.202 INFO:teuthology.orchestra.run.smithi071.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-14T05:27:44.227 INFO:teuthology.orchestra.run.smithi071.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 5.3 MB/s | 548 kB 00:00 2024-09-14T05:27:44.253 INFO:teuthology.orchestra.run.smithi071.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-14T05:27:44.289 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-14T05:27:44.376 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-14T05:27:44.452 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-14T05:27:44.477 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 6/6 2024-09-14T05:27:45.269 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 6/6 2024-09-14T05:27:45.269 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/6 2024-09-14T05:27:45.269 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-14T05:27:45.269 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-14T05:27:45.269 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-14T05:27:45.269 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-14T05:27:45.311 INFO:teuthology.orchestra.run.smithi071.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.3 MB/s | 6.0 MB 00:01 2024-09-14T05:27:45.314 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:45.314 INFO:teuthology.orchestra.run.smithi071.stdout:Total 4.4 MB/s | 7.0 MB 00:01 2024-09-14T05:27:45.385 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-14T05:27:45.396 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-14T05:27:45.396 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-14T05:27:45.496 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-14T05:27:45.496 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-14T05:27:45.596 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-14T05:27:45.596 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:45.596 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-14T05:27:45.596 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T05:27:45.596 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-14T05:27:45.596 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-14T05:27:45.596 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-14T05:27:45.597 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-14T05:27:45.597 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-14T05:27:45.597 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:45.597 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:27:45.810 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-14T05:27:45.838 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-fuse 2024-09-14T05:27:45.992 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-14T05:27:46.046 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-14T05:27:46.406 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:29 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:27:46.541 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:27:46.541 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:46.541 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-14T05:27:46.541 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:46.542 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-14T05:27:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 900 k 2024-09-14T05:27:46.542 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-14T05:27:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-14T05:27:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:46.542 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-14T05:27:46.542 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:46.542 INFO:teuthology.orchestra.run.smithi150.stdout:Install 2 Packages 2024-09-14T05:27:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:46.543 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 979 k 2024-09-14T05:27:46.543 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 2.9 M 2024-09-14T05:27:46.543 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-14T05:27:46.564 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-14T05:27:46.657 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-14T05:27:46.725 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-14T05:27:46.740 INFO:teuthology.orchestra.run.smithi150.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 530 kB/s | 80 kB 00:00 2024-09-14T05:27:46.750 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 6/6 2024-09-14T05:27:46.998 INFO:teuthology.orchestra.run.smithi150.stdout:(2/2): ceph-fuse-19.3.0-4775.g7f24e589.el9.x86_ 2.1 MB/s | 900 kB 00:00 2024-09-14T05:27:46.999 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:46.999 INFO:teuthology.orchestra.run.smithi150.stdout:Total 2.1 MB/s | 979 kB 00:00 2024-09-14T05:27:47.011 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-14T05:27:47.034 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-14T05:27:47.035 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-14T05:27:47.115 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-14T05:27:47.115 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-14T05:27:47.311 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-14T05:27:47.445 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-14T05:27:47.514 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 2/2 2024-09-14T05:27:47.533 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 6/6 2024-09-14T05:27:47.533 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/6 2024-09-14T05:27:47.533 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-14T05:27:47.533 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-14T05:27:47.534 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-14T05:27:47.534 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-14T05:27:47.805 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-14T05:27:47.806 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:47.806 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-14T05:27:47.806 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T05:27:47.806 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-14T05:27:47.806 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-14T05:27:47.806 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-14T05:27:47.806 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-14T05:27:47.806 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-14T05:27:47.806 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:47.806 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:27:48.058 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-fuse 2024-09-14T05:27:48.207 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 2/2 2024-09-14T05:27:48.207 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/2 2024-09-14T05:27:48.479 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-14T05:27:48.479 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:48.480 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-14T05:27:48.480 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-14T05:27:48.480 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:48.480 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:27:48.613 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:34 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:27:48.738 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-volume 2024-09-14T05:27:48.746 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:27:48.746 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:48.747 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-09-14T05:27:48.747 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:48.747 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-14T05:27:48.747 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 900 k 2024-09-14T05:27:48.747 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-09-14T05:27:48.747 INFO:teuthology.orchestra.run.smithi071.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-14T05:27:48.747 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:48.747 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-14T05:27:48.747 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:48.747 INFO:teuthology.orchestra.run.smithi071.stdout:Install 2 Packages 2024-09-14T05:27:48.747 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:48.748 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 979 k 2024-09-14T05:27:48.748 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 2.9 M 2024-09-14T05:27:48.748 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-14T05:27:48.896 INFO:teuthology.orchestra.run.smithi071.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 629 kB/s | 80 kB 00:00 2024-09-14T05:27:49.163 INFO:teuthology.orchestra.run.smithi071.stdout:(2/2): ceph-fuse-19.3.0-4775.g7f24e589.el9.x86_ 2.2 MB/s | 900 kB 00:00 2024-09-14T05:27:49.163 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:49.164 INFO:teuthology.orchestra.run.smithi071.stdout:Total 2.3 MB/s | 979 kB 00:00 2024-09-14T05:27:49.175 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-14T05:27:49.197 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-14T05:27:49.197 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-14T05:27:49.276 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-14T05:27:49.277 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-14T05:27:49.315 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:32 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:27:49.455 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:27:49.456 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-14T05:27:49.456 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:49.456 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-14T05:27:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 285 k 2024-09-14T05:27:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:49.456 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-14T05:27:49.456 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:49.456 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-14T05:27:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:49.456 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 285 k 2024-09-14T05:27:49.457 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 1.3 M 2024-09-14T05:27:49.457 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-14T05:27:49.460 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-14T05:27:49.607 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-14T05:27:49.670 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 2/2 2024-09-14T05:27:49.694 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-volume-19.3.0-4775.g7f24e589.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-14T05:27:49.695 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:49.695 INFO:teuthology.orchestra.run.smithi150.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-14T05:27:49.696 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-14T05:27:49.704 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-14T05:27:49.704 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-14T05:27:49.731 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-14T05:27:49.731 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-14T05:27:49.953 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-14T05:27:50.022 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T05:27:50.037 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T05:27:50.038 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T05:27:50.038 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-14T05:27:50.038 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:50.363 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 2/2 2024-09-14T05:27:50.363 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/2 2024-09-14T05:27:50.640 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-14T05:27:50.640 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:50.640 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-14T05:27:50.640 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-14T05:27:50.641 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:50.641 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:27:50.908 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-volume 2024-09-14T05:27:51.025 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T05:27:51.025 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:51.025 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-14T05:27:51.026 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T05:27:51.026 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:51.026 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:27:51.297 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install librados-devel 2024-09-14T05:27:51.486 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:37 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:27:51.625 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:27:51.625 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:51.626 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-09-14T05:27:51.626 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:51.626 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-14T05:27:51.626 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 285 k 2024-09-14T05:27:51.626 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:51.626 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-14T05:27:51.626 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:51.626 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-09-14T05:27:51.626 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:51.626 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 285 k 2024-09-14T05:27:51.626 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 1.3 M 2024-09-14T05:27:51.626 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-14T05:27:51.858 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-volume-19.3.0-4775.g7f24e589.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-14T05:27:51.859 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:51.859 INFO:teuthology.orchestra.run.smithi071.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-14T05:27:51.859 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-14T05:27:51.867 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-14T05:27:51.867 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-14T05:27:51.885 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:34 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:27:51.895 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-14T05:27:51.896 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-14T05:27:52.021 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:27:52.022 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:52.022 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repo Size 2024-09-14T05:27:52.022 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:52.022 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-14T05:27:52.023 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 126 k 2024-09-14T05:27:52.023 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:52.023 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-14T05:27:52.023 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:52.023 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-14T05:27:52.023 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:52.023 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 126 k 2024-09-14T05:27:52.023 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 456 k 2024-09-14T05:27:52.023 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-14T05:27:52.116 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-14T05:27:52.177 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T05:27:52.193 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T05:27:52.193 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T05:27:52.193 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-14T05:27:52.193 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:52.208 INFO:teuthology.orchestra.run.smithi150.stdout:librados-devel-19.3.0-4775.g7f24e589.el9.x86_64 685 kB/s | 126 kB 00:00 2024-09-14T05:27:52.208 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:52.209 INFO:teuthology.orchestra.run.smithi150.stdout:Total 679 kB/s | 126 kB 00:00 2024-09-14T05:27:52.209 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-14T05:27:52.215 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-14T05:27:52.215 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-14T05:27:52.278 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-14T05:27:52.278 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-14T05:27:52.425 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-14T05:27:52.532 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:27:53.011 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:27:53.260 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:27:53.261 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:53.261 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-14T05:27:53.261 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:27:53.261 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:53.261 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:27:53.280 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T05:27:53.280 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:53.280 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-14T05:27:53.280 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T05:27:53.280 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:53.281 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:27:53.505 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install libcephfs2 2024-09-14T05:27:53.599 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install librados-devel 2024-09-14T05:27:54.092 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:37 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:27:54.184 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:40 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:27:54.185 INFO:teuthology.orchestra.run.smithi150.stdout:Package libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T05:27:54.227 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:27:54.228 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-14T05:27:54.228 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:27:54.308 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install libcephfs-devel 2024-09-14T05:27:54.318 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:27:54.318 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:54.318 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repo Size 2024-09-14T05:27:54.319 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:54.319 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-14T05:27:54.319 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 126 k 2024-09-14T05:27:54.319 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:54.319 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-14T05:27:54.319 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:54.319 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-09-14T05:27:54.319 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:54.320 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 126 k 2024-09-14T05:27:54.320 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 456 k 2024-09-14T05:27:54.320 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-14T05:27:54.504 INFO:teuthology.orchestra.run.smithi071.stdout:librados-devel-19.3.0-4775.g7f24e589.el9.x86_64 687 kB/s | 126 kB 00:00 2024-09-14T05:27:54.505 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:54.505 INFO:teuthology.orchestra.run.smithi071.stdout:Total 680 kB/s | 126 kB 00:00 2024-09-14T05:27:54.505 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-14T05:27:54.511 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-14T05:27:54.511 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-14T05:27:54.578 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-14T05:27:54.578 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-14T05:27:54.739 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-14T05:27:54.837 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:27:54.880 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:37 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:27:55.016 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:27:55.017 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repo Size 2024-09-14T05:27:55.017 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:55.017 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-14T05:27:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 32 k 2024-09-14T05:27:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:55.017 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-14T05:27:55.017 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:27:55.017 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-14T05:27:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:55.018 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 32 k 2024-09-14T05:27:55.018 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 142 k 2024-09-14T05:27:55.018 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-14T05:27:55.169 INFO:teuthology.orchestra.run.smithi150.stdout:libcephfs-devel-19.3.0-4775.g7f24e589.el9.x86_6 209 kB/s | 32 kB 00:00 2024-09-14T05:27:55.170 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:55.170 INFO:teuthology.orchestra.run.smithi150.stdout:Total 208 kB/s | 32 kB 00:00 2024-09-14T05:27:55.170 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-14T05:27:55.174 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-14T05:27:55.174 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-14T05:27:55.201 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-14T05:27:55.201 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-14T05:27:55.288 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-14T05:27:55.308 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:27:55.400 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:27:55.581 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:27:55.581 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:55.581 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-14T05:27:55.581 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:27:55.582 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:55.582 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:27:55.806 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:27:55.863 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install libcephfs2 2024-09-14T05:27:56.036 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:27:56.036 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:56.036 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-14T05:27:56.036 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:27:56.037 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:27:56.037 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:27:56.196 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install librados2 2024-09-14T05:27:56.443 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:42 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:27:56.535 INFO:teuthology.orchestra.run.smithi071.stdout:Package libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T05:27:56.580 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:27:56.581 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-09-14T05:27:56.581 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:27:56.651 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install libcephfs-devel 2024-09-14T05:27:56.782 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:39 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:27:56.881 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T05:27:56.926 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:27:56.927 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-14T05:27:56.928 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:27:57.003 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install librbd1 2024-09-14T05:27:57.217 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:43 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:27:57.351 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:27:57.352 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:57.352 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repo Size 2024-09-14T05:27:57.352 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:57.352 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-14T05:27:57.352 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 32 k 2024-09-14T05:27:57.352 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:57.352 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-14T05:27:57.352 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:27:57.352 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-09-14T05:27:57.352 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:57.352 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 32 k 2024-09-14T05:27:57.353 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 142 k 2024-09-14T05:27:57.353 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-14T05:27:57.499 INFO:teuthology.orchestra.run.smithi071.stdout:libcephfs-devel-19.3.0-4775.g7f24e589.el9.x86_6 216 kB/s | 32 kB 00:00 2024-09-14T05:27:57.500 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:27:57.500 INFO:teuthology.orchestra.run.smithi071.stdout:Total 214 kB/s | 32 kB 00:00 2024-09-14T05:27:57.500 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-14T05:27:57.504 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-14T05:27:57.504 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-14T05:27:57.520 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-14T05:27:57.521 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-14T05:27:57.588 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:40 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:27:57.615 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-14T05:27:57.681 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T05:27:57.723 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:27:57.724 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-14T05:27:57.724 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:27:57.729 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:27:57.796 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-rados 2024-09-14T05:27:58.101 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:27:58.337 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:27:58.337 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:58.337 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-14T05:27:58.337 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:27:58.337 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:27:58.337 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:27:58.364 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:41 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:27:58.454 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T05:27:58.498 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:27:58.499 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-14T05:27:58.499 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:27:58.530 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install librados2 2024-09-14T05:27:58.563 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-rgw 2024-09-14T05:27:59.087 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:45 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:27:59.126 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:42 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:27:59.176 INFO:teuthology.orchestra.run.smithi071.stdout:Package librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T05:27:59.215 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T05:27:59.218 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:27:59.219 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-09-14T05:27:59.219 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:27:59.258 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:27:59.259 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-14T05:27:59.259 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:27:59.292 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install librbd1 2024-09-14T05:27:59.333 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-cephfs 2024-09-14T05:27:59.855 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:45 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:27:59.913 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:42 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:27:59.944 INFO:teuthology.orchestra.run.smithi071.stdout:Package librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T05:27:59.990 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:27:59.991 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-09-14T05:27:59.991 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:28:00.004 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T05:28:00.047 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:28:00.048 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-14T05:28:00.048 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:28:00.061 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install python3-rados 2024-09-14T05:28:00.109 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-rbd 2024-09-14T05:28:00.622 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:46 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:28:00.681 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:43 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:28:00.712 INFO:teuthology.orchestra.run.smithi071.stdout:Package python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T05:28:00.757 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:28:00.758 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-09-14T05:28:00.759 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:28:00.771 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T05:28:00.815 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:28:00.816 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-14T05:28:00.816 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:28:00.825 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install python3-rgw 2024-09-14T05:28:00.889 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install rbd-fuse 2024-09-14T05:28:01.386 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:47 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:28:01.457 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:44 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:28:01.476 INFO:teuthology.orchestra.run.smithi071.stdout:Package python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T05:28:01.519 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:28:01.520 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-09-14T05:28:01.520 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:28:01.591 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:28:01.592 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:28:01.592 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-09-14T05:28:01.592 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:28:01.592 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-14T05:28:01.592 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 85 k 2024-09-14T05:28:01.592 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:01.592 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-14T05:28:01.592 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:28:01.593 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-14T05:28:01.593 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:01.593 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 85 k 2024-09-14T05:28:01.593 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 231 k 2024-09-14T05:28:01.593 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-14T05:28:01.603 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install python3-cephfs 2024-09-14T05:28:01.782 INFO:teuthology.orchestra.run.smithi150.stdout:rbd-fuse-19.3.0-4775.g7f24e589.el9.x86_64.rpm 451 kB/s | 85 kB 00:00 2024-09-14T05:28:01.782 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:28:01.782 INFO:teuthology.orchestra.run.smithi150.stdout:Total 447 kB/s | 85 kB 00:00 2024-09-14T05:28:01.783 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-14T05:28:01.789 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-14T05:28:01.789 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-14T05:28:01.853 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-14T05:28:01.853 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-14T05:28:02.004 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-14T05:28:02.105 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:28:02.161 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:48 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:28:02.252 INFO:teuthology.orchestra.run.smithi071.stdout:Package python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T05:28:02.295 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:28:02.296 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-09-14T05:28:02.296 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:28:02.366 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install python3-rbd 2024-09-14T05:28:02.539 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:28:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:28:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:02.797 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-14T05:28:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:28:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:02.797 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:28:02.927 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:48 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:28:03.017 INFO:teuthology.orchestra.run.smithi071.stdout:Package python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T05:28:03.060 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:28:03.061 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-09-14T05:28:03.061 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:28:03.106 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install rbd-mirror 2024-09-14T05:28:03.127 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install rbd-fuse 2024-09-14T05:28:03.685 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:46 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:28:03.693 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:49 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:28:03.826 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:28:03.827 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:28:03.827 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repo Size 2024-09-14T05:28:03.827 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:28:03.827 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-14T05:28:03.827 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 3.2 M 2024-09-14T05:28:03.827 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:03.827 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-14T05:28:03.827 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:28:03.828 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-14T05:28:03.828 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:03.828 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 3.2 M 2024-09-14T05:28:03.828 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 14 M 2024-09-14T05:28:03.828 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-14T05:28:03.829 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:28:03.829 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:28:03.829 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-09-14T05:28:03.829 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:28:03.829 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-14T05:28:03.829 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 85 k 2024-09-14T05:28:03.829 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:03.829 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-14T05:28:03.830 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:28:03.830 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-09-14T05:28:03.830 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:03.830 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 85 k 2024-09-14T05:28:03.830 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 231 k 2024-09-14T05:28:03.830 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-14T05:28:04.019 INFO:teuthology.orchestra.run.smithi071.stdout:rbd-fuse-19.3.0-4775.g7f24e589.el9.x86_64.rpm 448 kB/s | 85 kB 00:00 2024-09-14T05:28:04.019 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:28:04.020 INFO:teuthology.orchestra.run.smithi071.stdout:Total 444 kB/s | 85 kB 00:00 2024-09-14T05:28:04.020 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-14T05:28:04.026 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-14T05:28:04.026 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-14T05:28:04.089 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-14T05:28:04.089 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-14T05:28:04.160 INFO:teuthology.orchestra.run.smithi150.stdout:rbd-mirror-19.3.0-4775.g7f24e589.el9.x86_64.rpm 9.6 MB/s | 3.2 MB 00:00 2024-09-14T05:28:04.161 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:28:04.161 INFO:teuthology.orchestra.run.smithi150.stdout:Total 9.5 MB/s | 3.2 MB 00:00 2024-09-14T05:28:04.161 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-14T05:28:04.170 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-14T05:28:04.170 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-14T05:28:04.242 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-14T05:28:04.253 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-14T05:28:04.253 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-14T05:28:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:28:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-14T05:28:04.761 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:28:04.787 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:28:04.787 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T05:28:04.787 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-14T05:28:04.787 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T05:28:04.787 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T05:28:04.787 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:04.808 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:28:05.064 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:28:05.064 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:05.064 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-14T05:28:05.064 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:28:05.065 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:05.065 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:28:05.347 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install rbd-mirror 2024-09-14T05:28:05.919 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:51 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:28:05.944 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:28:05.944 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:05.944 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-14T05:28:05.944 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:28:05.944 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:05.944 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:28:06.053 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:28:06.053 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:28:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repo Size 2024-09-14T05:28:06.054 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:28:06.054 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-14T05:28:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 3.2 M 2024-09-14T05:28:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:06.054 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-14T05:28:06.054 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:28:06.054 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-09-14T05:28:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:06.054 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 3.2 M 2024-09-14T05:28:06.054 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 14 M 2024-09-14T05:28:06.054 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-14T05:28:06.165 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install rbd-nbd 2024-09-14T05:28:06.397 INFO:teuthology.orchestra.run.smithi071.stdout:rbd-mirror-19.3.0-4775.g7f24e589.el9.x86_64.rpm 9.3 MB/s | 3.2 MB 00:00 2024-09-14T05:28:06.397 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:28:06.398 INFO:teuthology.orchestra.run.smithi071.stdout:Total 9.2 MB/s | 3.2 MB 00:00 2024-09-14T05:28:06.398 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-14T05:28:06.407 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-14T05:28:06.407 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-14T05:28:06.488 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-14T05:28:06.489 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-14T05:28:06.740 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:49 ago on Sat 14 Sep 2024 05:26:17 AM UTC. 2024-09-14T05:28:06.875 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-14T05:28:06.875 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:28:06.875 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-09-14T05:28:06.876 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:28:06.876 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-14T05:28:06.876 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 171 k 2024-09-14T05:28:06.876 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:06.876 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-14T05:28:06.876 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-14T05:28:06.876 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-14T05:28:06.876 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:06.876 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 171 k 2024-09-14T05:28:06.876 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 490 k 2024-09-14T05:28:06.876 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-14T05:28:06.924 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-14T05:28:06.981 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:28:07.006 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:28:07.006 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T05:28:07.006 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-14T05:28:07.007 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T05:28:07.007 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T05:28:07.007 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:07.111 INFO:teuthology.orchestra.run.smithi150.stdout:rbd-nbd-19.3.0-4775.g7f24e589.el9.x86_64.rpm 727 kB/s | 171 kB 00:00 2024-09-14T05:28:07.112 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:28:07.113 INFO:teuthology.orchestra.run.smithi150.stdout:Total 722 kB/s | 171 kB 00:00 2024-09-14T05:28:07.113 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-14T05:28:07.121 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-14T05:28:07.121 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-14T05:28:07.187 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-14T05:28:07.187 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-14T05:28:07.350 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-14T05:28:07.452 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:28:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:28:08.127 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:28:08.127 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:08.127 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-14T05:28:08.127 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:28:08.127 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:08.127 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:28:08.137 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:28:08.137 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:08.138 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-14T05:28:08.138 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:28:08.138 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:08.138 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-14T05:28:08.383 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install rbd-nbd 2024-09-14T05:28:08.403 DEBUG:teuthology.parallel:result is None 2024-09-14T05:28:08.962 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:54 ago on Sat 14 Sep 2024 05:26:14 AM UTC. 2024-09-14T05:28:09.097 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-14T05:28:09.098 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:28:09.098 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-09-14T05:28:09.098 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:28:09.098 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-14T05:28:09.098 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 171 k 2024-09-14T05:28:09.098 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:09.099 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-14T05:28:09.099 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-14T05:28:09.099 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-09-14T05:28:09.099 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:09.099 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 171 k 2024-09-14T05:28:09.099 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 490 k 2024-09-14T05:28:09.099 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-14T05:28:09.308 INFO:teuthology.orchestra.run.smithi071.stdout:rbd-nbd-19.3.0-4775.g7f24e589.el9.x86_64.rpm 816 kB/s | 171 kB 00:00 2024-09-14T05:28:09.309 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-14T05:28:09.313 INFO:teuthology.orchestra.run.smithi071.stdout:Total 810 kB/s | 171 kB 00:00 2024-09-14T05:28:09.313 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-14T05:28:09.317 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-14T05:28:09.317 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-14T05:28:09.381 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-14T05:28:09.381 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-14T05:28:09.551 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-14T05:28:09.657 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:28:10.101 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:28:10.352 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T05:28:10.352 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:10.352 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-14T05:28:10.352 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T05:28:10.353 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:10.353 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-14T05:28:10.603 DEBUG:teuthology.parallel:result is None 2024-09-14T05:28:10.604 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T05:28:10.746 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-14T05:28:10.767 INFO:teuthology.orchestra.run.smithi071.stdout:19.3.0-4775.g7f24e589.el9 2024-09-14T05:28:10.768 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4775.g7f24e589.el9 2024-09-14T05:28:10.768 INFO:teuthology.task.install:The correct ceph version 19.3.0-4775.g7f24e589 is installed. 2024-09-14T05:28:10.770 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T05:28:10.910 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-14T05:28:10.935 INFO:teuthology.orchestra.run.smithi150.stdout:19.3.0-4775.g7f24e589.el9 2024-09-14T05:28:10.935 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4775.g7f24e589.el9 2024-09-14T05:28:10.935 INFO:teuthology.task.install:The correct ceph version 19.3.0-4775.g7f24e589 is installed. 2024-09-14T05:28:10.937 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-14T05:28:10.938 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:10.938 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T05:28:10.963 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:28:10.964 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T05:28:11.004 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-14T05:28:11.005 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:11.005 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T05:28:11.030 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T05:28:11.095 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:28:11.095 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T05:28:11.126 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T05:28:11.193 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-14T05:28:11.193 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:11.193 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T05:28:11.217 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T05:28:11.281 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:28:11.281 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T05:28:11.308 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T05:28:11.373 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-14T05:28:11.373 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:11.374 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T05:28:11.413 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T05:28:11.478 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:28:11.478 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T05:28:11.504 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T05:28:11.569 INFO:teuthology.run_tasks:Running task ceph... 2024-09-14T05:28:11.646 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-14T05:28:11.646 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /var/log/ceph 2024-09-14T05:28:11.649 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /var/log/ceph 2024-09-14T05:28:11.674 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-14T05:28:11.674 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T05:28:11.713 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T05:28:11.738 INFO:tasks.ceph:Creating extra log directories... 2024-09-14T05:28:11.738 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T05:28:11.778 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T05:28:11.812 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-14T05:28:11.812 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-14T05:28:11.812 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900319', 'branch': 'main', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7900319', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '0d61375b8ab53be4c233f5b93e463edfdff9316a'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 950, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 5839, 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47', 'sleep_before_teardown': 0, 'subset': '99/128', 'suite': 'rbd', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '0d61375b8ab53be4c233f5b93e463edfdff9316a', 'targets': {'smithi071.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJAs2acobITiT2tDphg8y2qH2yrlISSYq5XeAOkDe/tq9gSWjDYGhMlT8HS7dJfTqg3ICR74VygRpwO8EbaneGQ=', 'smithi150.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBzm2iJBTdRQUrsd7xUwblriZV0viHx41Tw2uJxIklbZjcH1IOPZbAw3L9XvL31D7cPQYFspIZUfKfLtGJSFUQg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_20:16:17', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-14T05:28:11.813 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T05:28:11.843 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T05:28:11.858 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T05:28:11.898 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T05:28:11.923 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:11.923 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-09-14T05:28:11.976 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-14T05:28:11.977 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_1 2024-09-14T05:28:12.032 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-14T05:28:12.032 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T05:28:12.032 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-14T05:28:12.032 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T05:28:12.032 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T05:28:12.032 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-14 05:28:07.249842037 +0000 2024-09-14T05:28:12.032 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-14 05:25:34.431578947 +0000 2024-09-14T05:28:12.032 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-14 05:25:34.431578947 +0000 2024-09-14T05:28:12.032 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-14 05:25:34.431578947 +0000 2024-09-14T05:28:12.033 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-14T05:28:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-14T05:28:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-14T05:28:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000179948 s, 2.8 MB/s 2024-09-14T05:28:12.096 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-14T05:28:12.152 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_2 2024-09-14T05:28:12.207 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-14T05:28:12.207 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T05:28:12.207 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-14T05:28:12.207 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T05:28:12.207 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T05:28:12.207 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-14 05:28:07.249842037 +0000 2024-09-14T05:28:12.207 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-14 05:25:34.745573262 +0000 2024-09-14T05:28:12.207 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-14 05:25:34.745573262 +0000 2024-09-14T05:28:12.207 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-14 05:25:34.745573262 +0000 2024-09-14T05:28:12.208 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-14T05:28:12.269 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-14T05:28:12.269 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-14T05:28:12.269 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000187158 s, 2.7 MB/s 2024-09-14T05:28:12.271 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-14T05:28:12.327 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_3 2024-09-14T05:28:12.382 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-14T05:28:12.382 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T05:28:12.382 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-14T05:28:12.382 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T05:28:12.382 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T05:28:12.382 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-14 05:28:07.249842037 +0000 2024-09-14T05:28:12.382 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-14 05:25:35.062567522 +0000 2024-09-14T05:28:12.382 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-14 05:25:35.062567522 +0000 2024-09-14T05:28:12.382 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-14 05:25:35.062567522 +0000 2024-09-14T05:28:12.383 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-14T05:28:12.444 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-14T05:28:12.444 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-14T05:28:12.444 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000194194 s, 2.6 MB/s 2024-09-14T05:28:12.446 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-14T05:28:12.501 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_4 2024-09-14T05:28:12.557 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-14T05:28:12.558 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T05:28:12.558 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-14T05:28:12.558 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T05:28:12.558 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T05:28:12.558 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-14 05:28:07.250842019 +0000 2024-09-14T05:28:12.558 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-14 05:25:35.398561438 +0000 2024-09-14T05:28:12.558 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-14 05:25:35.398561438 +0000 2024-09-14T05:28:12.558 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-14 05:25:35.398561438 +0000 2024-09-14T05:28:12.558 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-14T05:28:12.620 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-14T05:28:12.620 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-14T05:28:12.620 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000182774 s, 2.8 MB/s 2024-09-14T05:28:12.621 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-14T05:28:12.678 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T05:28:12.678 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:28:12.678 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-09-14T05:28:12.695 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-14T05:28:12.695 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-09-14T05:28:12.752 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-14T05:28:12.752 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T05:28:12.752 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-14T05:28:12.752 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T05:28:12.752 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T05:28:12.752 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-14 05:28:05.053283919 +0000 2024-09-14T05:28:12.753 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-14 05:25:37.161201526 +0000 2024-09-14T05:28:12.753 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-14 05:25:37.161201526 +0000 2024-09-14T05:28:12.753 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-14 05:25:37.161201526 +0000 2024-09-14T05:28:12.753 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-14T05:28:12.817 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-14T05:28:12.817 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-14T05:28:12.817 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.00028943 s, 1.8 MB/s 2024-09-14T05:28:12.818 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-14T05:28:12.875 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-09-14T05:28:12.931 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-14T05:28:12.931 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T05:28:12.931 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-14T05:28:12.931 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T05:28:12.931 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T05:28:12.931 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-14 05:28:05.053283919 +0000 2024-09-14T05:28:12.931 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-14 05:25:37.492195287 +0000 2024-09-14T05:28:12.932 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-14 05:25:37.492195287 +0000 2024-09-14T05:28:12.932 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-14 05:25:37.492195287 +0000 2024-09-14T05:28:12.932 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-14T05:28:12.995 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-14T05:28:12.995 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-14T05:28:12.995 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.00018824 s, 2.7 MB/s 2024-09-14T05:28:12.996 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-14T05:28:13.053 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-09-14T05:28:13.107 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-14T05:28:13.108 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T05:28:13.108 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-14T05:28:13.108 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T05:28:13.108 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T05:28:13.108 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-14 05:28:05.053283919 +0000 2024-09-14T05:28:13.108 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-14 05:25:37.809189312 +0000 2024-09-14T05:28:13.108 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-14 05:25:37.809189312 +0000 2024-09-14T05:28:13.108 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-14 05:25:37.809189312 +0000 2024-09-14T05:28:13.108 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-14T05:28:13.170 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-14T05:28:13.171 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-14T05:28:13.171 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000174604 s, 2.9 MB/s 2024-09-14T05:28:13.172 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-14T05:28:13.229 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-09-14T05:28:13.284 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-14T05:28:13.284 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T05:28:13.284 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-14T05:28:13.284 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T05:28:13.284 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T05:28:13.284 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-14 05:28:05.054283900 +0000 2024-09-14T05:28:13.284 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-14 05:25:38.149182903 +0000 2024-09-14T05:28:13.284 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-14 05:25:38.149182903 +0000 2024-09-14T05:28:13.284 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-14 05:25:38.149182903 +0000 2024-09-14T05:28:13.285 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-14T05:28:13.346 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-14T05:28:13.346 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-14T05:28:13.347 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000178207 s, 2.9 MB/s 2024-09-14T05:28:13.348 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-14T05:28:13.404 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T05:28:13.404 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi071.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-14T05:28:13.404 INFO:tasks.ceph:Generating config... 2024-09-14T05:28:13.405 INFO:tasks.ceph:[client] rbd cache = True 2024-09-14T05:28:13.406 INFO:tasks.ceph:[client] rbd cache policy = writearound 2024-09-14T05:28:13.406 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-14T05:28:13.406 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-14T05:28:13.406 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-14T05:28:13.406 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-14T05:28:13.406 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-14T05:28:13.406 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-14T05:28:13.406 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-14T05:28:13.406 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-14T05:28:13.406 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-14T05:28:13.406 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-14T05:28:13.406 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-14T05:28:13.406 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-14T05:28:13.406 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-14T05:28:13.406 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-14T05:28:13.407 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-14T05:28:13.407 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-14T05:28:13.407 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-14T05:28:13.407 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-14T05:28:13.407 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-14T05:28:13.407 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-14T05:28:13.407 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-14T05:28:13.407 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-14T05:28:13.407 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-14T05:28:13.407 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2024-09-14T05:28:13.407 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-14T05:28:13.407 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-14T05:28:13.407 INFO:tasks.ceph:Setting up mon.a... 2024-09-14T05:28:13.408 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-14T05:28:13.465 INFO:teuthology.orchestra.run.smithi071.stdout:creating /etc/ceph/ceph.keyring 2024-09-14T05:28:13.467 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-14T05:28:13.513 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T05:28:13.539 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.71'), ('mon.c', '[v2:172.21.15.71:3301,v1:172.21.15.71:6790]'), ('mon.b', '172.21.15.150')] 2024-09-14T05:28:13.539 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.71,[v2:172.21.15.71:3301,v1:172.21.15.71:6790],172.21.15.150', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache policy': 'writearound'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-14T05:28:13.541 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:13.541 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-14T05:28:13.596 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.71 --addv c '[v2:172.21.15.71:3301,v1:172.21.15.71:6790]' --add b 172.21.15.150 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-14T05:28:13.683 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-14T05:28:13.684 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool: generated fsid 1127be44-5f14-4e7d-ad23-f608afe154f5 2024-09-14T05:28:13.684 INFO:teuthology.orchestra.run.smithi071.stdout:setting min_mon_release = quincy 2024-09-14T05:28:13.684 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 0 2024-09-14T05:28:13.684 INFO:teuthology.orchestra.run.smithi071.stdout:fsid 1127be44-5f14-4e7d-ad23-f608afe154f5 2024-09-14T05:28:13.684 INFO:teuthology.orchestra.run.smithi071.stdout:last_changed 2024-09-14T05:28:13.683729+0000 2024-09-14T05:28:13.684 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-09-14T05:28:13.683729+0000 2024-09-14T05:28:13.684 INFO:teuthology.orchestra.run.smithi071.stdout:min_mon_release 17 (quincy) 2024-09-14T05:28:13.684 INFO:teuthology.orchestra.run.smithi071.stdout:election_strategy: 1 2024-09-14T05:28:13.684 INFO:teuthology.orchestra.run.smithi071.stdout:0: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.a 2024-09-14T05:28:13.684 INFO:teuthology.orchestra.run.smithi071.stdout:1: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-09-14T05:28:13.684 INFO:teuthology.orchestra.run.smithi071.stdout:2: [v2:172.21.15.71:3301/0,v1:172.21.15.71:6790/0] mon.c 2024-09-14T05:28:13.684 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-14T05:28:13.686 DEBUG:teuthology.orchestra.run.smithi071:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-14T05:28:13.700 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1127be44-5f14-4e7d-ad23-f608afe154f5... 2024-09-14T05:28:13.701 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T05:28:13.743 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T05:28:13.783 INFO:teuthology.orchestra.run.smithi071.stdout:[global] 2024-09-14T05:28:13.783 INFO:teuthology.orchestra.run.smithi071.stdout: chdir = "" 2024-09-14T05:28:13.783 INFO:teuthology.orchestra.run.smithi071.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T05:28:13.783 INFO:teuthology.orchestra.run.smithi071.stdout: auth supported = cephx 2024-09-14T05:28:13.783 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.783 INFO:teuthology.orchestra.run.smithi071.stdout: filestore xattr use omap = true 2024-09-14T05:28:13.783 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.783 INFO:teuthology.orchestra.run.smithi071.stdout: mon clock drift allowed = 1.000 2024-09-14T05:28:13.784 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.784 INFO:teuthology.orchestra.run.smithi071.stdout: osd crush chooseleaf type = 0 2024-09-14T05:28:13.784 INFO:teuthology.orchestra.run.smithi071.stdout: auth debug = true 2024-09-14T05:28:13.784 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.784 INFO:teuthology.orchestra.run.smithi071.stdout: ms die on old message = true 2024-09-14T05:28:13.784 INFO:teuthology.orchestra.run.smithi071.stdout: ms die on bug = true 2024-09-14T05:28:13.784 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.784 INFO:teuthology.orchestra.run.smithi071.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T05:28:13.784 INFO:teuthology.orchestra.run.smithi071.stdout: mon pg warn max object skew = 0 2024-09-14T05:28:13.784 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.784 INFO:teuthology.orchestra.run.smithi071.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T05:28:13.784 INFO:teuthology.orchestra.run.smithi071.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T05:28:13.784 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.784 INFO:teuthology.orchestra.run.smithi071.stdout: osd pool default size = 2 2024-09-14T05:28:13.785 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.785 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd allow primary affinity = true 2024-09-14T05:28:13.785 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd allow pg remap = true 2024-09-14T05:28:13.785 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on legacy crush tunables = false 2024-09-14T05:28:13.785 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on crush straw calc version zero = false 2024-09-14T05:28:13.785 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on no sortbitwise = false 2024-09-14T05:28:13.785 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on osd down out interval zero = false 2024-09-14T05:28:13.785 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on too few osds = false 2024-09-14T05:28:13.785 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T05:28:13.785 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T05:28:13.785 INFO:teuthology.orchestra.run.smithi071.stdout: mon_allow_pool_size_one = true 2024-09-14T05:28:13.785 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.785 INFO:teuthology.orchestra.run.smithi071.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T05:28:13.786 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.786 INFO:teuthology.orchestra.run.smithi071.stdout: osd default data pool replay window = 5 2024-09-14T05:28:13.786 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.786 INFO:teuthology.orchestra.run.smithi071.stdout: mon allow pool delete = true 2024-09-14T05:28:13.786 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.786 INFO:teuthology.orchestra.run.smithi071.stdout: mon cluster log file level = debug 2024-09-14T05:28:13.786 INFO:teuthology.orchestra.run.smithi071.stdout: debug asserts on shutdown = true 2024-09-14T05:28:13.786 INFO:teuthology.orchestra.run.smithi071.stdout: mon health detail to clog = false 2024-09-14T05:28:13.786 INFO:teuthology.orchestra.run.smithi071.stdout: mon host = "172.21.15.71,[v2:172.21.15.71:3301,v1:172.21.15.71:6790],172.21.15.150" 2024-09-14T05:28:13.786 INFO:teuthology.orchestra.run.smithi071.stdout: mon client directed command retry = 5 2024-09-14T05:28:13.786 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on pool no app = False 2024-09-14T05:28:13.786 INFO:teuthology.orchestra.run.smithi071.stdout: ms inject socket failures = 5000 2024-09-14T05:28:13.786 INFO:teuthology.orchestra.run.smithi071.stdout: fsid = 1127be44-5f14-4e7d-ad23-f608afe154f5 2024-09-14T05:28:13.786 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.787 INFO:teuthology.orchestra.run.smithi071.stdout:[osd] 2024-09-14T05:28:13.787 INFO:teuthology.orchestra.run.smithi071.stdout: osd journal size = 100 2024-09-14T05:28:13.787 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.787 INFO:teuthology.orchestra.run.smithi071.stdout: osd scrub load threshold = 5.0 2024-09-14T05:28:13.787 INFO:teuthology.orchestra.run.smithi071.stdout: osd scrub max interval = 600 2024-09-14T05:28:13.787 INFO:teuthology.orchestra.run.smithi071.stdout: osd mclock profile = high_recovery_ops 2024-09-14T05:28:13.787 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.787 INFO:teuthology.orchestra.run.smithi071.stdout: osd recover clone overlap = true 2024-09-14T05:28:13.787 INFO:teuthology.orchestra.run.smithi071.stdout: osd recovery max chunk = 1048576 2024-09-14T05:28:13.787 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.787 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug shutdown = true 2024-09-14T05:28:13.787 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug op order = true 2024-09-14T05:28:13.787 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug verify stray on activate = true 2024-09-14T05:28:13.787 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug trim objects = true 2024-09-14T05:28:13.788 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.788 INFO:teuthology.orchestra.run.smithi071.stdout: osd open classes on start = true 2024-09-14T05:28:13.788 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug pg log writeout = true 2024-09-14T05:28:13.788 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.788 INFO:teuthology.orchestra.run.smithi071.stdout: osd deep scrub update digest min age = 30 2024-09-14T05:28:13.788 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.788 INFO:teuthology.orchestra.run.smithi071.stdout: osd map max advance = 10 2024-09-14T05:28:13.788 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.788 INFO:teuthology.orchestra.run.smithi071.stdout: journal zero on create = true 2024-09-14T05:28:13.788 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.788 INFO:teuthology.orchestra.run.smithi071.stdout: filestore ondisk finisher threads = 3 2024-09-14T05:28:13.788 INFO:teuthology.orchestra.run.smithi071.stdout: filestore apply finisher threads = 3 2024-09-14T05:28:13.788 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.788 INFO:teuthology.orchestra.run.smithi071.stdout: bdev debug aio = true 2024-09-14T05:28:13.789 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug misdirected ops = true 2024-09-14T05:28:13.789 INFO:teuthology.orchestra.run.smithi071.stdout: bdev async discard = True 2024-09-14T05:28:13.789 INFO:teuthology.orchestra.run.smithi071.stdout: bdev enable discard = True 2024-09-14T05:28:13.789 INFO:teuthology.orchestra.run.smithi071.stdout: bluestore block size = 96636764160 2024-09-14T05:28:13.789 INFO:teuthology.orchestra.run.smithi071.stdout: bluestore fsck on mount = True 2024-09-14T05:28:13.789 INFO:teuthology.orchestra.run.smithi071.stdout: debug bluefs = 1/20 2024-09-14T05:28:13.789 INFO:teuthology.orchestra.run.smithi071.stdout: debug bluestore = 1/20 2024-09-14T05:28:13.789 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-09-14T05:28:13.789 INFO:teuthology.orchestra.run.smithi071.stdout: debug osd = 20 2024-09-14T05:28:13.789 INFO:teuthology.orchestra.run.smithi071.stdout: debug rocksdb = 4/10 2024-09-14T05:28:13.789 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T05:28:13.789 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd full ratio = 0.9 2024-09-14T05:28:13.789 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd nearfull ratio = 0.8 2024-09-14T05:28:13.790 INFO:teuthology.orchestra.run.smithi071.stdout: osd blocked scrub grace period = 3600 2024-09-14T05:28:13.790 INFO:teuthology.orchestra.run.smithi071.stdout: osd failsafe full ratio = 0.95 2024-09-14T05:28:13.790 INFO:teuthology.orchestra.run.smithi071.stdout: osd max markdown count = 1000 2024-09-14T05:28:13.790 INFO:teuthology.orchestra.run.smithi071.stdout: osd memory target = 1610612736 2024-09-14T05:28:13.790 INFO:teuthology.orchestra.run.smithi071.stdout: osd objectstore = bluestore 2024-09-14T05:28:13.790 INFO:teuthology.orchestra.run.smithi071.stdout: osd shutdown pgref assert = True 2024-09-14T05:28:13.790 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.790 INFO:teuthology.orchestra.run.smithi071.stdout:[mgr] 2024-09-14T05:28:13.790 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-09-14T05:28:13.790 INFO:teuthology.orchestra.run.smithi071.stdout: debug mgr = 20 2024-09-14T05:28:13.790 INFO:teuthology.orchestra.run.smithi071.stdout: debug mon = 20 2024-09-14T05:28:13.790 INFO:teuthology.orchestra.run.smithi071.stdout: debug auth = 20 2024-09-14T05:28:13.790 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min pgs per osd = 4 2024-09-14T05:28:13.791 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min bytes per osd = 10 2024-09-14T05:28:13.791 INFO:teuthology.orchestra.run.smithi071.stdout: mgr/telemetry/nag = false 2024-09-14T05:28:13.791 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.791 INFO:teuthology.orchestra.run.smithi071.stdout:[mon] 2024-09-14T05:28:13.791 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-09-14T05:28:13.791 INFO:teuthology.orchestra.run.smithi071.stdout: debug mon = 20 2024-09-14T05:28:13.791 INFO:teuthology.orchestra.run.smithi071.stdout: debug paxos = 20 2024-09-14T05:28:13.791 INFO:teuthology.orchestra.run.smithi071.stdout: debug auth = 20 2024-09-14T05:28:13.791 INFO:teuthology.orchestra.run.smithi071.stdout: mon data avail warn = 5 2024-09-14T05:28:13.791 INFO:teuthology.orchestra.run.smithi071.stdout: mon mgr mkfs grace = 240 2024-09-14T05:28:13.791 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min pgs per osd = 4 2024-09-14T05:28:13.791 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd reporter subtree level = osd 2024-09-14T05:28:13.791 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd prime pg temp = true 2024-09-14T05:28:13.792 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min bytes per osd = 10 2024-09-14T05:28:13.792 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.792 INFO:teuthology.orchestra.run.smithi071.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T05:28:13.792 INFO:teuthology.orchestra.run.smithi071.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T05:28:13.792 INFO:teuthology.orchestra.run.smithi071.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T05:28:13.792 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.792 INFO:teuthology.orchestra.run.smithi071.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T05:28:13.792 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T05:28:13.792 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T05:28:13.792 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.792 INFO:teuthology.orchestra.run.smithi071.stdout: # 1m isn't quite enough 2024-09-14T05:28:13.792 INFO:teuthology.orchestra.run.smithi071.stdout: mon_down_mkfs_grace = 2m 2024-09-14T05:28:13.792 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.792 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_filestore_osds = false 2024-09-14T05:28:13.793 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:13.793 INFO:teuthology.orchestra.run.smithi071.stdout:[client] 2024-09-14T05:28:13.793 INFO:teuthology.orchestra.run.smithi071.stdout: rgw cache enabled = true 2024-09-14T05:28:13.793 INFO:teuthology.orchestra.run.smithi071.stdout: rgw enable ops log = true 2024-09-14T05:28:13.793 INFO:teuthology.orchestra.run.smithi071.stdout: rgw enable usage log = true 2024-09-14T05:28:13.793 INFO:teuthology.orchestra.run.smithi071.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T05:28:13.793 INFO:teuthology.orchestra.run.smithi071.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T05:28:13.793 INFO:teuthology.orchestra.run.smithi071.stdout: rbd cache = True 2024-09-14T05:28:13.793 INFO:teuthology.orchestra.run.smithi071.stdout: rbd cache policy = writearound 2024-09-14T05:28:13.793 INFO:teuthology.orchestra.run.smithi071.stdout:[mon.a] 2024-09-14T05:28:13.793 INFO:teuthology.orchestra.run.smithi071.stdout:[mon.c] 2024-09-14T05:28:13.793 INFO:teuthology.orchestra.run.smithi071.stdout:[mon.b] 2024-09-14T05:28:13.794 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2024-09-14T05:28:13.794 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2024-09-14T05:28:13.794 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T05:28:13.794 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2024-09-14T05:28:13.794 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.794 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2024-09-14T05:28:13.794 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.794 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2024-09-14T05:28:13.794 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.794 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2024-09-14T05:28:13.795 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2024-09-14T05:28:13.795 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.795 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2024-09-14T05:28:13.795 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2024-09-14T05:28:13.795 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.795 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T05:28:13.795 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2024-09-14T05:28:13.795 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.795 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T05:28:13.795 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T05:28:13.795 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.795 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2024-09-14T05:28:13.795 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.795 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2024-09-14T05:28:13.795 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2024-09-14T05:28:13.795 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2024-09-14T05:28:13.796 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2024-09-14T05:28:13.796 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2024-09-14T05:28:13.796 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2024-09-14T05:28:13.796 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2024-09-14T05:28:13.796 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T05:28:13.796 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T05:28:13.796 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2024-09-14T05:28:13.796 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.796 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T05:28:13.796 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.796 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2024-09-14T05:28:13.796 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.796 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2024-09-14T05:28:13.796 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.796 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2024-09-14T05:28:13.797 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2024-09-14T05:28:13.797 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2024-09-14T05:28:13.797 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = "172.21.15.71,[v2:172.21.15.71:3301,v1:172.21.15.71:6790],172.21.15.150" 2024-09-14T05:28:13.797 INFO:teuthology.orchestra.run.smithi150.stdout: mon client directed command retry = 5 2024-09-14T05:28:13.797 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on pool no app = False 2024-09-14T05:28:13.797 INFO:teuthology.orchestra.run.smithi150.stdout: ms inject socket failures = 5000 2024-09-14T05:28:13.797 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = 1127be44-5f14-4e7d-ad23-f608afe154f5 2024-09-14T05:28:13.797 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.797 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2024-09-14T05:28:13.797 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2024-09-14T05:28:13.797 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.797 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2024-09-14T05:28:13.797 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 600 2024-09-14T05:28:13.797 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock profile = high_recovery_ops 2024-09-14T05:28:13.798 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.798 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2024-09-14T05:28:13.798 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2024-09-14T05:28:13.798 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.798 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2024-09-14T05:28:13.798 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2024-09-14T05:28:13.798 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2024-09-14T05:28:13.798 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug trim objects = true 2024-09-14T05:28:13.798 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.798 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2024-09-14T05:28:13.798 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2024-09-14T05:28:13.798 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.798 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2024-09-14T05:28:13.799 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.799 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2024-09-14T05:28:13.799 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.799 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2024-09-14T05:28:13.799 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.799 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2024-09-14T05:28:13.799 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2024-09-14T05:28:13.799 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.799 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2024-09-14T05:28:13.799 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2024-09-14T05:28:13.799 INFO:teuthology.orchestra.run.smithi150.stdout: bdev async discard = True 2024-09-14T05:28:13.799 INFO:teuthology.orchestra.run.smithi150.stdout: bdev enable discard = True 2024-09-14T05:28:13.799 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2024-09-14T05:28:13.799 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2024-09-14T05:28:13.800 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 1/20 2024-09-14T05:28:13.800 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 1/20 2024-09-14T05:28:13.800 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-09-14T05:28:13.800 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2024-09-14T05:28:13.800 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 4/10 2024-09-14T05:28:13.800 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T05:28:13.800 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2024-09-14T05:28:13.800 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2024-09-14T05:28:13.800 INFO:teuthology.orchestra.run.smithi150.stdout: osd blocked scrub grace period = 3600 2024-09-14T05:28:13.800 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2024-09-14T05:28:13.800 INFO:teuthology.orchestra.run.smithi150.stdout: osd max markdown count = 1000 2024-09-14T05:28:13.800 INFO:teuthology.orchestra.run.smithi150.stdout: osd memory target = 1610612736 2024-09-14T05:28:13.800 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2024-09-14T05:28:13.801 INFO:teuthology.orchestra.run.smithi150.stdout: osd shutdown pgref assert = True 2024-09-14T05:28:13.801 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.801 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2024-09-14T05:28:13.801 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-09-14T05:28:13.801 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2024-09-14T05:28:13.801 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-09-14T05:28:13.801 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-09-14T05:28:13.801 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-09-14T05:28:13.801 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-09-14T05:28:13.801 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2024-09-14T05:28:13.801 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.801 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2024-09-14T05:28:13.801 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-09-14T05:28:13.801 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-09-14T05:28:13.801 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2024-09-14T05:28:13.802 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-09-14T05:28:13.802 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2024-09-14T05:28:13.802 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2024-09-14T05:28:13.802 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-09-14T05:28:13.802 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2024-09-14T05:28:13.802 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2024-09-14T05:28:13.802 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-09-14T05:28:13.802 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.802 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T05:28:13.802 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T05:28:13.802 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T05:28:13.802 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.802 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T05:28:13.802 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T05:28:13.802 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T05:28:13.802 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.803 INFO:teuthology.orchestra.run.smithi150.stdout: # 1m isn't quite enough 2024-09-14T05:28:13.803 INFO:teuthology.orchestra.run.smithi150.stdout: mon_down_mkfs_grace = 2m 2024-09-14T05:28:13.803 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.803 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_filestore_osds = false 2024-09-14T05:28:13.803 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-14T05:28:13.803 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2024-09-14T05:28:13.803 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2024-09-14T05:28:13.803 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2024-09-14T05:28:13.803 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2024-09-14T05:28:13.803 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T05:28:13.803 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T05:28:13.803 INFO:teuthology.orchestra.run.smithi150.stdout: rbd cache = True 2024-09-14T05:28:13.803 INFO:teuthology.orchestra.run.smithi150.stdout: rbd cache policy = writearound 2024-09-14T05:28:13.803 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2024-09-14T05:28:13.804 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.c] 2024-09-14T05:28:13.804 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.b] 2024-09-14T05:28:13.805 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-14T05:28:13.806 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-14T05:28:13.899 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-14T05:28:13.899 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:13.899 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-14T05:28:13.915 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:13.915 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-14T05:28:13.969 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi071.front.sepia.ceph.com 2024-09-14T05:28:13.970 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:13.970 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T05:28:13.970 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T05:28:14.044 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:14.044 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T05:28:14.099 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi150.front.sepia.ceph.com 2024-09-14T05:28:14.099 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:28:14.099 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T05:28:14.099 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T05:28:14.134 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:28:14.134 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T05:28:14.190 INFO:tasks.ceph:Setting up mon nodes... 2024-09-14T05:28:14.190 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-14T05:28:14.190 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-14T05:28:14.242 INFO:teuthology.orchestra.run.smithi071.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-14T05:28:14.244 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-14T05:28:14.305 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-14T05:28:14.308 INFO:tasks.ceph:Setting up mds nodes... 2024-09-14T05:28:14.308 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-14T05:28:14.308 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-14T05:28:14.351 INFO:teuthology.orchestra.run.smithi071.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-14T05:28:14.363 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-14T05:28:14.404 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-14T05:28:14.418 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-14T05:28:14.418 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi071.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-14T05:28:14.418 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-14T05:28:14.442 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T05:28:14.442 INFO:tasks.ceph:role: osd.0 2024-09-14T05:28:14.442 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-14T05:28:14.443 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-14T05:28:14.514 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T05:28:14.514 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T05:28:14.514 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T05:28:14.514 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T05:28:14.514 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T05:28:14.514 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2024-09-14T05:28:14.514 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T05:28:14.515 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T05:28:14.515 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T05:28:14.515 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T05:28:15.032 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2024-09-14T05:28:15.034 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2024-09-14T05:28:15.035 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-14T05:28:15.169 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-14T05:28:15.201 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-14T05:28:15.265 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T05:28:15.265 INFO:tasks.ceph:role: osd.1 2024-09-14T05:28:15.266 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-14T05:28:15.266 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-14T05:28:15.337 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T05:28:15.337 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T05:28:15.337 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T05:28:15.337 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T05:28:15.338 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T05:28:15.338 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2024-09-14T05:28:15.338 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T05:28:15.338 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T05:28:15.338 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T05:28:15.338 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T05:28:15.786 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2024-09-14T05:28:15.788 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2024-09-14T05:28:15.788 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-14T05:28:15.879 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-14T05:28:15.908 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-14T05:28:15.972 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T05:28:15.972 INFO:tasks.ceph:role: osd.2 2024-09-14T05:28:15.972 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-14T05:28:15.973 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-14T05:28:16.045 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T05:28:16.045 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T05:28:16.045 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T05:28:16.045 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T05:28:16.045 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T05:28:16.046 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2024-09-14T05:28:16.046 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T05:28:16.046 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T05:28:16.046 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T05:28:16.046 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T05:28:16.544 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2024-09-14T05:28:16.546 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2024-09-14T05:28:16.546 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-14T05:28:16.680 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-14T05:28:16.710 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-14T05:28:16.775 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T05:28:16.775 INFO:tasks.ceph:role: osd.3 2024-09-14T05:28:16.775 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-14T05:28:16.775 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-14T05:28:16.847 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T05:28:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T05:28:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T05:28:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T05:28:16.847 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T05:28:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2024-09-14T05:28:16.847 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T05:28:16.847 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T05:28:16.847 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T05:28:16.848 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T05:28:17.366 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2024-09-14T05:28:17.369 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2024-09-14T05:28:17.369 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-14T05:28:17.507 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-14T05:28:17.539 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T05:28:17.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-14T05:28:17.637+0000 7fdf93226640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-14T05:28:17.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-14T05:28:17.638+0000 7fdf93226640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-14T05:28:17.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-14T05:28:17.638+0000 7fdf93226640 -1 bdev(0x560bb5e49800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-14T05:28:17.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-14T05:28:17.638+0000 7fdf93226640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-14T05:28:21.295 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-14T05:28:21.325 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T05:28:21.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-14T05:28:21.417+0000 7f95881b9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-14T05:28:21.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-14T05:28:21.417+0000 7f95881b9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-14T05:28:21.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-14T05:28:21.417+0000 7f95881b9640 -1 bdev(0x556ccbb75800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-14T05:28:21.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-14T05:28:21.417+0000 7f95881b9640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-14T05:28:24.327 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-14T05:28:24.356 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T05:28:24.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-14T05:28:24.445+0000 7fc9d33a8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-14T05:28:24.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-14T05:28:24.445+0000 7fc9d33a8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-14T05:28:24.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-14T05:28:24.445+0000 7fc9d33a8640 -1 bdev(0x563ac39df800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-14T05:28:24.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-14T05:28:24.445+0000 7fc9d33a8640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-14T05:28:27.606 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-14T05:28:27.635 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T05:28:27.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-14T05:28:27.725+0000 7f630142c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-14T05:28:27.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-14T05:28:27.725+0000 7f630142c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-14T05:28:27.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-14T05:28:27.725+0000 7f630142c640 -1 bdev(0x55b2a0e05800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-14T05:28:27.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-14T05:28:27.726+0000 7f630142c640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-14T05:28:30.639 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-14T05:28:30.668 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-14T05:28:30.698 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T05:28:30.698 INFO:tasks.ceph:role: osd.4 2024-09-14T05:28:30.698 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-14T05:28:30.699 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-14T05:28:30.775 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T05:28:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T05:28:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T05:28:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T05:28:30.775 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T05:28:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-09-14T05:28:30.776 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T05:28:30.776 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T05:28:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T05:28:30.776 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T05:28:31.279 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-09-14T05:28:31.281 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-09-14T05:28:31.281 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-14T05:28:31.418 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-14T05:28:31.449 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-14T05:28:31.517 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T05:28:31.517 INFO:tasks.ceph:role: osd.5 2024-09-14T05:28:31.517 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-14T05:28:31.517 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-14T05:28:31.590 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T05:28:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T05:28:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T05:28:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T05:28:31.590 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T05:28:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-09-14T05:28:31.590 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T05:28:31.590 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T05:28:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T05:28:31.590 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T05:28:32.045 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-09-14T05:28:32.047 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-09-14T05:28:32.047 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-14T05:28:32.142 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-14T05:28:32.171 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-14T05:28:32.236 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T05:28:32.236 INFO:tasks.ceph:role: osd.6 2024-09-14T05:28:32.237 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-14T05:28:32.237 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-14T05:28:32.309 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T05:28:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T05:28:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T05:28:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T05:28:32.311 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T05:28:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-09-14T05:28:32.311 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T05:28:32.311 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T05:28:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T05:28:32.311 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T05:28:32.795 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-09-14T05:28:32.797 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-09-14T05:28:32.797 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-14T05:28:32.949 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-14T05:28:32.981 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-14T05:28:33.046 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T05:28:33.047 INFO:tasks.ceph:role: osd.7 2024-09-14T05:28:33.047 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-14T05:28:33.047 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-14T05:28:33.119 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T05:28:33.119 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T05:28:33.119 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T05:28:33.119 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T05:28:33.119 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T05:28:33.119 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-09-14T05:28:33.119 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T05:28:33.119 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T05:28:33.119 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T05:28:33.119 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T05:28:33.638 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-09-14T05:28:33.641 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-09-14T05:28:33.641 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-14T05:28:33.788 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-14T05:28:33.822 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T05:28:33.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-14T05:28:33.931+0000 7fd1d4611640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-14T05:28:33.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-14T05:28:33.933+0000 7fd1d4611640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-14T05:28:33.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-14T05:28:33.933+0000 7fd1d4611640 -1 bdev(0x5598063a3800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-14T05:28:33.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-14T05:28:33.933+0000 7fd1d4611640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-14T05:28:37.090 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-14T05:28:37.119 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T05:28:37.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-14T05:28:37.210+0000 7f240a1c5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-14T05:28:37.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-14T05:28:37.211+0000 7f240a1c5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-14T05:28:37.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-14T05:28:37.211+0000 7f240a1c5640 -1 bdev(0x5627b2829800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-14T05:28:37.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-14T05:28:37.211+0000 7f240a1c5640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-14T05:28:40.615 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-14T05:28:40.643 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T05:28:40.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-14T05:28:40.733+0000 7f870b815640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-14T05:28:40.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-14T05:28:40.733+0000 7f870b815640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-14T05:28:40.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-14T05:28:40.733+0000 7f870b815640 -1 bdev(0x55a956969800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-14T05:28:40.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-14T05:28:40.733+0000 7f870b815640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-14T05:28:43.886 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-14T05:28:43.916 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T05:28:44.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-14T05:28:44.007+0000 7f8af63bb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-14T05:28:44.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-14T05:28:44.007+0000 7f8af63bb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-14T05:28:44.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-14T05:28:44.007+0000 7f8af63bb640 -1 bdev(0x558deefdb800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-14T05:28:44.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-14T05:28:44.007+0000 7f8af63bb640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-14T05:28:46.928 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-14T05:28:46.957 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-14T05:28:46.957 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:46.957 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-14T05:28:46.987 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:46.987 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-14T05:28:47.056 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:47.056 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-14T05:28:47.123 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:47.123 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-14T05:28:47.189 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:47.189 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-14T05:28:47.255 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:28:47.255 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-14T05:28:47.285 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:28:47.285 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-14T05:28:47.354 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:28:47.354 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-14T05:28:47.422 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:28:47.422 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-14T05:28:47.489 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:28:47.489 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-14T05:28:47.556 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:47.556 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-14T05:28:47.573 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:28:47.573 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-14T05:28:47.612 INFO:tasks.ceph:Adding keys to all mons... 2024-09-14T05:28:47.612 DEBUG:teuthology.orchestra.run.smithi071:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-14T05:28:47.616 DEBUG:teuthology.orchestra.run.smithi150:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-14T05:28:47.656 INFO:teuthology.orchestra.run.smithi071.stdout:[mgr.y] 2024-09-14T05:28:47.656 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQDuHuVmrGlvDhAAAgE37wqWod0/jMEi1ckXOw== 2024-09-14T05:28:47.656 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.0] 2024-09-14T05:28:47.656 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQDxHuVmj20UJhAA9WNC0pIz84uWhoFKd3ruxA== 2024-09-14T05:28:47.656 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.1] 2024-09-14T05:28:47.656 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQD1HuVmbSLvGBAA4iPWoWT6OP86lkCAC+jsiA== 2024-09-14T05:28:47.656 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.2] 2024-09-14T05:28:47.657 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQD4HuVmwtKbGhAA1TmT0yU1aH4aes4SUJEc8w== 2024-09-14T05:28:47.657 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.3] 2024-09-14T05:28:47.657 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQD7HuVme81OKxAAIxTVGSP1bYJJ2aC1lKdZwA== 2024-09-14T05:28:47.657 INFO:teuthology.orchestra.run.smithi071.stdout:[mgr.x] 2024-09-14T05:28:47.657 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQDuHuVmfDQ6EhAA2eOanr3AKRzwcksheODV1g== 2024-09-14T05:28:47.657 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.4] 2024-09-14T05:28:47.657 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQABH+VmWUqXNxAAScvdcqyW5U5sWogDLHwYHQ== 2024-09-14T05:28:47.657 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.5] 2024-09-14T05:28:47.657 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQAFH+Vm2iOeDBAAMYnccq9XxtPJAJJacOFLqg== 2024-09-14T05:28:47.657 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.6] 2024-09-14T05:28:47.657 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQAIH+Vm4jzAKxAAeMzwdPemHLP+FVdnJDc68w== 2024-09-14T05:28:47.657 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.7] 2024-09-14T05:28:47.657 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQAMH+VmWJp8ABAAlwGEUJKx6qx9rdCpC6vwLg== 2024-09-14T05:28:47.658 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-09-14T05:28:47.658 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQDuHuVmX3/xFBAAMozyehEVAvgoEG9qFac+lA== 2024-09-14T05:28:47.658 INFO:teuthology.orchestra.run.smithi071.stdout:[client.1] 2024-09-14T05:28:47.658 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQDuHuVmP2cjGBAABACwhr4qhBpkfCSGY3ywaQ== 2024-09-14T05:28:47.676 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.y] 2024-09-14T05:28:47.676 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDuHuVmrGlvDhAAAgE37wqWod0/jMEi1ckXOw== 2024-09-14T05:28:47.676 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.0] 2024-09-14T05:28:47.676 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDxHuVmj20UJhAA9WNC0pIz84uWhoFKd3ruxA== 2024-09-14T05:28:47.676 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.1] 2024-09-14T05:28:47.677 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQD1HuVmbSLvGBAA4iPWoWT6OP86lkCAC+jsiA== 2024-09-14T05:28:47.677 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.2] 2024-09-14T05:28:47.677 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQD4HuVmwtKbGhAA1TmT0yU1aH4aes4SUJEc8w== 2024-09-14T05:28:47.677 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.3] 2024-09-14T05:28:47.677 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQD7HuVme81OKxAAIxTVGSP1bYJJ2aC1lKdZwA== 2024-09-14T05:28:47.677 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.x] 2024-09-14T05:28:47.677 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDuHuVmfDQ6EhAA2eOanr3AKRzwcksheODV1g== 2024-09-14T05:28:47.677 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.4] 2024-09-14T05:28:47.677 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQABH+VmWUqXNxAAScvdcqyW5U5sWogDLHwYHQ== 2024-09-14T05:28:47.677 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.5] 2024-09-14T05:28:47.677 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAFH+Vm2iOeDBAAMYnccq9XxtPJAJJacOFLqg== 2024-09-14T05:28:47.677 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.6] 2024-09-14T05:28:47.677 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAIH+Vm4jzAKxAAeMzwdPemHLP+FVdnJDc68w== 2024-09-14T05:28:47.678 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.7] 2024-09-14T05:28:47.678 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAMH+VmWJp8ABAAlwGEUJKx6qx9rdCpC6vwLg== 2024-09-14T05:28:47.678 INFO:teuthology.orchestra.run.smithi150.stdout:[client.0] 2024-09-14T05:28:47.678 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDuHuVmX3/xFBAAMozyehEVAvgoEG9qFac+lA== 2024-09-14T05:28:47.678 INFO:teuthology.orchestra.run.smithi150.stdout:[client.1] 2024-09-14T05:28:47.678 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDuHuVmP2cjGBAABACwhr4qhBpkfCSGY3ywaQ== 2024-09-14T05:28:47.679 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T05:28:47.701 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T05:28:47.770 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T05:28:47.773 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T05:28:47.824 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T05:28:47.826 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T05:28:47.877 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T05:28:47.880 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T05:28:47.930 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T05:28:47.933 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T05:28:47.984 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T05:28:47.986 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T05:28:48.037 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T05:28:48.039 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T05:28:48.091 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T05:28:48.093 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T05:28:48.144 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T05:28:48.147 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T05:28:48.197 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T05:28:48.199 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T05:28:48.250 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T05:28:48.252 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T05:28:48.306 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T05:28:48.309 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T05:28:48.364 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-14T05:28:48.364 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-14T05:28:48.388 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T05:28:48.483 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-14T05:28:48.508 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-14T05:28:48.572 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T05:28:48.668 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-14T05:28:48.693 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-14T05:28:48.719 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T05:28:48.818 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-14T05:28:48.843 DEBUG:teuthology.orchestra.run.smithi071:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-14T05:28:48.846 DEBUG:teuthology.orchestra.run.smithi150:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-14T05:28:48.899 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-14T05:28:48.899 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-14T05:28:48.899 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-14T05:28:48.903 INFO:tasks.ceph.mon.a:Started 2024-09-14T05:28:48.903 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-14T05:28:48.903 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-14T05:28:48.905 INFO:tasks.ceph.mon.c:Started 2024-09-14T05:28:48.905 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-14T05:28:48.905 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-14T05:28:48.941 INFO:tasks.ceph.mon.b:Started 2024-09-14T05:28:48.941 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-14T05:28:48.942 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-14T05:28:48.942 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-14T05:28:48.944 INFO:tasks.ceph.mgr.y:Started 2024-09-14T05:28:48.944 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-14T05:28:48.944 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-14T05:28:48.946 INFO:tasks.ceph.mgr.x:Started 2024-09-14T05:28:48.946 DEBUG:tasks.ceph:set 0 configs 2024-09-14T05:28:48.946 INFO:tasks.ceph:Setting crush tunables to default 2024-09-14T05:28:48.946 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph osd crush tunables default 2024-09-14T05:28:54.300 INFO:teuthology.orchestra.run.smithi071.stderr:adjusted tunables profile to default 2024-09-14T05:28:54.312 INFO:tasks.ceph:check_enable_crimson: False 2024-09-14T05:28:54.313 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-14T05:28:54.313 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:54.313 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-14T05:28:54.338 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:54.338 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-14T05:28:54.403 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:54.404 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-14T05:28:54.468 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-14T05:28:54.468 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-14T05:28:54.533 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:28:54.533 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-14T05:28:54.558 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:28:54.558 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-14T05:28:54.622 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:28:54.622 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-14T05:28:54.687 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-14T05:28:54.688 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-14T05:28:54.752 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new e7c7817f-4a65-4f66-9af5-d31d0a92cf30 0 2024-09-14T05:28:55.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-14T05:28:55.097+0000 7f7586c1c640 0 --2- 172.21.15.150:0/2815637176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75801401a0 0x7f75801d8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-14T05:28:55.177 INFO:teuthology.orchestra.run.smithi150.stdout:0 2024-09-14T05:28:55.188 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new e184604e-b973-4b7a-bba2-6f471753a0fd 1 2024-09-14T05:28:55.371 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-09-14T05:28:55.383 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new d8b70f09-5afb-4296-b8c9-719c172e0e70 2 2024-09-14T05:28:55.566 INFO:teuthology.orchestra.run.smithi150.stdout:2 2024-09-14T05:28:55.578 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c 3 2024-09-14T05:28:55.762 INFO:teuthology.orchestra.run.smithi150.stdout:3 2024-09-14T05:28:55.774 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 515cb7d6-16eb-4aff-973a-7f7ed49325ba 4 2024-09-14T05:28:55.959 INFO:teuthology.orchestra.run.smithi150.stdout:4 2024-09-14T05:28:55.970 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new c434b53a-04be-45b9-b89a-193654133950 5 2024-09-14T05:28:56.155 INFO:teuthology.orchestra.run.smithi150.stdout:5 2024-09-14T05:28:56.167 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 3d2f8405-9286-4b40-8acd-036f67373ac6 6 2024-09-14T05:28:56.347 INFO:teuthology.orchestra.run.smithi150.stdout:6 2024-09-14T05:28:56.359 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 6724f080-4dc3-465a-939f-b023985bf998 7 2024-09-14T05:28:56.541 INFO:teuthology.orchestra.run.smithi150.stdout:7 2024-09-14T05:28:56.549 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-14T05:28:56.549 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-14T05:28:56.551 INFO:tasks.ceph.osd.0:Started 2024-09-14T05:28:56.552 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-14T05:28:56.552 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-14T05:28:56.554 INFO:tasks.ceph.osd.1:Started 2024-09-14T05:28:56.554 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-14T05:28:56.554 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-14T05:28:56.556 INFO:tasks.ceph.osd.2:Started 2024-09-14T05:28:56.556 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-14T05:28:56.556 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-14T05:28:56.558 INFO:tasks.ceph.osd.3:Started 2024-09-14T05:28:56.558 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-14T05:28:56.558 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-14T05:28:56.560 INFO:tasks.ceph.osd.4:Started 2024-09-14T05:28:56.561 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-14T05:28:56.561 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-14T05:28:56.563 INFO:tasks.ceph.osd.5:Started 2024-09-14T05:28:56.563 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-14T05:28:56.563 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-14T05:28:56.565 INFO:tasks.ceph.osd.6:Started 2024-09-14T05:28:56.565 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-14T05:28:56.565 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-14T05:28:56.567 INFO:tasks.ceph.osd.7:Started 2024-09-14T05:28:56.568 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T05:28:56.761 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:28:56.761 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":10,"fsid":"1127be44-5f14-4e7d-ad23-f608afe154f5","created":"2024-09-14T05:28:54.203072+0000","modified":"2024-09-14T05:28:56.535048+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T05:28:56.535048+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e7c7817f-4a65-4f66-9af5-d31d0a92cf30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e184604e-b973-4b7a-bba2-6f471753a0fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d8b70f09-5afb-4296-b8c9-719c172e0e70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"515cb7d6-16eb-4aff-973a-7f7ed49325ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c434b53a-04be-45b9-b89a-193654133950","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3d2f8405-9286-4b40-8acd-036f67373ac6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6724f080-4dc3-465a-939f-b023985bf998","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T05:28:56.772 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-14T05:28:56.773 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-14T05:28:57.535 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:28:57.534+0000 7f961a20e640 -1 Falling back to public interface 2024-09-14T05:28:57.579 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T05:28:57.738 INFO:teuthology.misc.health.smithi071.stdout: 2024-09-14T05:28:57.738 INFO:teuthology.misc.health.smithi071.stdout:{"epoch":10,"fsid":"1127be44-5f14-4e7d-ad23-f608afe154f5","created":"2024-09-14T05:28:54.203072+0000","modified":"2024-09-14T05:28:56.535048+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T05:28:56.535048+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e7c7817f-4a65-4f66-9af5-d31d0a92cf30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e184604e-b973-4b7a-bba2-6f471753a0fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d8b70f09-5afb-4296-b8c9-719c172e0e70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"515cb7d6-16eb-4aff-973a-7f7ed49325ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c434b53a-04be-45b9-b89a-193654133950","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3d2f8405-9286-4b40-8acd-036f67373ac6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6724f080-4dc3-465a-939f-b023985bf998","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T05:28:57.748 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-14T05:28:57.996 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:28:57.995+0000 7fca45d43640 -1 Falling back to public interface 2024-09-14T05:28:58.017 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:28:58.015+0000 7f6473440640 -1 Falling back to public interface 2024-09-14T05:28:58.023 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:28:58.021+0000 7f3131820640 -1 Falling back to public interface 2024-09-14T05:28:58.266 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:28:58.265+0000 7f77d7ce2640 -1 Falling back to public interface 2024-09-14T05:28:58.291 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:28:58.291+0000 7f3cd4d23640 -1 Falling back to public interface 2024-09-14T05:28:58.508 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:28:58.507+0000 7f9c58219640 -1 Falling back to public interface 2024-09-14T05:28:58.530 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:28:58.529+0000 7fbc32c10640 -1 Falling back to public interface 2024-09-14T05:28:59.443 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-09-14T05:28:59.441+0000 7f2bdf852640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-14T05:28:59.444 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-09-14T05:28:59.442+0000 7f2bdf852640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-14T05:29:01.062 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:01.062+0000 7f961a20e640 -1 osd.6 0 log_to_monitors true 2024-09-14T05:29:01.758 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:01.757+0000 7fca45d43640 -1 osd.5 0 log_to_monitors true 2024-09-14T05:29:02.089 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:02.088+0000 7fbc32c10640 -1 osd.1 0 log_to_monitors true 2024-09-14T05:29:02.100 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:02.099+0000 7f6473440640 -1 osd.2 0 log_to_monitors true 2024-09-14T05:29:02.307 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:02.306+0000 7f77d7ce2640 -1 osd.4 0 log_to_monitors true 2024-09-14T05:29:02.314 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:02.314+0000 7f3131820640 -1 osd.3 0 log_to_monitors true 2024-09-14T05:29:02.352 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:02.351+0000 7f3cd4d23640 -1 osd.7 0 log_to_monitors true 2024-09-14T05:29:02.505 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:02.504+0000 7f9c58219640 -1 osd.0 0 log_to_monitors true 2024-09-14T05:29:04.545 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:04.544+0000 7f9615afe640 -1 osd.6 0 waiting for initial osdmap 2024-09-14T05:29:04.556 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T05:29:04.572 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:04.571+0000 7f960d91d640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T05:29:04.971 INFO:teuthology.misc.health.smithi071.stdout: 2024-09-14T05:29:04.971 INFO:teuthology.misc.health.smithi071.stdout:{"epoch":14,"fsid":"1127be44-5f14-4e7d-ad23-f608afe154f5","created":"2024-09-14T05:28:54.203072+0000","modified":"2024-09-14T05:29:04.450581+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T05:28:56.535048+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e7c7817f-4a65-4f66-9af5-d31d0a92cf30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e184604e-b973-4b7a-bba2-6f471753a0fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d8b70f09-5afb-4296-b8c9-719c172e0e70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"515cb7d6-16eb-4aff-973a-7f7ed49325ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c434b53a-04be-45b9-b89a-193654133950","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3d2f8405-9286-4b40-8acd-036f67373ac6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6724f080-4dc3-465a-939f-b023985bf998","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T05:29:04.987 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-14T05:29:05.248 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:05.247+0000 7f77d3c71640 -1 osd.4 0 waiting for initial osdmap 2024-09-14T05:29:05.263 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:05.262+0000 7fca424e6640 -1 osd.5 0 waiting for initial osdmap 2024-09-14T05:29:05.264 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:05.264+0000 7f77cba90640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T05:29:05.279 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:05.278+0000 7f3cd14c6640 -1 osd.7 0 waiting for initial osdmap 2024-09-14T05:29:05.281 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:05.281+0000 7fca39af1640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T05:29:05.298 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:05.298+0000 7f3cc8ad1640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T05:29:05.456 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:05.455+0000 7f312d7af640 -1 osd.3 0 waiting for initial osdmap 2024-09-14T05:29:05.465 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:05.464+0000 7f646f3cf640 -1 osd.2 0 waiting for initial osdmap 2024-09-14T05:29:05.475 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:05.474+0000 7f31255ce640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T05:29:05.481 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:05.481+0000 7f64671ee640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T05:29:05.489 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:05.488+0000 7fbc2ed12640 -1 osd.1 0 waiting for initial osdmap 2024-09-14T05:29:05.503 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:05.503+0000 7fbc2631d640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T05:29:05.753 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:05.752+0000 7f9c549bc640 -1 osd.0 0 waiting for initial osdmap 2024-09-14T05:29:05.764 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:05.763+0000 7f9c4bfc7640 -1 osd.0 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T05:29:11.794 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T05:29:12.082 INFO:teuthology.misc.health.smithi071.stdout: 2024-09-14T05:29:12.083 INFO:teuthology.misc.health.smithi071.stdout:{"epoch":18,"fsid":"1127be44-5f14-4e7d-ad23-f608afe154f5","created":"2024-09-14T05:28:54.203072+0000","modified":"2024-09-14T05:29:08.768979+0000","last_up_change":"2024-09-14T05:29:06.755531+0000","last_in_change":"2024-09-14T05:28:56.535048+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T05:29:06.452744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7c7817f-4a65-4f66-9af5-d31d0a92cf30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6819","nonce":2926109885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6821","nonce":2926109885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6825","nonce":2926109885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6823","nonce":2926109885}]},"public_addr":"172.21.15.71:6819/2926109885","cluster_addr":"172.21.15.71:6821/2926109885","heartbeat_back_addr":"172.21.15.71:6825/2926109885","heartbeat_front_addr":"172.21.15.71:6823/2926109885","state":["exists","up"]},{"osd":1,"uuid":"e184604e-b973-4b7a-bba2-6f471753a0fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6827","nonce":345649648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6829","nonce":345649648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6833","nonce":345649648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6831","nonce":345649648}]},"public_addr":"172.21.15.71:6827/345649648","cluster_addr":"172.21.15.71:6829/345649648","heartbeat_back_addr":"172.21.15.71:6833/345649648","heartbeat_front_addr":"172.21.15.71:6831/345649648","state":["exists","up"]},{"osd":2,"uuid":"d8b70f09-5afb-4296-b8c9-719c172e0e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6801","nonce":2721860691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6803","nonce":2721860691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6807","nonce":2721860691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6805","nonce":2721860691}]},"public_addr":"172.21.15.71:6801/2721860691","cluster_addr":"172.21.15.71:6803/2721860691","heartbeat_back_addr":"172.21.15.71:6807/2721860691","heartbeat_front_addr":"172.21.15.71:6805/2721860691","state":["exists","up"]},{"osd":3,"uuid":"26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6809","nonce":494607463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6811","nonce":494607463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6815","nonce":494607463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6813","nonce":494607463}]},"public_addr":"172.21.15.71:6809/494607463","cluster_addr":"172.21.15.71:6811/494607463","heartbeat_back_addr":"172.21.15.71:6815/494607463","heartbeat_front_addr":"172.21.15.71:6813/494607463","state":["exists","up"]},{"osd":4,"uuid":"515cb7d6-16eb-4aff-973a-7f7ed49325ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6817","nonce":2240133417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6819","nonce":2240133417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6823","nonce":2240133417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6821","nonce":2240133417}]},"public_addr":"172.21.15.150:6817/2240133417","cluster_addr":"172.21.15.150:6819/2240133417","heartbeat_back_addr":"172.21.15.150:6823/2240133417","heartbeat_front_addr":"172.21.15.150:6821/2240133417","state":["exists","up"]},{"osd":5,"uuid":"c434b53a-04be-45b9-b89a-193654133950","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6809","nonce":2393825223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6811","nonce":2393825223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6815","nonce":2393825223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6813","nonce":2393825223}]},"public_addr":"172.21.15.150:6809/2393825223","cluster_addr":"172.21.15.150:6811/2393825223","heartbeat_back_addr":"172.21.15.150:6815/2393825223","heartbeat_front_addr":"172.21.15.150:6813/2393825223","state":["exists","up"]},{"osd":6,"uuid":"3d2f8405-9286-4b40-8acd-036f67373ac6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6801","nonce":3235729544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6803","nonce":3235729544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6807","nonce":3235729544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6805","nonce":3235729544}]},"public_addr":"172.21.15.150:6801/3235729544","cluster_addr":"172.21.15.150:6803/3235729544","heartbeat_back_addr":"172.21.15.150:6807/3235729544","heartbeat_front_addr":"172.21.15.150:6805/3235729544","state":["exists","up"]},{"osd":7,"uuid":"6724f080-4dc3-465a-939f-b023985bf998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6825","nonce":2381001355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6827","nonce":2381001355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6831","nonce":2381001355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6829","nonce":2381001355}]},"public_addr":"172.21.15.150:6825/2381001355","cluster_addr":"172.21.15.150:6827/2381001355","heartbeat_back_addr":"172.21.15.150:6831/2381001355","heartbeat_front_addr":"172.21.15.150:6829/2381001355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.548803+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.075023+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.117036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.332113+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.330371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:02.753946+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:02.071186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.378781+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T05:29:12.094 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-14T05:29:12.094 INFO:tasks.ceph:Creating RBD pool 2024-09-14T05:29:12.094 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-14T05:29:12.824 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'rbd' created 2024-09-14T05:29:12.838 DEBUG:teuthology.orchestra.run.smithi071:> rbd --cluster ceph pool init rbd 2024-09-14T05:29:15.856 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-14T05:29:15.857 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-14T05:29:15.858 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-14T05:29:16.152 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:29:16.165 INFO:teuthology.orchestra.run.smithi071.stdout:[{"version":9,"timestamp":"2024-09-14T05:29:05.753942+0000","name":"","changes":[{"name":"osd.0/osd_mclock_max_capacity_iops_ssd","new_value":"5090.784296"}]}] 2024-09-14T05:29:16.165 INFO:tasks.ceph_manager:config epoch is 9 2024-09-14T05:29:16.165 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-14T05:29:16.165 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-14T05:29:16.165 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-14T05:29:16.499 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:29:16.509 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2042499113},{"type":"v1","addr":"172.21.15.71:6817","nonce":2042499113}]},"active_addr":"172.21.15.71:6817/2042499113","active_change":"2024-09-14T05:28:58.430354+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4102,"name":"x","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":852700406}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":1863085609}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":209369259}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":3279908902}]}]} 2024-09-14T05:29:16.514 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-14T05:29:16.514 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-14T05:29:16.514 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T05:29:16.798 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:29:16.798 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":22,"fsid":"1127be44-5f14-4e7d-ad23-f608afe154f5","created":"2024-09-14T05:28:54.203072+0000","modified":"2024-09-14T05:29:15.838400+0000","last_up_change":"2024-09-14T05:29:06.755531+0000","last_in_change":"2024-09-14T05:28:56.535048+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T05:29:06.452744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T05:29:12.380843+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7c7817f-4a65-4f66-9af5-d31d0a92cf30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6819","nonce":2926109885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6821","nonce":2926109885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6825","nonce":2926109885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6823","nonce":2926109885}]},"public_addr":"172.21.15.71:6819/2926109885","cluster_addr":"172.21.15.71:6821/2926109885","heartbeat_back_addr":"172.21.15.71:6825/2926109885","heartbeat_front_addr":"172.21.15.71:6823/2926109885","state":["exists","up"]},{"osd":1,"uuid":"e184604e-b973-4b7a-bba2-6f471753a0fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6827","nonce":345649648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6829","nonce":345649648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6833","nonce":345649648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6831","nonce":345649648}]},"public_addr":"172.21.15.71:6827/345649648","cluster_addr":"172.21.15.71:6829/345649648","heartbeat_back_addr":"172.21.15.71:6833/345649648","heartbeat_front_addr":"172.21.15.71:6831/345649648","state":["exists","up"]},{"osd":2,"uuid":"d8b70f09-5afb-4296-b8c9-719c172e0e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6801","nonce":2721860691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6803","nonce":2721860691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6807","nonce":2721860691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6805","nonce":2721860691}]},"public_addr":"172.21.15.71:6801/2721860691","cluster_addr":"172.21.15.71:6803/2721860691","heartbeat_back_addr":"172.21.15.71:6807/2721860691","heartbeat_front_addr":"172.21.15.71:6805/2721860691","state":["exists","up"]},{"osd":3,"uuid":"26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6809","nonce":494607463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6811","nonce":494607463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6815","nonce":494607463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6813","nonce":494607463}]},"public_addr":"172.21.15.71:6809/494607463","cluster_addr":"172.21.15.71:6811/494607463","heartbeat_back_addr":"172.21.15.71:6815/494607463","heartbeat_front_addr":"172.21.15.71:6813/494607463","state":["exists","up"]},{"osd":4,"uuid":"515cb7d6-16eb-4aff-973a-7f7ed49325ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6817","nonce":2240133417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6819","nonce":2240133417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6823","nonce":2240133417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6821","nonce":2240133417}]},"public_addr":"172.21.15.150:6817/2240133417","cluster_addr":"172.21.15.150:6819/2240133417","heartbeat_back_addr":"172.21.15.150:6823/2240133417","heartbeat_front_addr":"172.21.15.150:6821/2240133417","state":["exists","up"]},{"osd":5,"uuid":"c434b53a-04be-45b9-b89a-193654133950","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6809","nonce":2393825223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6811","nonce":2393825223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6815","nonce":2393825223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6813","nonce":2393825223}]},"public_addr":"172.21.15.150:6809/2393825223","cluster_addr":"172.21.15.150:6811/2393825223","heartbeat_back_addr":"172.21.15.150:6815/2393825223","heartbeat_front_addr":"172.21.15.150:6813/2393825223","state":["exists","up"]},{"osd":6,"uuid":"3d2f8405-9286-4b40-8acd-036f67373ac6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6801","nonce":3235729544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6803","nonce":3235729544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6807","nonce":3235729544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6805","nonce":3235729544}]},"public_addr":"172.21.15.150:6801/3235729544","cluster_addr":"172.21.15.150:6803/3235729544","heartbeat_back_addr":"172.21.15.150:6807/3235729544","heartbeat_front_addr":"172.21.15.150:6805/3235729544","state":["exists","up"]},{"osd":7,"uuid":"6724f080-4dc3-465a-939f-b023985bf998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6825","nonce":2381001355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6827","nonce":2381001355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6831","nonce":2381001355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6829","nonce":2381001355}]},"public_addr":"172.21.15.150:6825/2381001355","cluster_addr":"172.21.15.150:6827/2381001355","heartbeat_back_addr":"172.21.15.150:6831/2381001355","heartbeat_front_addr":"172.21.15.150:6829/2381001355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.548803+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.075023+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.117036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.332113+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.330371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:02.753946+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:02.071186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.378781+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T05:29:16.810 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-14T05:29:16.811 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T05:29:17.094 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:29:17.095 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":22,"fsid":"1127be44-5f14-4e7d-ad23-f608afe154f5","created":"2024-09-14T05:28:54.203072+0000","modified":"2024-09-14T05:29:15.838400+0000","last_up_change":"2024-09-14T05:29:06.755531+0000","last_in_change":"2024-09-14T05:28:56.535048+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T05:29:06.452744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T05:29:12.380843+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7c7817f-4a65-4f66-9af5-d31d0a92cf30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6819","nonce":2926109885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6821","nonce":2926109885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6825","nonce":2926109885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6823","nonce":2926109885}]},"public_addr":"172.21.15.71:6819/2926109885","cluster_addr":"172.21.15.71:6821/2926109885","heartbeat_back_addr":"172.21.15.71:6825/2926109885","heartbeat_front_addr":"172.21.15.71:6823/2926109885","state":["exists","up"]},{"osd":1,"uuid":"e184604e-b973-4b7a-bba2-6f471753a0fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6827","nonce":345649648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6829","nonce":345649648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6833","nonce":345649648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6831","nonce":345649648}]},"public_addr":"172.21.15.71:6827/345649648","cluster_addr":"172.21.15.71:6829/345649648","heartbeat_back_addr":"172.21.15.71:6833/345649648","heartbeat_front_addr":"172.21.15.71:6831/345649648","state":["exists","up"]},{"osd":2,"uuid":"d8b70f09-5afb-4296-b8c9-719c172e0e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6801","nonce":2721860691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6803","nonce":2721860691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6807","nonce":2721860691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6805","nonce":2721860691}]},"public_addr":"172.21.15.71:6801/2721860691","cluster_addr":"172.21.15.71:6803/2721860691","heartbeat_back_addr":"172.21.15.71:6807/2721860691","heartbeat_front_addr":"172.21.15.71:6805/2721860691","state":["exists","up"]},{"osd":3,"uuid":"26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6809","nonce":494607463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6811","nonce":494607463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6815","nonce":494607463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6813","nonce":494607463}]},"public_addr":"172.21.15.71:6809/494607463","cluster_addr":"172.21.15.71:6811/494607463","heartbeat_back_addr":"172.21.15.71:6815/494607463","heartbeat_front_addr":"172.21.15.71:6813/494607463","state":["exists","up"]},{"osd":4,"uuid":"515cb7d6-16eb-4aff-973a-7f7ed49325ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6817","nonce":2240133417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6819","nonce":2240133417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6823","nonce":2240133417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6821","nonce":2240133417}]},"public_addr":"172.21.15.150:6817/2240133417","cluster_addr":"172.21.15.150:6819/2240133417","heartbeat_back_addr":"172.21.15.150:6823/2240133417","heartbeat_front_addr":"172.21.15.150:6821/2240133417","state":["exists","up"]},{"osd":5,"uuid":"c434b53a-04be-45b9-b89a-193654133950","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6809","nonce":2393825223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6811","nonce":2393825223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6815","nonce":2393825223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6813","nonce":2393825223}]},"public_addr":"172.21.15.150:6809/2393825223","cluster_addr":"172.21.15.150:6811/2393825223","heartbeat_back_addr":"172.21.15.150:6815/2393825223","heartbeat_front_addr":"172.21.15.150:6813/2393825223","state":["exists","up"]},{"osd":6,"uuid":"3d2f8405-9286-4b40-8acd-036f67373ac6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6801","nonce":3235729544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6803","nonce":3235729544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6807","nonce":3235729544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6805","nonce":3235729544}]},"public_addr":"172.21.15.150:6801/3235729544","cluster_addr":"172.21.15.150:6803/3235729544","heartbeat_back_addr":"172.21.15.150:6807/3235729544","heartbeat_front_addr":"172.21.15.150:6805/3235729544","state":["exists","up"]},{"osd":7,"uuid":"6724f080-4dc3-465a-939f-b023985bf998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6825","nonce":2381001355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6827","nonce":2381001355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6831","nonce":2381001355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6829","nonce":2381001355}]},"public_addr":"172.21.15.150:6825/2381001355","cluster_addr":"172.21.15.150:6827/2381001355","heartbeat_back_addr":"172.21.15.150:6831/2381001355","heartbeat_front_addr":"172.21.15.150:6829/2381001355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.548803+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.075023+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.117036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.332113+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.330371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:02.753946+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:02.071186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.378781+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T05:29:17.107 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-14T05:29:17.108 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-14T05:29:17.108 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-14T05:29:17.109 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-14T05:29:17.109 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-14T05:29:17.109 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-14T05:29:17.110 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-14T05:29:17.110 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-14T05:29:17.352 INFO:teuthology.orchestra.run.smithi071.stdout:64424509445 2024-09-14T05:29:17.352 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T05:29:17.353 INFO:teuthology.orchestra.run.smithi071.stdout:68719476741 2024-09-14T05:29:17.353 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T05:29:17.354 INFO:teuthology.orchestra.run.smithi071.stdout:64424509447 2024-09-14T05:29:17.354 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T05:29:17.355 INFO:teuthology.orchestra.run.smithi071.stdout:64424509445 2024-09-14T05:29:17.355 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T05:29:17.356 INFO:teuthology.orchestra.run.smithi071.stdout:64424509444 2024-09-14T05:29:17.357 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T05:29:17.357 INFO:teuthology.orchestra.run.smithi071.stdout:64424509445 2024-09-14T05:29:17.358 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T05:29:17.358 INFO:teuthology.orchestra.run.smithi071.stdout:64424509444 2024-09-14T05:29:17.359 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T05:29:17.360 INFO:teuthology.orchestra.run.smithi071.stdout:64424509446 2024-09-14T05:29:17.360 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T05:29:17.908 INFO:teuthology.orchestra.run.smithi071.stdout:64424509443 2024-09-14T05:29:17.927 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509443 for osd.3 2024-09-14T05:29:17.960 INFO:teuthology.orchestra.run.smithi071.stdout:68719476740 2024-09-14T05:29:17.973 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476741 got 68719476740 for osd.0 2024-09-14T05:29:17.999 INFO:teuthology.orchestra.run.smithi071.stdout:64424509445 2024-09-14T05:29:18.005 INFO:teuthology.orchestra.run.smithi071.stdout:64424509443 2024-09-14T05:29:18.011 INFO:teuthology.orchestra.run.smithi071.stdout:64424509444 2024-09-14T05:29:18.018 INFO:teuthology.orchestra.run.smithi071.stdout:64424509442 2024-09-14T05:29:18.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509445 for osd.1 2024-09-14T05:29:18.019 INFO:teuthology.orchestra.run.smithi071.stdout:64424509443 2024-09-14T05:29:18.019 INFO:teuthology.orchestra.run.smithi071.stdout:64424509442 2024-09-14T05:29:18.026 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509443 for osd.4 2024-09-14T05:29:18.032 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.6 2024-09-14T05:29:18.036 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509442 for osd.5 2024-09-14T05:29:18.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509443 for osd.7 2024-09-14T05:29:18.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509442 for osd.2 2024-09-14T05:29:18.928 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T05:29:18.974 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T05:29:19.019 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T05:29:19.027 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T05:29:19.033 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T05:29:19.037 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T05:29:19.037 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T05:29:19.037 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T05:29:19.416 INFO:teuthology.orchestra.run.smithi071.stdout:64424509445 2024-09-14T05:29:19.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.3 2024-09-14T05:29:19.437 DEBUG:teuthology.parallel:result is None 2024-09-14T05:29:19.510 INFO:teuthology.orchestra.run.smithi071.stdout:68719476742 2024-09-14T05:29:19.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476741 got 68719476742 for osd.0 2024-09-14T05:29:19.530 DEBUG:teuthology.parallel:result is None 2024-09-14T05:29:19.538 INFO:teuthology.orchestra.run.smithi071.stdout:64424509444 2024-09-14T05:29:19.554 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.2 2024-09-14T05:29:19.555 DEBUG:teuthology.parallel:result is None 2024-09-14T05:29:19.590 INFO:teuthology.orchestra.run.smithi071.stdout:64424509445 2024-09-14T05:29:19.592 INFO:teuthology.orchestra.run.smithi071.stdout:64424509447 2024-09-14T05:29:19.604 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.4 2024-09-14T05:29:19.604 DEBUG:teuthology.parallel:result is None 2024-09-14T05:29:19.610 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.1 2024-09-14T05:29:19.610 DEBUG:teuthology.parallel:result is None 2024-09-14T05:29:19.633 INFO:teuthology.orchestra.run.smithi071.stdout:64424509444 2024-09-14T05:29:19.646 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.5 2024-09-14T05:29:19.646 INFO:teuthology.orchestra.run.smithi071.stdout:64424509445 2024-09-14T05:29:19.646 DEBUG:teuthology.parallel:result is None 2024-09-14T05:29:19.654 INFO:teuthology.orchestra.run.smithi071.stdout:64424509446 2024-09-14T05:29:19.659 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.7 2024-09-14T05:29:19.659 DEBUG:teuthology.parallel:result is None 2024-09-14T05:29:19.667 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.6 2024-09-14T05:29:19.667 DEBUG:teuthology.parallel:result is None 2024-09-14T05:29:19.667 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T05:29:19.668 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:29:19.985 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:29:19.986 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:29:19.998 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-14T05:29:18.987903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219168,"kb_used_data":2896,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755552,"statfs":{"total":773094113280,"available":772869685248,"internally_reserved":0,"allocated":2965504,"data_stored":1777174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.161970"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.849668+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:29:15.849668+0000","last_peered":"2024-09-14T05:29:15.849668+0000","last_clean":"2024-09-14T05:29:15.849668+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:29:15.849668+0000","last_undegraded":"2024-09-14T05:29:15.849668+0000","last_fullsized":"2024-09-14T05:29:15.849668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.852000+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:29:15.852000+0000","last_peered":"2024-09-14T05:29:15.852000+0000","last_clean":"2024-09-14T05:29:15.852000+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:29:15.852000+0000","last_undegraded":"2024-09-14T05:29:15.852000+0000","last_fullsized":"2024-09-14T05:29:15.852000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.850751+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:29:15.850751+0000","last_peered":"2024-09-14T05:29:15.850751+0000","last_clean":"2024-09-14T05:29:15.850751+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:29:15.850751+0000","last_undegraded":"2024-09-14T05:29:15.850751+0000","last_fullsized":"2024-09-14T05:29:15.850751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.994953+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:29:15.994953+0000","last_peered":"2024-09-14T05:29:15.994953+0000","last_clean":"2024-09-14T05:29:15.994953+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:29:15.994953+0000","last_undegraded":"2024-09-14T05:29:15.994953+0000","last_fullsized":"2024-09-14T05:29:15.994953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.850987+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:29:15.850987+0000","last_peered":"2024-09-14T05:29:15.850987+0000","last_clean":"2024-09-14T05:29:15.850987+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:29:15.850987+0000","last_undegraded":"2024-09-14T05:29:15.850987+0000","last_fullsized":"2024-09-14T05:29:15.850987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.994837+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:29:15.994837+0000","last_peered":"2024-09-14T05:29:15.994837+0000","last_clean":"2024-09-14T05:29:15.994837+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:29:15.994837+0000","last_undegraded":"2024-09-14T05:29:15.994837+0000","last_fullsized":"2024-09-14T05:29:15.994837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.850350+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:29:15.850350+0000","last_peered":"2024-09-14T05:29:15.850350+0000","last_clean":"2024-09-14T05:29:15.850350+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:29:15.850350+0000","last_undegraded":"2024-09-14T05:29:15.850350+0000","last_fullsized":"2024-09-14T05:29:15.850350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.850582+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:29:15.850582+0000","last_peered":"2024-09-14T05:29:15.850582+0000","last_clean":"2024-09-14T05:29:15.850582+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:29:15.850582+0000","last_undegraded":"2024-09-14T05:29:15.850582+0000","last_fullsized":"2024-09-14T05:29:15.850582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.846368+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:29:15.846368+0000","last_peered":"2024-09-14T05:29:15.846368+0000","last_clean":"2024-09-14T05:29:15.846368+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:29:15.846368+0000","last_undegraded":"2024-09-14T05:29:15.846368+0000","last_fullsized":"2024-09-14T05:29:15.846368+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476742,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":221184,"data_stored":74550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":225280,"data_stored":74569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":664918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":74569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":221184,"data_stored":74550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":664918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:29:19.999 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:29:20.281 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:29:20.282 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:29:20.293 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-14T05:29:18.987903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219168,"kb_used_data":2896,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755552,"statfs":{"total":773094113280,"available":772869685248,"internally_reserved":0,"allocated":2965504,"data_stored":1777174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.161970"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.849668+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:29:15.849668+0000","last_peered":"2024-09-14T05:29:15.849668+0000","last_clean":"2024-09-14T05:29:15.849668+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:29:15.849668+0000","last_undegraded":"2024-09-14T05:29:15.849668+0000","last_fullsized":"2024-09-14T05:29:15.849668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.852000+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:29:15.852000+0000","last_peered":"2024-09-14T05:29:15.852000+0000","last_clean":"2024-09-14T05:29:15.852000+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:29:15.852000+0000","last_undegraded":"2024-09-14T05:29:15.852000+0000","last_fullsized":"2024-09-14T05:29:15.852000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.850751+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:29:15.850751+0000","last_peered":"2024-09-14T05:29:15.850751+0000","last_clean":"2024-09-14T05:29:15.850751+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:29:15.850751+0000","last_undegraded":"2024-09-14T05:29:15.850751+0000","last_fullsized":"2024-09-14T05:29:15.850751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.994953+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:29:15.994953+0000","last_peered":"2024-09-14T05:29:15.994953+0000","last_clean":"2024-09-14T05:29:15.994953+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:29:15.994953+0000","last_undegraded":"2024-09-14T05:29:15.994953+0000","last_fullsized":"2024-09-14T05:29:15.994953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.850987+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:29:15.850987+0000","last_peered":"2024-09-14T05:29:15.850987+0000","last_clean":"2024-09-14T05:29:15.850987+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:29:15.850987+0000","last_undegraded":"2024-09-14T05:29:15.850987+0000","last_fullsized":"2024-09-14T05:29:15.850987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.994837+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:29:15.994837+0000","last_peered":"2024-09-14T05:29:15.994837+0000","last_clean":"2024-09-14T05:29:15.994837+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:29:15.994837+0000","last_undegraded":"2024-09-14T05:29:15.994837+0000","last_fullsized":"2024-09-14T05:29:15.994837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.850350+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:29:15.850350+0000","last_peered":"2024-09-14T05:29:15.850350+0000","last_clean":"2024-09-14T05:29:15.850350+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:29:15.850350+0000","last_undegraded":"2024-09-14T05:29:15.850350+0000","last_fullsized":"2024-09-14T05:29:15.850350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.850582+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:29:15.850582+0000","last_peered":"2024-09-14T05:29:15.850582+0000","last_clean":"2024-09-14T05:29:15.850582+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:29:15.850582+0000","last_undegraded":"2024-09-14T05:29:15.850582+0000","last_fullsized":"2024-09-14T05:29:15.850582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.846368+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:29:15.846368+0000","last_peered":"2024-09-14T05:29:15.846368+0000","last_clean":"2024-09-14T05:29:15.846368+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:29:15.846368+0000","last_undegraded":"2024-09-14T05:29:15.846368+0000","last_fullsized":"2024-09-14T05:29:15.846368+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476742,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":221184,"data_stored":74550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":225280,"data_stored":74569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":664918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":74569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":221184,"data_stored":74550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":664918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:29:20.294 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T05:29:20.294 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-14T05:29:20.294 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-14T05:29:20.294 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-14T05:29:20.613 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:29:20.613 INFO:teuthology.orchestra.run.smithi071.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-14T05:29:20.626 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-14T05:29:20.626 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-14T05:29:20.638 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-14T05:29:20.639 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '0d61375b8ab53be4c233f5b93e463edfdff9316a'}} 2024-09-14T05:29:20.639 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-14T05:29:20.639 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-14T05:29:20.942 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-14T05:29:20.942 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T05:29:20.942 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:29:21.226 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:29:21.227 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:29:21.238 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-14T05:29:20.438403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219168,"kb_used_data":2896,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755552,"statfs":{"total":773094113280,"available":772869685248,"internally_reserved":0,"allocated":2965504,"data_stored":1777174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000929"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.849668+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:29:15.849668+0000","last_peered":"2024-09-14T05:29:15.849668+0000","last_clean":"2024-09-14T05:29:15.849668+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:29:15.849668+0000","last_undegraded":"2024-09-14T05:29:15.849668+0000","last_fullsized":"2024-09-14T05:29:15.849668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.852000+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:29:15.852000+0000","last_peered":"2024-09-14T05:29:15.852000+0000","last_clean":"2024-09-14T05:29:15.852000+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:29:15.852000+0000","last_undegraded":"2024-09-14T05:29:15.852000+0000","last_fullsized":"2024-09-14T05:29:15.852000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.850751+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:29:15.850751+0000","last_peered":"2024-09-14T05:29:15.850751+0000","last_clean":"2024-09-14T05:29:15.850751+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:29:15.850751+0000","last_undegraded":"2024-09-14T05:29:15.850751+0000","last_fullsized":"2024-09-14T05:29:15.850751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.994953+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:29:15.994953+0000","last_peered":"2024-09-14T05:29:15.994953+0000","last_clean":"2024-09-14T05:29:15.994953+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:29:15.994953+0000","last_undegraded":"2024-09-14T05:29:15.994953+0000","last_fullsized":"2024-09-14T05:29:15.994953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.850987+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:29:15.850987+0000","last_peered":"2024-09-14T05:29:15.850987+0000","last_clean":"2024-09-14T05:29:15.850987+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:29:15.850987+0000","last_undegraded":"2024-09-14T05:29:15.850987+0000","last_fullsized":"2024-09-14T05:29:15.850987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.994837+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:29:15.994837+0000","last_peered":"2024-09-14T05:29:15.994837+0000","last_clean":"2024-09-14T05:29:15.994837+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:29:15.994837+0000","last_undegraded":"2024-09-14T05:29:15.994837+0000","last_fullsized":"2024-09-14T05:29:15.994837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.850350+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:29:15.850350+0000","last_peered":"2024-09-14T05:29:15.850350+0000","last_clean":"2024-09-14T05:29:15.850350+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:29:15.850350+0000","last_undegraded":"2024-09-14T05:29:15.850350+0000","last_fullsized":"2024-09-14T05:29:15.850350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.850582+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:29:15.850582+0000","last_peered":"2024-09-14T05:29:15.850582+0000","last_clean":"2024-09-14T05:29:15.850582+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:29:15.850582+0000","last_undegraded":"2024-09-14T05:29:15.850582+0000","last_fullsized":"2024-09-14T05:29:15.850582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.846368+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:29:15.846368+0000","last_peered":"2024-09-14T05:29:15.846368+0000","last_clean":"2024-09-14T05:29:15.846368+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:29:15.846368+0000","last_undegraded":"2024-09-14T05:29:15.846368+0000","last_fullsized":"2024-09-14T05:29:15.846368+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476742,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":221184,"data_stored":74550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":225280,"data_stored":74569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":664918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":74569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":221184,"data_stored":74550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":664918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:29:21.239 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:29:21.519 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:29:21.520 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:29:21.532 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-14T05:29:20.438403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219168,"kb_used_data":2896,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755552,"statfs":{"total":773094113280,"available":772869685248,"internally_reserved":0,"allocated":2965504,"data_stored":1777174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000929"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.849668+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:29:15.849668+0000","last_peered":"2024-09-14T05:29:15.849668+0000","last_clean":"2024-09-14T05:29:15.849668+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:29:15.849668+0000","last_undegraded":"2024-09-14T05:29:15.849668+0000","last_fullsized":"2024-09-14T05:29:15.849668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.852000+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:29:15.852000+0000","last_peered":"2024-09-14T05:29:15.852000+0000","last_clean":"2024-09-14T05:29:15.852000+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:29:15.852000+0000","last_undegraded":"2024-09-14T05:29:15.852000+0000","last_fullsized":"2024-09-14T05:29:15.852000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.850751+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:29:15.850751+0000","last_peered":"2024-09-14T05:29:15.850751+0000","last_clean":"2024-09-14T05:29:15.850751+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:29:15.850751+0000","last_undegraded":"2024-09-14T05:29:15.850751+0000","last_fullsized":"2024-09-14T05:29:15.850751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.994953+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:29:15.994953+0000","last_peered":"2024-09-14T05:29:15.994953+0000","last_clean":"2024-09-14T05:29:15.994953+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:29:15.994953+0000","last_undegraded":"2024-09-14T05:29:15.994953+0000","last_fullsized":"2024-09-14T05:29:15.994953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.850987+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:29:15.850987+0000","last_peered":"2024-09-14T05:29:15.850987+0000","last_clean":"2024-09-14T05:29:15.850987+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:29:15.850987+0000","last_undegraded":"2024-09-14T05:29:15.850987+0000","last_fullsized":"2024-09-14T05:29:15.850987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.994837+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:29:15.994837+0000","last_peered":"2024-09-14T05:29:15.994837+0000","last_clean":"2024-09-14T05:29:15.994837+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:29:15.994837+0000","last_undegraded":"2024-09-14T05:29:15.994837+0000","last_fullsized":"2024-09-14T05:29:15.994837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.850350+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:29:15.850350+0000","last_peered":"2024-09-14T05:29:15.850350+0000","last_clean":"2024-09-14T05:29:15.850350+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:29:15.850350+0000","last_undegraded":"2024-09-14T05:29:15.850350+0000","last_fullsized":"2024-09-14T05:29:15.850350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.850582+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:29:15.850582+0000","last_peered":"2024-09-14T05:29:15.850582+0000","last_clean":"2024-09-14T05:29:15.850582+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:29:15.850582+0000","last_undegraded":"2024-09-14T05:29:15.850582+0000","last_fullsized":"2024-09-14T05:29:15.850582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T05:29:15.846368+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:29:15.846368+0000","last_peered":"2024-09-14T05:29:15.846368+0000","last_clean":"2024-09-14T05:29:15.846368+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:29:15.846368+0000","last_undegraded":"2024-09-14T05:29:15.846368+0000","last_fullsized":"2024-09-14T05:29:15.846368+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476742,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":221184,"data_stored":74550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":225280,"data_stored":74569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":664918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":74569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":221184,"data_stored":74550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":664918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:29:21.533 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T05:29:21.533 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-14T05:29:21.816 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 23 2024-09-14T05:29:21.816 INFO:teuthology.orchestra.run.smithi071.stdout:fsid 1127be44-5f14-4e7d-ad23-f608afe154f5 2024-09-14T05:29:21.817 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-09-14T05:28:54.203072+0000 2024-09-14T05:29:21.817 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-09-14T05:29:18.986033+0000 2024-09-14T05:29:21.817 INFO:teuthology.orchestra.run.smithi071.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-14T05:29:21.817 INFO:teuthology.orchestra.run.smithi071.stdout:crush_version 6 2024-09-14T05:29:21.817 INFO:teuthology.orchestra.run.smithi071.stdout:full_ratio 0.95 2024-09-14T05:29:21.817 INFO:teuthology.orchestra.run.smithi071.stdout:backfillfull_ratio 0.9 2024-09-14T05:29:21.817 INFO:teuthology.orchestra.run.smithi071.stdout:nearfull_ratio 0.85 2024-09-14T05:29:21.817 INFO:teuthology.orchestra.run.smithi071.stdout:require_min_compat_client luminous 2024-09-14T05:29:21.817 INFO:teuthology.orchestra.run.smithi071.stdout:min_compat_client jewel 2024-09-14T05:29:21.817 INFO:teuthology.orchestra.run.smithi071.stdout:require_osd_release squid 2024-09-14T05:29:21.817 INFO:teuthology.orchestra.run.smithi071.stdout:stretch_mode_enabled false 2024-09-14T05:29:21.817 INFO:teuthology.orchestra.run.smithi071.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-14T05:29:21.817 INFO:teuthology.orchestra.run.smithi071.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 22 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 4.98 2024-09-14T05:29:21.818 INFO:teuthology.orchestra.run.smithi071.stdout:max_osd 8 2024-09-14T05:29:21.818 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 up in weight 1 up_from 16 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6818/2926109885,v1:172.21.15.71:6819/2926109885] [v2:172.21.15.71:6820/2926109885,v1:172.21.15.71:6821/2926109885] exists,up e7c7817f-4a65-4f66-9af5-d31d0a92cf30 2024-09-14T05:29:21.818 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6826/345649648,v1:172.21.15.71:6827/345649648] [v2:172.21.15.71:6828/345649648,v1:172.21.15.71:6829/345649648] exists,up e184604e-b973-4b7a-bba2-6f471753a0fd 2024-09-14T05:29:21.818 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6800/2721860691,v1:172.21.15.71:6801/2721860691] [v2:172.21.15.71:6802/2721860691,v1:172.21.15.71:6803/2721860691] exists,up d8b70f09-5afb-4296-b8c9-719c172e0e70 2024-09-14T05:29:21.818 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6808/494607463,v1:172.21.15.71:6809/494607463] [v2:172.21.15.71:6810/494607463,v1:172.21.15.71:6811/494607463] exists,up 26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c 2024-09-14T05:29:21.818 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6816/2240133417,v1:172.21.15.150:6817/2240133417] [v2:172.21.15.150:6818/2240133417,v1:172.21.15.150:6819/2240133417] exists,up 515cb7d6-16eb-4aff-973a-7f7ed49325ba 2024-09-14T05:29:21.818 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6808/2393825223,v1:172.21.15.150:6809/2393825223] [v2:172.21.15.150:6810/2393825223,v1:172.21.15.150:6811/2393825223] exists,up c434b53a-04be-45b9-b89a-193654133950 2024-09-14T05:29:21.818 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6800/3235729544,v1:172.21.15.150:6801/3235729544] [v2:172.21.15.150:6802/3235729544,v1:172.21.15.150:6803/3235729544] exists,up 3d2f8405-9286-4b40-8acd-036f67373ac6 2024-09-14T05:29:21.818 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6824/2381001355,v1:172.21.15.150:6825/2381001355] [v2:172.21.15.150:6826/2381001355,v1:172.21.15.150:6827/2381001355] exists,up 6724f080-4dc3-465a-939f-b023985bf998 2024-09-14T05:29:21.829 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 16 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6818/2926109885,v1:172.21.15.71:6819/2926109885] [v2:172.21.15.71:6820/2926109885,v1:172.21.15.71:6821/2926109885] exists,up e7c7817f-4a65-4f66-9af5-d31d0a92cf30', 'osd.1 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6826/345649648,v1:172.21.15.71:6827/345649648] [v2:172.21.15.71:6828/345649648,v1:172.21.15.71:6829/345649648] exists,up e184604e-b973-4b7a-bba2-6f471753a0fd', 'osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6800/2721860691,v1:172.21.15.71:6801/2721860691] [v2:172.21.15.71:6802/2721860691,v1:172.21.15.71:6803/2721860691] exists,up d8b70f09-5afb-4296-b8c9-719c172e0e70', 'osd.3 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6808/494607463,v1:172.21.15.71:6809/494607463] [v2:172.21.15.71:6810/494607463,v1:172.21.15.71:6811/494607463] exists,up 26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c', 'osd.4 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6816/2240133417,v1:172.21.15.150:6817/2240133417] [v2:172.21.15.150:6818/2240133417,v1:172.21.15.150:6819/2240133417] exists,up 515cb7d6-16eb-4aff-973a-7f7ed49325ba', 'osd.5 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6808/2393825223,v1:172.21.15.150:6809/2393825223] [v2:172.21.15.150:6810/2393825223,v1:172.21.15.150:6811/2393825223] exists,up c434b53a-04be-45b9-b89a-193654133950', 'osd.6 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6800/3235729544,v1:172.21.15.150:6801/3235729544] [v2:172.21.15.150:6802/3235729544,v1:172.21.15.150:6803/3235729544] exists,up 3d2f8405-9286-4b40-8acd-036f67373ac6', 'osd.7 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6824/2381001355,v1:172.21.15.150:6825/2381001355] [v2:172.21.15.150:6826/2381001355,v1:172.21.15.150:6827/2381001355] exists,up 6724f080-4dc3-465a-939f-b023985bf998'] 2024-09-14T05:29:21.829 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-14T05:29:21.927 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:21.928 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mon.a", 2024-09-14T05:29:21.928 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster": "ceph", 2024-09-14T05:29:21.928 INFO:teuthology.orchestra.run.smithi071.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-14T05:29:21.928 INFO:teuthology.orchestra.run.smithi071.stdout: "admin_socket_mode": "", 2024-09-14T05:29:21.928 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-14T05:29:21.928 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_client_required": "cephx, none", 2024-09-14T05:29:21.928 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_cluster_required": "cephx", 2024-09-14T05:29:21.928 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_debug": "true", 2024-09-14T05:29:21.928 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-14T05:29:21.928 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-14T05:29:21.929 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_service_required": "cephx", 2024-09-14T05:29:21.929 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-14T05:29:21.929 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_supported": "cephx", 2024-09-14T05:29:21.929 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_aio": "true", 2024-09-14T05:29:21.929 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-14T05:29:21.929 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_aio_poll_ms": "250", 2024-09-14T05:29:21.929 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_aio_reap_max": "16", 2024-09-14T05:29:21.929 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_async_discard_threads": "0", 2024-09-14T05:29:21.929 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_block_size": "4096", 2024-09-14T05:29:21.929 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_debug_aio": "false", 2024-09-14T05:29:21.930 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-14T05:29:21.930 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-14T05:29:21.930 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_debug_inflight_ios": "false", 2024-09-14T05:29:21.930 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_enable_discard": "false", 2024-09-14T05:29:21.930 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_flock_retry": "3", 2024-09-14T05:29:21.930 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-14T05:29:21.930 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_inject_crash": "0", 2024-09-14T05:29:21.930 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-14T05:29:21.930 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_ioring": "false", 2024-09-14T05:29:21.930 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_ioring_hipri": "false", 2024-09-14T05:29:21.930 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-14T05:29:21.930 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-14T05:29:21.930 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-14T05:29:21.930 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-14T05:29:21.931 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-14T05:29:21.931 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-14T05:29:21.931 INFO:teuthology.orchestra.run.smithi071.stdout: "bdev_type": "", 2024-09-14T05:29:21.931 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_alloc_size": "1048576", 2024-09-14T05:29:21.931 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_allocator": "hybrid", 2024-09-14T05:29:21.931 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_buffered_io": "true", 2024-09-14T05:29:21.931 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_check_for_zeros": "false", 2024-09-14T05:29:21.931 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-14T05:29:21.931 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-14T05:29:21.931 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_compact_log_sync": "false", 2024-09-14T05:29:21.931 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-14T05:29:21.931 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-14T05:29:21.931 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-14T05:29:21.932 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-14T05:29:21.932 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_max_log_runway": "4194304", 2024-09-14T05:29:21.932 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_max_prefetch": "1048576", 2024-09-14T05:29:21.932 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_min_flush_size": "524288", 2024-09-14T05:29:21.932 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_min_log_runway": "1048576", 2024-09-14T05:29:21.932 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_replay_recovery": "false", 2024-09-14T05:29:21.932 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-14T05:29:21.932 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-14T05:29:21.932 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_sync_write": "false", 2024-09-14T05:29:21.932 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-14T05:29:21.932 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-14T05:29:21.932 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-14T05:29:21.933 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_allocation_from_file": "true", 2024-09-14T05:29:21.933 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_allocator": "hybrid", 2024-09-14T05:29:21.933 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_async_db_compaction": "true", 2024-09-14T05:29:21.933 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-14T05:29:21.933 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-14T05:29:21.933 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-14T05:29:21.933 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-14T05:29:21.933 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_bdev_label_multi": "true", 2024-09-14T05:29:21.933 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-14T05:29:21.933 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-14T05:29:21.933 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-14T05:29:21.933 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-14T05:29:21.934 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-14T05:29:21.934 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_create": "true", 2024-09-14T05:29:21.934 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_db_create": "false", 2024-09-14T05:29:21.934 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_db_path": "", 2024-09-14T05:29:21.934 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_db_size": "0", 2024-09-14T05:29:21.934 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_path": "", 2024-09-14T05:29:21.934 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_preallocate_file": "false", 2024-09-14T05:29:21.934 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_size": "107374182400", 2024-09-14T05:29:21.934 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_wal_create": "false", 2024-09-14T05:29:21.934 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_wal_path": "", 2024-09-14T05:29:21.934 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_block_wal_size": "100663296", 2024-09-14T05:29:21.934 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_bluefs": "true", 2024-09-14T05:29:21.934 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-14T05:29:21.934 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-14T05:29:21.935 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-14T05:29:21.935 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-14T05:29:21.935 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-14T05:29:21.935 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T05:29:21.935 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T05:29:21.935 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-14T05:29:21.935 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T05:29:21.935 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_autotune": "true", 2024-09-14T05:29:21.935 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-14T05:29:21.935 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-14T05:29:21.935 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-14T05:29:21.935 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-14T05:29:21.936 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_size": "0", 2024-09-14T05:29:21.936 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-14T05:29:21.936 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-14T05:29:21.936 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-14T05:29:21.936 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-14T05:29:21.936 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_type": "2q", 2024-09-14T05:29:21.936 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-14T05:29:21.936 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_clone_cow": "true", 2024-09-14T05:29:21.936 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-14T05:29:21.936 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-14T05:29:21.936 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-14T05:29:21.936 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-14T05:29:21.937 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-14T05:29:21.937 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-14T05:29:21.937 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-14T05:29:21.937 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_compression_mode": "none", 2024-09-14T05:29:21.937 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-14T05:29:21.937 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_csum_type": "crc32c", 2024-09-14T05:29:21.937 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-14T05:29:21.937 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-14T05:29:21.937 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_freelist": "false", 2024-09-14T05:29:21.937 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-14T05:29:21.937 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-14T05:29:21.937 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-14T05:29:21.937 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-14T05:29:21.938 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-14T05:29:21.938 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-14T05:29:21.938 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-14T05:29:21.938 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-14T05:29:21.938 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-14T05:29:21.938 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-14T05:29:21.938 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-14T05:29:21.938 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-14T05:29:21.938 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_small_allocations": "0", 2024-09-14T05:29:21.938 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-14T05:29:21.938 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_default_buffered_read": "true", 2024-09-14T05:29:21.938 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_default_buffered_write": "false", 2024-09-14T05:29:21.939 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-14T05:29:21.939 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-14T05:29:21.939 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-14T05:29:21.939 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-14T05:29:21.939 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-14T05:29:21.939 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-14T05:29:21.939 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-14T05:29:21.939 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-14T05:29:21.939 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-14T05:29:21.939 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fail_eio": "false", 2024-09-14T05:29:21.939 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-14T05:29:21.939 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-14T05:29:21.939 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-14T05:29:21.939 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-14T05:29:21.940 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-14T05:29:21.940 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-14T05:29:21.940 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_on_mount": "false", 2024-09-14T05:29:21.940 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-14T05:29:21.940 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_on_umount": "false", 2024-09-14T05:29:21.940 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-14T05:29:21.940 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-14T05:29:21.940 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-14T05:29:21.940 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-14T05:29:21.940 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-14T05:29:21.940 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-14T05:29:21.940 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-14T05:29:21.941 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-14T05:29:21.941 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_ignore_data_csum": "false", 2024-09-14T05:29:21.941 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-14T05:29:21.941 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-14T05:29:21.941 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-14T05:29:21.941 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-14T05:29:21.941 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_log_op_age": "5.000000", 2024-09-14T05:29:21.941 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_max_alloc_size": "0", 2024-09-14T05:29:21.941 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_max_blob_size": "0", 2024-09-14T05:29:21.941 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-14T05:29:21.941 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-14T05:29:21.941 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-14T05:29:21.941 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_max_deferred_txc": "32", 2024-09-14T05:29:21.941 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_min_alloc_size": "0", 2024-09-14T05:29:21.941 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-14T05:29:21.942 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-14T05:29:21.942 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_nid_prealloc": "1024", 2024-09-14T05:29:21.942 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-14T05:29:21.942 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-14T05:29:21.942 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-14T05:29:21.942 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-14T05:29:21.942 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_retry_disk_reads": "3", 2024-09-14T05:29:21.942 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_rocksdb_cf": "true", 2024-09-14T05:29:21.942 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-14T05:29:21.942 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-14T05:29:21.942 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-14T05:29:21.942 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-14T05:29:21.942 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-14T05:29:21.942 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-14T05:29:21.943 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-14T05:29:21.943 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-14T05:29:21.943 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_spdk_mem": "512", 2024-09-14T05:29:21.943 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-14T05:29:21.943 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-14T05:29:21.943 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-14T05:29:21.943 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-14T05:29:21.943 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-14T05:29:21.943 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-14T05:29:21.943 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-14T05:29:21.943 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_tracing": "false", 2024-09-14T05:29:21.943 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-14T05:29:21.943 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-14T05:29:21.943 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-14T05:29:21.944 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-14T05:29:21.944 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-14T05:29:21.944 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-14T05:29:21.944 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-14T05:29:21.944 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-14T05:29:21.944 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-14T05:29:21.944 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_write_v2": "false", 2024-09-14T05:29:21.944 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_write_v2_random": "false", 2024-09-14T05:29:21.944 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_zero_block_detection": "false", 2024-09-14T05:29:21.944 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-14T05:29:21.944 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-14T05:29:21.944 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-14T05:29:21.944 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-14T05:29:21.944 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-14T05:29:21.945 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-14T05:29:21.945 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-14T05:29:21.945 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-14T05:29:21.945 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-14T05:29:21.945 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-14T05:29:21.945 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-14T05:29:21.945 INFO:teuthology.orchestra.run.smithi071.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-14T05:29:21.945 INFO:teuthology.orchestra.run.smithi071.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-14T05:29:21.945 INFO:teuthology.orchestra.run.smithi071.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-14T05:29:21.945 INFO:teuthology.orchestra.run.smithi071.stdout: "cephx_cluster_require_signatures": "false", 2024-09-14T05:29:21.945 INFO:teuthology.orchestra.run.smithi071.stdout: "cephx_cluster_require_version": "2", 2024-09-14T05:29:21.945 INFO:teuthology.orchestra.run.smithi071.stdout: "cephx_require_signatures": "false", 2024-09-14T05:29:21.945 INFO:teuthology.orchestra.run.smithi071.stdout: "cephx_require_version": "2", 2024-09-14T05:29:21.945 INFO:teuthology.orchestra.run.smithi071.stdout: "cephx_service_require_signatures": "false", 2024-09-14T05:29:21.945 INFO:teuthology.orchestra.run.smithi071.stdout: "cephx_service_require_version": "2", 2024-09-14T05:29:21.946 INFO:teuthology.orchestra.run.smithi071.stdout: "cephx_sign_messages": "true", 2024-09-14T05:29:21.946 INFO:teuthology.orchestra.run.smithi071.stdout: "chdir": "", 2024-09-14T05:29:21.946 INFO:teuthology.orchestra.run.smithi071.stdout: "client_acl_type": "", 2024-09-14T05:29:21.946 INFO:teuthology.orchestra.run.smithi071.stdout: "client_asio_thread_count": "2", 2024-09-14T05:29:21.946 INFO:teuthology.orchestra.run.smithi071.stdout: "client_cache_mid": "0.750000", 2024-09-14T05:29:21.946 INFO:teuthology.orchestra.run.smithi071.stdout: "client_cache_size": "16384", 2024-09-14T05:29:21.946 INFO:teuthology.orchestra.run.smithi071.stdout: "client_caps_release_delay": "5", 2024-09-14T05:29:21.946 INFO:teuthology.orchestra.run.smithi071.stdout: "client_check_pool_perm": "true", 2024-09-14T05:29:21.946 INFO:teuthology.orchestra.run.smithi071.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-14T05:29:21.946 INFO:teuthology.orchestra.run.smithi071.stdout: "client_debug_force_sync_read": "false", 2024-09-14T05:29:21.946 INFO:teuthology.orchestra.run.smithi071.stdout: "client_debug_getattr_caps": "false", 2024-09-14T05:29:21.946 INFO:teuthology.orchestra.run.smithi071.stdout: "client_debug_inject_features": "", 2024-09-14T05:29:21.946 INFO:teuthology.orchestra.run.smithi071.stdout: "client_debug_inject_tick_delay": "0", 2024-09-14T05:29:21.946 INFO:teuthology.orchestra.run.smithi071.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-14T05:29:21.946 INFO:teuthology.orchestra.run.smithi071.stdout: "client_die_on_failed_remount": "false", 2024-09-14T05:29:21.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client_dirsize_rbytes": "true", 2024-09-14T05:29:21.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client_force_lazyio": "false", 2024-09-14T05:29:21.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client_fs": "", 2024-09-14T05:29:21.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-14T05:29:21.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client_inject_release_failure": "false", 2024-09-14T05:29:21.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client_max_inline_size": "4096", 2024-09-14T05:29:21.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-14T05:29:21.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client_mds_namespace": "", 2024-09-14T05:29:21.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client_metadata": "", 2024-09-14T05:29:21.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client_mount_gid": "-1", 2024-09-14T05:29:21.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client_mount_timeout": "300", 2024-09-14T05:29:21.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client_mount_uid": "-1", 2024-09-14T05:29:21.947 INFO:teuthology.orchestra.run.smithi071.stdout: "client_mountpoint": "/", 2024-09-14T05:29:21.948 INFO:teuthology.orchestra.run.smithi071.stdout: "client_notify_timeout": "10", 2024-09-14T05:29:21.948 INFO:teuthology.orchestra.run.smithi071.stdout: "client_oc": "true", 2024-09-14T05:29:21.948 INFO:teuthology.orchestra.run.smithi071.stdout: "client_oc_max_dirty": "104857600", 2024-09-14T05:29:21.948 INFO:teuthology.orchestra.run.smithi071.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-14T05:29:21.948 INFO:teuthology.orchestra.run.smithi071.stdout: "client_oc_max_objects": "1000", 2024-09-14T05:29:21.948 INFO:teuthology.orchestra.run.smithi071.stdout: "client_oc_size": "209715200", 2024-09-14T05:29:21.948 INFO:teuthology.orchestra.run.smithi071.stdout: "client_oc_target_dirty": "8388608", 2024-09-14T05:29:21.948 INFO:teuthology.orchestra.run.smithi071.stdout: "client_permissions": "true", 2024-09-14T05:29:21.948 INFO:teuthology.orchestra.run.smithi071.stdout: "client_quota": "true", 2024-09-14T05:29:21.948 INFO:teuthology.orchestra.run.smithi071.stdout: "client_quota_df": "true", 2024-09-14T05:29:21.948 INFO:teuthology.orchestra.run.smithi071.stdout: "client_readahead_max_bytes": "0", 2024-09-14T05:29:21.948 INFO:teuthology.orchestra.run.smithi071.stdout: "client_readahead_max_periods": "4", 2024-09-14T05:29:21.948 INFO:teuthology.orchestra.run.smithi071.stdout: "client_readahead_min": "131072", 2024-09-14T05:29:21.949 INFO:teuthology.orchestra.run.smithi071.stdout: "client_reconnect_stale": "false", 2024-09-14T05:29:21.949 INFO:teuthology.orchestra.run.smithi071.stdout: "client_shutdown_timeout": "30", 2024-09-14T05:29:21.949 INFO:teuthology.orchestra.run.smithi071.stdout: "client_snapdir": ".snap", 2024-09-14T05:29:21.949 INFO:teuthology.orchestra.run.smithi071.stdout: "client_tick_interval": "1", 2024-09-14T05:29:21.949 INFO:teuthology.orchestra.run.smithi071.stdout: "client_trace": "", 2024-09-14T05:29:21.949 INFO:teuthology.orchestra.run.smithi071.stdout: "client_try_dentry_invalidate": "false", 2024-09-14T05:29:21.949 INFO:teuthology.orchestra.run.smithi071.stdout: "client_use_faked_inos": "false", 2024-09-14T05:29:21.949 INFO:teuthology.orchestra.run.smithi071.stdout: "client_use_random_mds": "false", 2024-09-14T05:29:21.949 INFO:teuthology.orchestra.run.smithi071.stdout: "clog_to_graylog": "false", 2024-09-14T05:29:21.949 INFO:teuthology.orchestra.run.smithi071.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-14T05:29:21.949 INFO:teuthology.orchestra.run.smithi071.stdout: "clog_to_graylog_port": "12201", 2024-09-14T05:29:21.949 INFO:teuthology.orchestra.run.smithi071.stdout: "clog_to_monitors": "default=true", 2024-09-14T05:29:21.949 INFO:teuthology.orchestra.run.smithi071.stdout: "clog_to_syslog": "false", 2024-09-14T05:29:21.950 INFO:teuthology.orchestra.run.smithi071.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-14T05:29:21.950 INFO:teuthology.orchestra.run.smithi071.stdout: "clog_to_syslog_level": "info", 2024-09-14T05:29:21.950 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "-", 2024-09-14T05:29:21.950 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_network": "", 2024-09-14T05:29:21.950 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_network_interface": "", 2024-09-14T05:29:21.950 INFO:teuthology.orchestra.run.smithi071.stdout: "compressor_zlib_isal": "false", 2024-09-14T05:29:21.950 INFO:teuthology.orchestra.run.smithi071.stdout: "compressor_zlib_level": "5", 2024-09-14T05:29:21.950 INFO:teuthology.orchestra.run.smithi071.stdout: "compressor_zlib_winsize": "-15", 2024-09-14T05:29:21.950 INFO:teuthology.orchestra.run.smithi071.stdout: "compressor_zstd_level": "1", 2024-09-14T05:29:21.950 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-14T05:29:21.950 INFO:teuthology.orchestra.run.smithi071.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-14T05:29:21.950 INFO:teuthology.orchestra.run.smithi071.stdout: "crimson_alien_op_num_threads": "6", 2024-09-14T05:29:21.950 INFO:teuthology.orchestra.run.smithi071.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-14T05:29:21.950 INFO:teuthology.orchestra.run.smithi071.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-14T05:29:21.951 INFO:teuthology.orchestra.run.smithi071.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-14T05:29:21.951 INFO:teuthology.orchestra.run.smithi071.stdout: "crimson_osd_stat_interval": "0", 2024-09-14T05:29:21.951 INFO:teuthology.orchestra.run.smithi071.stdout: "crimson_seastar_cpu_cores": "", 2024-09-14T05:29:21.951 INFO:teuthology.orchestra.run.smithi071.stdout: "crimson_seastar_num_threads": "0", 2024-09-14T05:29:21.951 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_location": "", 2024-09-14T05:29:21.951 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_location_hook": "", 2024-09-14T05:29:21.951 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_location_hook_timeout": "10", 2024-09-14T05:29:21.951 INFO:teuthology.orchestra.run.smithi071.stdout: "daemonize": "false", 2024-09-14T05:29:21.951 INFO:teuthology.orchestra.run.smithi071.stdout: "daos_pool": "tank", 2024-09-14T05:29:21.951 INFO:teuthology.orchestra.run.smithi071.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-14T05:29:21.951 INFO:teuthology.orchestra.run.smithi071.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-14T05:29:21.951 INFO:teuthology.orchestra.run.smithi071.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-14T05:29:21.951 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_alienstore": "0/5", 2024-09-14T05:29:21.951 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_allow_any_pool_priority": "false", 2024-09-14T05:29:21.951 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_asok": "1/5", 2024-09-14T05:29:21.952 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_asok_assert_abort": "false", 2024-09-14T05:29:21.952 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_asserts_on_shutdown": "true", 2024-09-14T05:29:21.952 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_auth": "20/20", 2024-09-14T05:29:21.952 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_bdev": "1/3", 2024-09-14T05:29:21.952 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_bluefs": "1/5", 2024-09-14T05:29:21.952 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_bluestore": "1/5", 2024-09-14T05:29:21.952 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_buffer": "0/1", 2024-09-14T05:29:21.952 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_ceph_dedup": "0/5", 2024-09-14T05:29:21.952 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_ceph_exporter": "1/5", 2024-09-14T05:29:21.952 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_cephfs_mirror": "0/5", 2024-09-14T05:29:21.952 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_cephsqlite": "0/5", 2024-09-14T05:29:21.952 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_client": "0/5", 2024-09-14T05:29:21.953 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_compressor": "1/5", 2024-09-14T05:29:21.953 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_context": "0/1", 2024-09-14T05:29:21.953 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_crimson_interrupt": "0/5", 2024-09-14T05:29:21.953 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_crush": "1/1", 2024-09-14T05:29:21.953 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_crypto": "1/5", 2024-09-14T05:29:21.953 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_cyanstore": "0/5", 2024-09-14T05:29:21.953 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_deliberately_leak_memory": "false", 2024-09-14T05:29:21.953 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_disable_randomized_ping": "false", 2024-09-14T05:29:21.953 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_dpdk": "1/5", 2024-09-14T05:29:21.953 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_eventtrace": "1/5", 2024-09-14T05:29:21.953 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_filer": "0/1", 2024-09-14T05:29:21.953 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_filestore": "1/3", 2024-09-14T05:29:21.953 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_finisher": "1/1", 2024-09-14T05:29:21.954 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_fuse": "1/5", 2024-09-14T05:29:21.954 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_heartbeat_testing_span": "0", 2024-09-14T05:29:21.954 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_heartbeatmap": "1/5", 2024-09-14T05:29:21.954 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-14T05:29:21.954 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_javaclient": "1/5", 2024-09-14T05:29:21.954 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_journal": "1/3", 2024-09-14T05:29:21.954 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_journaler": "0/5", 2024-09-14T05:29:21.954 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_kstore": "1/5", 2024-09-14T05:29:21.954 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_lockdep": "0/1", 2024-09-14T05:29:21.954 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mclock": "1/5", 2024-09-14T05:29:21.954 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mds": "1/5", 2024-09-14T05:29:21.954 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mds_balancer": "1/5", 2024-09-14T05:29:21.954 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mds_locker": "1/5", 2024-09-14T05:29:21.955 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mds_log": "1/5", 2024-09-14T05:29:21.955 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mds_log_expire": "1/5", 2024-09-14T05:29:21.955 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mds_migrator": "1/5", 2024-09-14T05:29:21.955 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mds_quiesce": "3/5", 2024-09-14T05:29:21.955 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_memstore": "1/5", 2024-09-14T05:29:21.955 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mgr": "2/5", 2024-09-14T05:29:21.955 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mgrc": "1/5", 2024-09-14T05:29:21.955 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_mon": "20/20", 2024-09-14T05:29:21.955 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_monc": "0/10", 2024-09-14T05:29:21.955 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_ms": "1/1", 2024-09-14T05:29:21.955 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_none": "0/5", 2024-09-14T05:29:21.955 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_objclass": "0/5", 2024-09-14T05:29:21.955 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_objectcacher": "0/5", 2024-09-14T05:29:21.956 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_objecter": "0/1", 2024-09-14T05:29:21.956 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_optracker": "0/5", 2024-09-14T05:29:21.956 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_osd": "1/5", 2024-09-14T05:29:21.956 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_paxos": "20/20", 2024-09-14T05:29:21.956 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_perfcounter": "1/5", 2024-09-14T05:29:21.956 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_prioritycache": "1/5", 2024-09-14T05:29:21.956 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rados": "0/5", 2024-09-14T05:29:21.956 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rbd": "0/5", 2024-09-14T05:29:21.956 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rbd_mirror": "0/5", 2024-09-14T05:29:21.956 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rbd_pwl": "0/5", 2024-09-14T05:29:21.956 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rbd_replay": "0/5", 2024-09-14T05:29:21.956 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_refs": "0/0", 2024-09-14T05:29:21.956 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_reserver": "1/1", 2024-09-14T05:29:21.957 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rgw": "1/5", 2024-09-14T05:29:21.957 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rgw_access": "1/5", 2024-09-14T05:29:21.957 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rgw_datacache": "1/5", 2024-09-14T05:29:21.957 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rgw_dbstore": "1/5", 2024-09-14T05:29:21.957 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rgw_flight": "1/5", 2024-09-14T05:29:21.957 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-14T05:29:21.957 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rgw_notification": "1/5", 2024-09-14T05:29:21.957 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rgw_sync": "1/5", 2024-09-14T05:29:21.957 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_rocksdb": "4/5", 2024-09-14T05:29:21.957 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore": "0/5", 2024-09-14T05:29:21.957 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_backref": "0/5", 2024-09-14T05:29:21.957 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_cache": "0/5", 2024-09-14T05:29:21.958 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_cleaner": "0/5", 2024-09-14T05:29:21.958 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_device": "0/5", 2024-09-14T05:29:21.958 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_epm": "0/5", 2024-09-14T05:29:21.958 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-14T05:29:21.958 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_journal": "0/5", 2024-09-14T05:29:21.958 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_lba": "0/5", 2024-09-14T05:29:21.958 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_odata": "0/5", 2024-09-14T05:29:21.958 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_omap": "0/5", 2024-09-14T05:29:21.958 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_onode": "0/5", 2024-09-14T05:29:21.958 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_t": "0/5", 2024-09-14T05:29:21.958 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_seastore_tm": "0/5", 2024-09-14T05:29:21.958 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_striper": "0/1", 2024-09-14T05:29:21.958 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_test": "0/5", 2024-09-14T05:29:21.959 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_throttle": "1/1", 2024-09-14T05:29:21.959 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_timer": "0/1", 2024-09-14T05:29:21.959 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_tp": "0/5", 2024-09-14T05:29:21.959 INFO:teuthology.orchestra.run.smithi071.stdout: "debug_trace": "1/5", 2024-09-14T05:29:21.959 INFO:teuthology.orchestra.run.smithi071.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-14T05:29:21.959 INFO:teuthology.orchestra.run.smithi071.stdout: "device_failure_prediction_mode": "none", 2024-09-14T05:29:21.959 INFO:teuthology.orchestra.run.smithi071.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-14T05:29:21.959 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-14T05:29:21.959 INFO:teuthology.orchestra.run.smithi071.stdout: "err_to_graylog": "false", 2024-09-14T05:29:21.959 INFO:teuthology.orchestra.run.smithi071.stdout: "err_to_journald": "false", 2024-09-14T05:29:21.959 INFO:teuthology.orchestra.run.smithi071.stdout: "err_to_stderr": "true", 2024-09-14T05:29:21.959 INFO:teuthology.orchestra.run.smithi071.stdout: "err_to_syslog": "false", 2024-09-14T05:29:21.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event_tracing": "false", 2024-09-14T05:29:21.960 INFO:teuthology.orchestra.run.smithi071.stdout: "exporter_addr": "0.0.0.0", 2024-09-14T05:29:21.960 INFO:teuthology.orchestra.run.smithi071.stdout: "exporter_cert_file": "", 2024-09-14T05:29:21.960 INFO:teuthology.orchestra.run.smithi071.stdout: "exporter_http_port": "9926", 2024-09-14T05:29:21.960 INFO:teuthology.orchestra.run.smithi071.stdout: "exporter_key_file": "", 2024-09-14T05:29:21.960 INFO:teuthology.orchestra.run.smithi071.stdout: "exporter_prio_limit": "5", 2024-09-14T05:29:21.960 INFO:teuthology.orchestra.run.smithi071.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-14T05:29:21.960 INFO:teuthology.orchestra.run.smithi071.stdout: "exporter_sort_metrics": "true", 2024-09-14T05:29:21.960 INFO:teuthology.orchestra.run.smithi071.stdout: "exporter_stats_period": "5", 2024-09-14T05:29:21.960 INFO:teuthology.orchestra.run.smithi071.stdout: "fake_statfs_for_testing": "0", 2024-09-14T05:29:21.960 INFO:teuthology.orchestra.run.smithi071.stdout: "fatal_signal_handlers": "true", 2024-09-14T05:29:21.960 INFO:teuthology.orchestra.run.smithi071.stdout: "filer_max_purge_ops": "10", 2024-09-14T05:29:21.960 INFO:teuthology.orchestra.run.smithi071.stdout: "filer_max_truncate_ops": "128", 2024-09-14T05:29:21.960 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_apply_finisher_threads": "1", 2024-09-14T05:29:21.961 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_blackhole": "false", 2024-09-14T05:29:21.961 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_btrfs_clone_range": "true", 2024-09-14T05:29:21.961 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_btrfs_snap": "true", 2024-09-14T05:29:21.961 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_caller_concurrency": "10", 2024-09-14T05:29:21.961 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_collect_device_partition_information": "true", 2024-09-14T05:29:21.961 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_commit_timeout": "600.000000", 2024-09-14T05:29:21.961 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_debug_inject_read_err": "false", 2024-09-14T05:29:21.961 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_debug_omap_check": "false", 2024-09-14T05:29:21.961 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_debug_verify_split": "false", 2024-09-14T05:29:21.961 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_dump_file": "", 2024-09-14T05:29:21.961 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-14T05:29:21.961 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-14T05:29:21.962 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_fadvise": "true", 2024-09-14T05:29:21.962 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_fail_eio": "true", 2024-09-14T05:29:21.962 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_fd_cache_shards": "16", 2024-09-14T05:29:21.962 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_fd_cache_size": "128", 2024-09-14T05:29:21.962 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_fiemap": "false", 2024-09-14T05:29:21.962 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_fiemap_threshold": "4096", 2024-09-14T05:29:21.962 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-14T05:29:21.962 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-14T05:29:21.962 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_inject_stall": "0", 2024-09-14T05:29:21.962 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_journal_parallel": "false", 2024-09-14T05:29:21.963 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_journal_trailing": "false", 2024-09-14T05:29:21.963 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_journal_writeahead": "false", 2024-09-14T05:29:21.963 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_kill_at": "0", 2024-09-14T05:29:21.963 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-14T05:29:21.963 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-14T05:29:21.963 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-14T05:29:21.963 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-14T05:29:21.963 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-14T05:29:21.963 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_inline_xattrs": "0", 2024-09-14T05:29:21.963 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-14T05:29:21.963 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-14T05:29:21.963 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-14T05:29:21.963 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-14T05:29:21.964 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_xattr_value_size": "0", 2024-09-14T05:29:21.964 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-14T05:29:21.964 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-14T05:29:21.964 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-14T05:29:21.964 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_merge_threshold": "-10", 2024-09-14T05:29:21.964 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-14T05:29:21.964 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_odsync_write": "false", 2024-09-14T05:29:21.964 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_omap_backend": "rocksdb", 2024-09-14T05:29:21.964 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_omap_backend_path": "", 2024-09-14T05:29:21.964 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-14T05:29:21.964 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-14T05:29:21.964 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-14T05:29:21.964 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_op_thread_timeout": "60", 2024-09-14T05:29:21.965 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_op_threads": "2", 2024-09-14T05:29:21.965 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_punch_hole": "false", 2024-09-14T05:29:21.965 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-14T05:29:21.965 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-14T05:29:21.965 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-14T05:29:21.965 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-14T05:29:21.965 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-14T05:29:21.965 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-14T05:29:21.965 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-14T05:29:21.965 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-14T05:29:21.965 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-14T05:29:21.965 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_queue_max_ops": "50", 2024-09-14T05:29:21.965 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-14T05:29:21.966 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_seek_data_hole": "false", 2024-09-14T05:29:21.966 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_sloppy_crc": "false", 2024-09-14T05:29:21.966 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-14T05:29:21.966 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_splice": "false", 2024-09-14T05:29:21.966 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_split_multiple": "2", 2024-09-14T05:29:21.966 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_split_rand_factor": "20", 2024-09-14T05:29:21.966 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_update_to": "1000", 2024-09-14T05:29:21.966 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-14T05:29:21.966 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-14T05:29:21.966 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-14T05:29:21.966 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-14T05:29:21.966 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-14T05:29:21.966 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-14T05:29:21.966 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_enable": "true", 2024-09-14T05:29:21.967 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-14T05:29:21.967 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-14T05:29:21.967 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-14T05:29:21.967 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-14T05:29:21.967 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-14T05:29:21.967 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-14T05:29:21.967 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_xfs_extsize": "false", 2024-09-14T05:29:21.967 INFO:teuthology.orchestra.run.smithi071.stdout: "filestore_zfs_snap": "false", 2024-09-14T05:29:21.967 INFO:teuthology.orchestra.run.smithi071.stdout: "fio_dir": "/tmp/fio", 2024-09-14T05:29:21.967 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "1127be44-5f14-4e7d-ad23-f608afe154f5", 2024-09-14T05:29:21.967 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_allow_other": "true", 2024-09-14T05:29:21.967 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_atomic_o_trunc": "true", 2024-09-14T05:29:21.967 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_big_writes": "true", 2024-09-14T05:29:21.967 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_debug": "false", 2024-09-14T05:29:21.967 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_default_permissions": "false", 2024-09-14T05:29:21.968 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_disable_pagecache": "false", 2024-09-14T05:29:21.968 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_max_write": "0", 2024-09-14T05:29:21.968 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_multithreaded": "true", 2024-09-14T05:29:21.968 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_require_active_mds": "true", 2024-09-14T05:29:21.968 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_set_user_groups": "true", 2024-09-14T05:29:21.968 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_splice_move": "true", 2024-09-14T05:29:21.968 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_splice_read": "true", 2024-09-14T05:29:21.968 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_splice_write": "true", 2024-09-14T05:29:21.968 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-14T05:29:21.968 INFO:teuthology.orchestra.run.smithi071.stdout: "fuse_use_invalidate_cb": "true", 2024-09-14T05:29:21.968 INFO:teuthology.orchestra.run.smithi071.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-14T05:29:21.968 INFO:teuthology.orchestra.run.smithi071.stdout: "gss_target_name": "ceph", 2024-09-14T05:29:21.968 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_file": "", 2024-09-14T05:29:21.968 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_inject_failure": "0", 2024-09-14T05:29:21.969 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_interval": "5", 2024-09-14T05:29:21.969 INFO:teuthology.orchestra.run.smithi071.stdout: "host": "smithi071", 2024-09-14T05:29:21.969 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-14T05:29:21.969 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-14T05:29:21.969 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-14T05:29:21.969 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-14T05:29:21.969 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-14T05:29:21.969 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-14T05:29:21.969 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-14T05:29:21.969 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-14T05:29:21.969 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-14T05:29:21.969 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-14T05:29:21.969 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-14T05:29:21.970 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-14T05:29:21.970 INFO:teuthology.orchestra.run.smithi071.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-14T05:29:21.970 INFO:teuthology.orchestra.run.smithi071.stdout: "inject_early_sigterm": "false", 2024-09-14T05:29:21.970 INFO:teuthology.orchestra.run.smithi071.stdout: "jaeger_agent_port": "6799", 2024-09-14T05:29:21.970 INFO:teuthology.orchestra.run.smithi071.stdout: "jaeger_tracing_enable": "false", 2024-09-14T05:29:21.970 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_aio": "true", 2024-09-14T05:29:21.970 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_align_min_size": "65536", 2024-09-14T05:29:21.970 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_block_align": "true", 2024-09-14T05:29:21.970 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_block_size": "4096", 2024-09-14T05:29:21.970 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_dio": "true", 2024-09-14T05:29:21.970 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_discard": "false", 2024-09-14T05:29:21.970 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_force_aio": "false", 2024-09-14T05:29:21.970 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_ignore_corruption": "false", 2024-09-14T05:29:21.970 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_max_write_bytes": "10485760", 2024-09-14T05:29:21.970 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_max_write_entries": "100", 2024-09-14T05:29:21.971 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_replay_from": "0", 2024-09-14T05:29:21.971 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-14T05:29:21.971 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-14T05:29:21.971 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-14T05:29:21.971 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-14T05:29:21.971 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_write_header_frequency": "0", 2024-09-14T05:29:21.971 INFO:teuthology.orchestra.run.smithi071.stdout: "journal_zero_on_create": "false", 2024-09-14T05:29:21.971 INFO:teuthology.orchestra.run.smithi071.stdout: "journaler_prefetch_periods": "10", 2024-09-14T05:29:21.971 INFO:teuthology.orchestra.run.smithi071.stdout: "journaler_prezero_periods": "5", 2024-09-14T05:29:21.971 INFO:teuthology.orchestra.run.smithi071.stdout: "journaler_write_head_interval": "15", 2024-09-14T05:29:21.971 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "", 2024-09-14T05:29:21.971 INFO:teuthology.orchestra.run.smithi071.stdout: "keyfile": "", 2024-09-14T05:29:21.971 INFO:teuthology.orchestra.run.smithi071.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-14T05:29:21.971 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_backend": "rocksdb", 2024-09-14T05:29:21.972 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_default_stripe_size": "65536", 2024-09-14T05:29:21.972 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_fsck_on_mount": "false", 2024-09-14T05:29:21.972 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-14T05:29:21.972 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_max_bytes": "67108864", 2024-09-14T05:29:21.972 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_max_ops": "512", 2024-09-14T05:29:21.972 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_nid_prealloc": "1024", 2024-09-14T05:29:21.972 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_onode_map_size": "1024", 2024-09-14T05:29:21.972 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-14T05:29:21.972 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_sync_submit_transaction": "false", 2024-09-14T05:29:21.972 INFO:teuthology.orchestra.run.smithi071.stdout: "kstore_sync_transaction": "false", 2024-09-14T05:29:21.972 INFO:teuthology.orchestra.run.smithi071.stdout: "librados_thread_count": "2", 2024-09-14T05:29:21.972 INFO:teuthology.orchestra.run.smithi071.stdout: "lockdep": "false", 2024-09-14T05:29:21.972 INFO:teuthology.orchestra.run.smithi071.stdout: "lockdep_force_backtrace": "false", 2024-09-14T05:29:21.973 INFO:teuthology.orchestra.run.smithi071.stdout: "log_coarse_timestamps": "true", 2024-09-14T05:29:21.973 INFO:teuthology.orchestra.run.smithi071.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-14T05:29:21.973 INFO:teuthology.orchestra.run.smithi071.stdout: "log_flush_on_exit": "false", 2024-09-14T05:29:21.973 INFO:teuthology.orchestra.run.smithi071.stdout: "log_graylog_host": "127.0.0.1", 2024-09-14T05:29:21.973 INFO:teuthology.orchestra.run.smithi071.stdout: "log_graylog_port": "12201", 2024-09-14T05:29:21.973 INFO:teuthology.orchestra.run.smithi071.stdout: "log_max_new": "1000", 2024-09-14T05:29:21.973 INFO:teuthology.orchestra.run.smithi071.stdout: "log_max_recent": "10000", 2024-09-14T05:29:21.973 INFO:teuthology.orchestra.run.smithi071.stdout: "log_stderr_prefix": "", 2024-09-14T05:29:21.973 INFO:teuthology.orchestra.run.smithi071.stdout: "log_stop_at_utilization": "0.970000", 2024-09-14T05:29:21.973 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": "true", 2024-09-14T05:29:21.973 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_graylog": "false", 2024-09-14T05:29:21.973 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_journald": "false", 2024-09-14T05:29:21.973 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_stderr": "false", 2024-09-14T05:29:21.973 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_syslog": "false", 2024-09-14T05:29:21.974 INFO:teuthology.orchestra.run.smithi071.stdout: "max_rotating_auth_attempts": "10", 2024-09-14T05:29:21.974 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-14T05:29:21.974 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_action_on_write_error": "1", 2024-09-14T05:29:21.974 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_alternate_name_max": "8192", 2024-09-14T05:29:21.974 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_asio_thread_count": "2", 2024-09-14T05:29:21.974 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_export_pin": "true", 2024-09-14T05:29:21.974 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_fragment_dirs": "true", 2024-09-14T05:29:21.974 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-14T05:29:21.974 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_fragment_interval": "5", 2024-09-14T05:29:21.974 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-14T05:29:21.974 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-14T05:29:21.974 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_interval": "10", 2024-09-14T05:29:21.974 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_max": "-1", 2024-09-14T05:29:21.974 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_max_until": "-1", 2024-09-14T05:29:21.974 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_merge_size": "50", 2024-09-14T05:29:21.975 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_midchunk": "0.300000", 2024-09-14T05:29:21.975 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-14T05:29:21.975 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_min_start": "0.200000", 2024-09-14T05:29:21.975 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_minchunk": "0.001000", 2024-09-14T05:29:21.975 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_mode": "0", 2024-09-14T05:29:21.975 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_need_max": "1.200000", 2024-09-14T05:29:21.975 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_need_min": "0.800000", 2024-09-14T05:29:21.975 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_overload_epochs": "2", 2024-09-14T05:29:21.975 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-14T05:29:21.975 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-14T05:29:21.975 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_split_bits": "3", 2024-09-14T05:29:21.975 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-14T05:29:21.975 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_split_size": "10000", 2024-09-14T05:29:21.975 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-14T05:29:21.975 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_target_decay": "10.000000", 2024-09-14T05:29:21.976 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-14T05:29:21.976 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_beacon_grace": "15.000000", 2024-09-14T05:29:21.976 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_beacon_interval": "4.000000", 2024-09-14T05:29:21.976 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-14T05:29:21.976 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-14T05:29:21.976 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_mid": "0.700000", 2024-09-14T05:29:21.976 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-14T05:29:21.976 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_quiesce_delay": "0", 2024-09-14T05:29:21.976 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-14T05:29:21.976 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-14T05:29:21.976 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-14T05:29:21.976 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_release_free_interval": "10", 2024-09-14T05:29:21.976 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_reservation": "0.050000", 2024-09-14T05:29:21.976 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-14T05:29:21.977 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_trim_interval": "1", 2024-09-14T05:29:21.977 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache_trim_threshold": "262144", 2024-09-14T05:29:21.977 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-14T05:29:21.977 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-14T05:29:21.977 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-14T05:29:21.977 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_client_prealloc_inos": "1000", 2024-09-14T05:29:21.977 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-14T05:29:21.977 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_connect_bootstrapping": "false", 2024-09-14T05:29:21.977 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_damage_table_max_entries": "10000", 2024-09-14T05:29:21.977 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-14T05:29:21.977 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_debug_auth_pins": "false", 2024-09-14T05:29:21.977 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_debug_frag": "false", 2024-09-14T05:29:21.977 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_debug_scatterstat": "false", 2024-09-14T05:29:21.977 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_debug_subtrees": "false", 2024-09-14T05:29:21.978 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_decay_halflife": "5.000000", 2024-09-14T05:29:21.978 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_default_dir_hash": "2", 2024-09-14T05:29:21.978 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_defer_session_stale": "true", 2024-09-14T05:29:21.978 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_deny_all_reconnect": "false", 2024-09-14T05:29:21.978 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_dir_keys_per_op": "16384", 2024-09-14T05:29:21.978 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_dir_max_commit_size": "10", 2024-09-14T05:29:21.978 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_dir_max_entries": "0", 2024-09-14T05:29:21.978 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_dir_prefetch": "true", 2024-09-14T05:29:21.978 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-14T05:29:21.978 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-14T05:29:21.978 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_dump_cache_on_map": "false", 2024-09-14T05:29:21.978 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-14T05:29:21.978 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-14T05:29:21.978 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_early_reply": "true", 2024-09-14T05:29:21.978 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_enable_op_tracker": "true", 2024-09-14T05:29:21.979 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_enforce_unique_name": "true", 2024-09-14T05:29:21.979 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-14T05:29:21.979 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-14T05:29:21.979 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_export_ephemeral_random": "true", 2024-09-14T05:29:21.979 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-14T05:29:21.979 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-14T05:29:21.979 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-14T05:29:21.979 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-14T05:29:21.979 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-14T05:29:21.979 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-14T05:29:21.979 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-14T05:29:21.979 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-14T05:29:21.979 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_health_summarize_threshold": "10", 2024-09-14T05:29:21.979 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-14T05:29:21.979 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-14T05:29:21.980 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_inject_health_dummy": "false", 2024-09-14T05:29:21.980 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-14T05:29:21.980 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_inject_migrator_session_race": "false", 2024-09-14T05:29:21.980 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-14T05:29:21.980 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-14T05:29:21.980 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-14T05:29:21.980 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_join_fs": "", 2024-09-14T05:29:21.980 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_journal_format": "1", 2024-09-14T05:29:21.980 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-14T05:29:21.980 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_create_at": "0", 2024-09-14T05:29:21.980 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_dirfrag_at": "0", 2024-09-14T05:29:21.980 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_export_at": "0", 2024-09-14T05:29:21.981 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_import_at": "0", 2024-09-14T05:29:21.981 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_journal_at": "0", 2024-09-14T05:29:21.981 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_journal_expire_at": "0", 2024-09-14T05:29:21.981 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_journal_replay_at": "0", 2024-09-14T05:29:21.981 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_link_at": "0", 2024-09-14T05:29:21.981 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_mdstable_at": "0", 2024-09-14T05:29:21.981 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_openc_at": "0", 2024-09-14T05:29:21.981 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_rename_at": "0", 2024-09-14T05:29:21.981 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_kill_shutdown_at": "0", 2024-09-14T05:29:21.981 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_event_large_threshold": "524288", 2024-09-14T05:29:21.981 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_events_per_segment": "1024", 2024-09-14T05:29:21.981 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-14T05:29:21.981 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_max_events": "-1", 2024-09-14T05:29:21.982 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_max_segments": "128", 2024-09-14T05:29:21.982 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_pause": "false", 2024-09-14T05:29:21.982 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_segment_size": "0", 2024-09-14T05:29:21.982 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-14T05:29:21.982 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-14T05:29:21.982 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-14T05:29:21.982 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_trim_threshold": "128", 2024-09-14T05:29:21.982 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-14T05:29:21.982 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log_warn_factor": "2.000000", 2024-09-14T05:29:21.982 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_caps_per_client": "1048576", 2024-09-14T05:29:21.982 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_completed_flushes": "100000", 2024-09-14T05:29:21.982 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_completed_requests": "100000", 2024-09-14T05:29:21.982 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_export_size": "20971520", 2024-09-14T05:29:21.982 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_file_recover": "32", 2024-09-14T05:29:21.983 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_purge_files": "64", 2024-09-14T05:29:21.983 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_purge_ops": "8192", 2024-09-14T05:29:21.983 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-14T05:29:21.983 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-14T05:29:21.983 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_max_snaps_per_dir": "100", 2024-09-14T05:29:21.983 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_metrics_update_interval": "2", 2024-09-14T05:29:21.983 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_min_caps_per_client": "100", 2024-09-14T05:29:21.983 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_min_caps_working_set": "10000", 2024-09-14T05:29:21.983 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-14T05:29:21.983 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_numa_node": "-1", 2024-09-14T05:29:21.983 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-14T05:29:21.983 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_op_complaint_time": "30.000000", 2024-09-14T05:29:21.983 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_op_history_duration": "600", 2024-09-14T05:29:21.984 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_op_history_size": "20", 2024-09-14T05:29:21.984 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_op_history_slow_op_size": "20", 2024-09-14T05:29:21.984 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-14T05:29:21.984 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_op_log_threshold": "5", 2024-09-14T05:29:21.984 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_ping_grace": "15", 2024-09-14T05:29:21.984 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_ping_interval": "5", 2024-09-14T05:29:21.984 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-14T05:29:21.984 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-14T05:29:21.984 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_recall_max_caps": "30000", 2024-09-14T05:29:21.984 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-14T05:29:21.984 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-14T05:29:21.984 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-14T05:29:21.984 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_recall_warning_threshold": "262144", 2024-09-14T05:29:21.984 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-14T05:29:21.985 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_replay_interval": "1.000000", 2024-09-14T05:29:21.985 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-14T05:29:21.985 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-14T05:29:21.985 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_root_ino_gid": "0", 2024-09-14T05:29:21.985 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_root_ino_uid": "0", 2024-09-14T05:29:21.985 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-14T05:29:21.985 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_scrub_stats_review_period": "1", 2024-09-14T05:29:21.985 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-14T05:29:21.985 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-14T05:29:21.985 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-14T05:29:21.985 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-14T05:29:21.985 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-14T05:29:21.986 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-14T05:29:21.986 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-14T05:29:21.986 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-14T05:29:21.986 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-14T05:29:21.986 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-14T05:29:21.986 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-14T05:29:21.986 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_shutdown_check": "0", 2024-09-14T05:29:21.986 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_skip_ino": "0", 2024-09-14T05:29:21.986 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-14T05:29:21.986 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_snap_max_uid": "4294967294", 2024-09-14T05:29:21.986 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_snap_min_uid": "0", 2024-09-14T05:29:21.986 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_snap_rstat": "false", 2024-09-14T05:29:21.986 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_standby_replay_damaged": "false", 2024-09-14T05:29:21.986 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_symlink_recovery": "true", 2024-09-14T05:29:21.987 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-14T05:29:21.987 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_thrash_exports": "0", 2024-09-14T05:29:21.987 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_thrash_fragments": "0", 2024-09-14T05:29:21.987 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_tick_interval": "5.000000", 2024-09-14T05:29:21.987 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_valgrind_exit": "false", 2024-09-14T05:29:21.987 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_verify_backtrace": "1", 2024-09-14T05:29:21.987 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_verify_scatter": "false", 2024-09-14T05:29:21.987 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-14T05:29:21.987 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_wipe_sessions": "false", 2024-09-14T05:29:21.987 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool_debug": "false", 2024-09-14T05:29:21.987 INFO:teuthology.orchestra.run.smithi071.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-14T05:29:21.987 INFO:teuthology.orchestra.run.smithi071.stdout: "memstore_device_bytes": "1073741824", 2024-09-14T05:29:21.987 INFO:teuthology.orchestra.run.smithi071.stdout: "memstore_page_set": "false", 2024-09-14T05:29:21.988 INFO:teuthology.orchestra.run.smithi071.stdout: "memstore_page_size": "65536", 2024-09-14T05:29:21.988 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_client_bytes": "134217728", 2024-09-14T05:29:21.988 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_client_messages": "512", 2024-09-14T05:29:21.988 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-14T05:29:21.988 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-14T05:29:21.988 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-14T05:29:21.988 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-14T05:29:21.988 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_disabled_modules": "", 2024-09-14T05:29:21.988 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_enable_op_tracker": "true", 2024-09-14T05:29:21.988 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-14T05:29:21.988 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_max_pg_creating": "1024", 2024-09-14T05:29:21.988 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_max_pg_num_change": "128", 2024-09-14T05:29:21.988 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_mds_bytes": "134217728", 2024-09-14T05:29:21.989 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_mds_messages": "128", 2024-09-14T05:29:21.989 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-14T05:29:21.989 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_mon_bytes": "134217728", 2024-09-14T05:29:21.989 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_mon_messages": "128", 2024-09-14T05:29:21.989 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-14T05:29:21.989 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-14T05:29:21.989 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_op_history_duration": "600", 2024-09-14T05:29:21.989 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_op_history_size": "20", 2024-09-14T05:29:21.989 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-14T05:29:21.989 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-14T05:29:21.989 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_op_log_threshold": "5", 2024-09-14T05:29:21.989 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_osd_bytes": "536870912", 2024-09-14T05:29:21.989 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_osd_messages": "8192", 2024-09-14T05:29:21.989 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_pool": "true", 2024-09-14T05:29:21.990 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-14T05:29:21.990 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_standby_modules": "true", 2024-09-14T05:29:21.990 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_stats_period": "5", 2024-09-14T05:29:21.990 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_stats_threshold": "5", 2024-09-14T05:29:21.990 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_tick_period": "2", 2024-09-14T05:29:21.990 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-14T05:29:21.990 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-14T05:29:21.990 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_allow_pool_delete": "true", 2024-09-14T05:29:21.990 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_allow_pool_size_one": "true", 2024-09-14T05:29:21.990 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_auth_validate_all_caps": "true", 2024-09-14T05:29:21.990 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-14T05:29:21.990 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-14T05:29:21.990 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_bytes": "104857600", 2024-09-14T05:29:21.991 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_directed_command_retry": "5", 2024-09-14T05:29:21.991 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-14T05:29:21.991 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-14T05:29:21.991 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-14T05:29:21.991 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-14T05:29:21.991 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_hunt_parallel": "3", 2024-09-14T05:29:21.991 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_log_interval": "1.000000", 2024-09-14T05:29:21.991 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-14T05:29:21.991 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_ping_interval": "10.000000", 2024-09-14T05:29:21.991 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-14T05:29:21.991 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_client_target_rank": "-1", 2024-09-14T05:29:21.991 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-14T05:29:21.991 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-14T05:29:21.991 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-14T05:29:21.992 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_level": "debug", 2024-09-14T05:29:21.992 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_to_file": "true", 2024-09-14T05:29:21.992 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-14T05:29:21.992 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-14T05:29:21.992 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-14T05:29:21.992 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_to_journald": "false", 2024-09-14T05:29:21.992 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-14T05:29:21.992 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-14T05:29:21.992 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-14T05:29:21.992 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_compact_on_bootstrap": "false", 2024-09-14T05:29:21.992 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_compact_on_start": "false", 2024-09-14T05:29:21.992 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_compact_on_trim": "true", 2024-09-14T05:29:21.992 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-14T05:29:21.992 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-14T05:29:21.992 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-14T05:29:21.993 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_cpu_threads": "4", 2024-09-14T05:29:21.993 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_crush_min_required_version": "hammer", 2024-09-14T05:29:21.993 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_daemon_bytes": "419430400", 2024-09-14T05:29:21.993 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-14T05:29:21.993 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_data_avail_crit": "5", 2024-09-14T05:29:21.993 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_data_avail_warn": "5", 2024-09-14T05:29:21.993 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_data_size_warn": "16106127360", 2024-09-14T05:29:21.993 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-14T05:29:21.993 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-14T05:29:21.993 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_dump_json": "false", 2024-09-14T05:29:21.993 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-14T05:29:21.993 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_dump_transactions": "false", 2024-09-14T05:29:21.993 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_extra_checks": "false", 2024-09-14T05:29:21.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-14T05:29:21.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-14T05:29:21.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_no_require_reef": "false", 2024-09-14T05:29:21.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_no_require_squid": "false", 2024-09-14T05:29:21.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-14T05:29:21.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-14T05:29:21.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-14T05:29:21.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_down_mkfs_grace": "120", 2024-09-14T05:29:21.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_down_uptime_grace": "60", 2024-09-14T05:29:21.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_election_default_strategy": "1", 2024-09-14T05:29:21.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_election_timeout": "5.000000", 2024-09-14T05:29:21.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-14T05:29:21.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_elector_ping_divisor": "2", 2024-09-14T05:29:21.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-14T05:29:21.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_enable_op_tracker": "true", 2024-09-14T05:29:21.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_fake_pool_delete": "false", 2024-09-14T05:29:21.995 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_force_quorum_join": "false", 2024-09-14T05:29:21.995 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-14T05:29:21.995 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_globalid_prealloc": "10000", 2024-09-14T05:29:21.995 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_health_detail_to_clog": "false", 2024-09-14T05:29:21.995 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_health_log_update_period": "5", 2024-09-14T05:29:21.995 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_health_max_detail": "50", 2024-09-14T05:29:21.995 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_health_to_clog": "true", 2024-09-14T05:29:21.995 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_health_to_clog_interval": "600", 2024-09-14T05:29:21.995 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-14T05:29:21.995 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_host": "172.21.15.71,[v2:172.21.15.71:3301,v1:172.21.15.71:6790],172.21.15.150", 2024-09-14T05:29:21.995 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_host_override": "", 2024-09-14T05:29:21.995 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_initial_members": "", 2024-09-14T05:29:21.995 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-14T05:29:21.995 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-14T05:29:21.996 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-14T05:29:21.996 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-14T05:29:21.996 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-14T05:29:21.996 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_lease": "5.000000", 2024-09-14T05:29:21.996 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-14T05:29:21.996 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-14T05:29:21.996 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_log_full_interval": "50", 2024-09-14T05:29:21.996 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_log_max": "10000", 2024-09-14T05:29:21.996 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_log_max_summary": "50", 2024-09-14T05:29:21.996 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-14T05:29:21.996 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_max_log_epochs": "500", 2024-09-14T05:29:21.996 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-14T05:29:21.996 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-14T05:29:21.997 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-14T05:29:21.997 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_max_osd": "10000", 2024-09-14T05:29:21.997 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_max_pg_per_osd": "10000", 2024-09-14T05:29:21.997 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_max_pool_pg_num": "65536", 2024-09-14T05:29:21.997 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-14T05:29:21.997 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-14T05:29:21.997 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_mds_force_trim_to": "0", 2024-09-14T05:29:21.997 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_mds_skip_sanity": "false", 2024-09-14T05:29:21.997 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_memory_autotune": "true", 2024-09-14T05:29:21.997 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_memory_target": "2147483648", 2024-09-14T05:29:21.997 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_mgr_beacon_grace": "30", 2024-09-14T05:29:21.997 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-14T05:29:21.997 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_mgr_digest_period": "5", 2024-09-14T05:29:21.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_mgr_inactive_grace": "60", 2024-09-14T05:29:21.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-14T05:29:21.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-14T05:29:21.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_min_osdmap_epochs": "500", 2024-09-14T05:29:21.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-14T05:29:21.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-14T05:29:21.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_op_complaint_time": "30", 2024-09-14T05:29:21.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_op_history_duration": "600", 2024-09-14T05:29:21.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_op_history_size": "20", 2024-09-14T05:29:21.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_op_history_slow_op_size": "20", 2024-09-14T05:29:21.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-14T05:29:21.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_op_log_threshold": "5", 2024-09-14T05:29:21.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-14T05:29:21.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-14T05:29:21.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-14T05:29:21.999 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_auto_mark_in": "false", 2024-09-14T05:29:21.999 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-14T05:29:21.999 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-14T05:29:21.999 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-14T05:29:21.999 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_cache_size": "500", 2024-09-14T05:29:21.999 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-14T05:29:21.999 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-14T05:29:21.999 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-14T05:29:21.999 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_down_out_interval": "600", 2024-09-14T05:29:21.999 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-14T05:29:21.999 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-14T05:29:21.999 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_force_trim_to": "0", 2024-09-14T05:29:21.999 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-14T05:29:22.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-14T05:29:22.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-14T05:29:22.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-14T05:29:22.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-14T05:29:22.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-14T05:29:22.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-14T05:29:22.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_min_down_reporters": "2", 2024-09-14T05:29:22.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-14T05:29:22.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-14T05:29:22.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-14T05:29:22.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-14T05:29:22.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-14T05:29:22.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-14T05:29:22.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_report_timeout": "900", 2024-09-14T05:29:22.001 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-14T05:29:22.001 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-14T05:29:22.001 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-14T05:29:22.001 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-14T05:29:22.001 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-14T05:29:22.001 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-14T05:29:22.001 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-14T05:29:22.001 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-14T05:29:22.001 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-14T05:29:22.001 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_pg_stuck_threshold": "60", 2024-09-14T05:29:22.001 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-14T05:29:22.001 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-14T05:29:22.001 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-14T05:29:22.002 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-14T05:29:22.002 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-14T05:29:22.002 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-14T05:29:22.002 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_probe_timeout": "2.000000", 2024-09-14T05:29:22.002 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_reweight_max_change": "0.050000", 2024-09-14T05:29:22.002 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_reweight_max_osds": "4", 2024-09-14T05:29:22.002 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-14T05:29:22.002 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-14T05:29:22.002 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-14T05:29:22.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-14T05:29:22.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-14T05:29:22.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_scrub_interval": "86400", 2024-09-14T05:29:22.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_scrub_max_keys": "100", 2024-09-14T05:29:22.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_scrub_timeout": "300", 2024-09-14T05:29:22.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_session_timeout": "300", 2024-09-14T05:29:22.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_smart_report_timeout": "5", 2024-09-14T05:29:22.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_stat_smooth_intervals": "6", 2024-09-14T05:29:22.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-14T05:29:22.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_stretch_pool_min_size": "2", 2024-09-14T05:29:22.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_stretch_pool_size": "4", 2024-09-14T05:29:22.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-14T05:29:22.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-14T05:29:22.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_sync_debug": "false", 2024-09-14T05:29:22.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-14T05:29:22.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-14T05:29:22.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_sync_provider_kill_at": "0", 2024-09-14T05:29:22.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_sync_requester_kill_at": "0", 2024-09-14T05:29:22.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_sync_timeout": "60.000000", 2024-09-14T05:29:22.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_target_pg_per_osd": "100", 2024-09-14T05:29:22.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_tick_interval": "5", 2024-09-14T05:29:22.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_timecheck_interval": "300.000000", 2024-09-14T05:29:22.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-14T05:29:22.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_use_min_delay_socket": "false", 2024-09-14T05:29:22.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_older_version_delay": "604800", 2024-09-14T05:29:22.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-14T05:29:22.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-14T05:29:22.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-14T05:29:22.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-14T05:29:22.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-14T05:29:22.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-14T05:29:22.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-14T05:29:22.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_misplaced": "false", 2024-09-14T05:29:22.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-14T05:29:22.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_older_version": "true", 2024-09-14T05:29:22.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-14T05:29:22.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-14T05:29:22.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-14T05:29:22.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-14T05:29:22.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-14T05:29:22.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-14T05:29:22.006 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-14T05:29:22.006 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-14T05:29:22.006 INFO:teuthology.orchestra.run.smithi071.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-14T05:29:22.006 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": "", 2024-09-14T05:29:22.006 INFO:teuthology.orchestra.run.smithi071.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-14T05:29:22.006 INFO:teuthology.orchestra.run.smithi071.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-14T05:29:22.006 INFO:teuthology.orchestra.run.smithi071.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-14T05:29:22.006 INFO:teuthology.orchestra.run.smithi071.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-14T05:29:22.006 INFO:teuthology.orchestra.run.smithi071.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-14T05:29:22.006 INFO:teuthology.orchestra.run.smithi071.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-14T05:29:22.006 INFO:teuthology.orchestra.run.smithi071.stdout: "motr_tracing_enabled": "false", 2024-09-14T05:29:22.006 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_op_threads": "3", 2024-09-14T05:29:22.006 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-14T05:29:22.006 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_cm": "false", 2024-09-14T05:29:22.007 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_device_name": "", 2024-09-14T05:29:22.007 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_dscp": "96", 2024-09-14T05:29:22.007 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-14T05:29:22.007 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-14T05:29:22.007 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_local_gid": "", 2024-09-14T05:29:22.007 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-14T05:29:22.007 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_port_num": "1", 2024-09-14T05:29:22.007 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-14T05:29:22.007 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-14T05:29:22.007 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-14T05:29:22.007 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-14T05:29:22.007 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_sl": "3", 2024-09-14T05:29:22.007 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_support_srq": "true", 2024-09-14T05:29:22.007 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_rdma_type": "ib", 2024-09-14T05:29:22.007 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_async_reap_threshold": "5", 2024-09-14T05:29:22.008 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_before_connect": "false", 2024-09-14T05:29:22.008 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_ipv4": "true", 2024-09-14T05:29:22.008 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_ipv6": "false", 2024-09-14T05:29:22.008 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_msgr1": "true", 2024-09-14T05:29:22.008 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_msgr2": "true", 2024-09-14T05:29:22.008 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_port_max": "7568", 2024-09-14T05:29:22.008 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_port_min": "6800", 2024-09-14T05:29:22.008 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-14T05:29:22.008 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_retry_count": "3", 2024-09-14T05:29:22.008 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_bind_retry_delay": "5", 2024-09-14T05:29:22.008 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_blackhole_client": "false", 2024-09-14T05:29:22.008 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_blackhole_mds": "false", 2024-09-14T05:29:22.008 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_blackhole_mgr": "false", 2024-09-14T05:29:22.009 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_blackhole_mon": "false", 2024-09-14T05:29:22.009 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_blackhole_osd": "false", 2024-09-14T05:29:22.009 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_client_mode": "crc secure", 2024-09-14T05:29:22.009 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-14T05:29:22.009 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_cluster_mode": "crc secure", 2024-09-14T05:29:22.009 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_cluster_type": "", 2024-09-14T05:29:22.009 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_compress_secure": "false", 2024-09-14T05:29:22.009 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_connection_idle_timeout": "900", 2024-09-14T05:29:22.009 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_connection_ready_timeout": "10", 2024-09-14T05:29:22.009 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_crc_data": "true", 2024-09-14T05:29:22.009 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_crc_header": "true", 2024-09-14T05:29:22.009 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_die_on_bad_msg": "false", 2024-09-14T05:29:22.009 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_die_on_bug": "true", 2024-09-14T05:29:22.010 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_die_on_old_message": "true", 2024-09-14T05:29:22.010 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_die_on_skipped_message": "false", 2024-09-14T05:29:22.010 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-14T05:29:22.010 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-14T05:29:22.010 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_coremask": "0xF", 2024-09-14T05:29:22.010 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-14T05:29:22.010 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-14T05:29:22.010 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_enable_tso": "true", 2024-09-14T05:29:22.010 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-14T05:29:22.010 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-14T05:29:22.010 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_hugepages": "", 2024-09-14T05:29:22.010 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-14T05:29:22.010 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-14T05:29:22.011 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_lro": "true", 2024-09-14T05:29:22.011 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_memory_channel": "4", 2024-09-14T05:29:22.011 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-14T05:29:22.011 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_pmd": "", 2024-09-14T05:29:22.011 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_port_id": "0", 2024-09-14T05:29:22.011 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-14T05:29:22.011 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-14T05:29:22.011 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_dump_on_send": "false", 2024-09-14T05:29:22.011 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_initial_backoff": "0.200000", 2024-09-14T05:29:22.011 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_inject_delay_max": "1.000000", 2024-09-14T05:29:22.011 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-14T05:29:22.011 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_inject_delay_type": "", 2024-09-14T05:29:22.011 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-14T05:29:22.011 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_inject_network_congestion": "0", 2024-09-14T05:29:22.011 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_inject_socket_failures": "5000", 2024-09-14T05:29:22.012 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_learn_addr_from_peer": "true", 2024-09-14T05:29:22.012 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_max_accept_failures": "4", 2024-09-14T05:29:22.012 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_max_backoff": "15.000000", 2024-09-14T05:29:22.012 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_mon_client_mode": "secure crc", 2024-09-14T05:29:22.012 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-14T05:29:22.012 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_mon_service_mode": "secure crc", 2024-09-14T05:29:22.012 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_osd_compress_min_size": "1024", 2024-09-14T05:29:22.012 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_osd_compress_mode": "none", 2024-09-14T05:29:22.012 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-14T05:29:22.012 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-14T05:29:22.012 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_pq_min_cost": "65536", 2024-09-14T05:29:22.012 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_public_type": "", 2024-09-14T05:29:22.012 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_service_mode": "crc secure", 2024-09-14T05:29:22.012 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_tcp_listen_backlog": "512", 2024-09-14T05:29:22.012 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_tcp_nodelay": "true", 2024-09-14T05:29:22.013 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-14T05:29:22.013 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_tcp_rcvbuf": "0", 2024-09-14T05:29:22.013 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-14T05:29:22.013 INFO:teuthology.orchestra.run.smithi071.stdout: "ms_type": "async+posix", 2024-09-14T05:29:22.013 INFO:teuthology.orchestra.run.smithi071.stdout: "no_config_file": "false", 2024-09-14T05:29:22.013 INFO:teuthology.orchestra.run.smithi071.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-14T05:29:22.013 INFO:teuthology.orchestra.run.smithi071.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-14T05:29:22.013 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter_completion_locks_per_session": "32", 2024-09-14T05:29:22.013 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-14T05:29:22.013 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-14T05:29:22.013 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter_inflight_ops": "1024", 2024-09-14T05:29:22.013 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-14T05:29:22.013 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-14T05:29:22.013 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter_tick_interval": "5.000000", 2024-09-14T05:29:22.013 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter_timeout": "10.000000", 2024-09-14T05:29:22.014 INFO:teuthology.orchestra.run.smithi071.stdout: "objectstore_blackhole": "false", 2024-09-14T05:29:22.014 INFO:teuthology.orchestra.run.smithi071.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-14T05:29:22.014 INFO:teuthology.orchestra.run.smithi071.stdout: "openssl_engine_opts": "", 2024-09-14T05:29:22.014 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_agent_delay_time": "5.000000", 2024-09-14T05:29:22.014 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_agent_hist_halflife": "1000", 2024-09-14T05:29:22.014 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_agent_max_low_ops": "2", 2024-09-14T05:29:22.014 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_agent_max_ops": "4", 2024-09-14T05:29:22.014 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-14T05:29:22.014 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-14T05:29:22.014 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_agent_slop": "0.020000", 2024-09-14T05:29:22.014 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-14T05:29:22.014 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-14T05:29:22.014 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_asio_thread_count": "2", 2024-09-14T05:29:22.014 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_async_recovery_min_cost": "100", 2024-09-14T05:29:22.014 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-14T05:29:22.015 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-14T05:29:22.015 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_backfill_scan_max": "512", 2024-09-14T05:29:22.015 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_backfill_scan_min": "64", 2024-09-14T05:29:22.015 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_backoff_on_degraded": "false", 2024-09-14T05:29:22.015 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_backoff_on_peering": "false", 2024-09-14T05:29:22.015 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_backoff_on_unfound": "true", 2024-09-14T05:29:22.015 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_beacon_report_interval": "300", 2024-09-14T05:29:22.015 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bench_duration": "30", 2024-09-14T05:29:22.015 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-14T05:29:22.015 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bench_max_block_size": "67108864", 2024-09-14T05:29:22.015 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-14T05:29:22.015 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_blkin_trace_all": "false", 2024-09-14T05:29:22.015 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-14T05:29:22.015 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-14T05:29:22.015 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-14T05:29:22.016 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-14T05:29:22.016 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_check_for_log_corruption": "false", 2024-09-14T05:29:22.016 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-14T05:29:22.016 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-14T05:29:22.016 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-14T05:29:22.016 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-14T05:29:22.016 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_class_update_on_start": "true", 2024-09-14T05:29:22.016 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_client_message_cap": "256", 2024-09-14T05:29:22.016 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_client_message_size_cap": "524288000", 2024-09-14T05:29:22.016 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_client_op_priority": "63", 2024-09-14T05:29:22.016 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_client_watch_timeout": "30", 2024-09-14T05:29:22.016 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_command_max_records": "256", 2024-09-14T05:29:22.016 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_compact_on_start": "false", 2024-09-14T05:29:22.016 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-14T05:29:22.017 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-14T05:29:22.017 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-14T05:29:22.017 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_crush_update_on_start": "true", 2024-09-14T05:29:22.017 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_crush_update_weight_set": "true", 2024-09-14T05:29:22.017 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-14T05:29:22.017 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-14T05:29:22.017 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-14T05:29:22.017 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-14T05:29:22.017 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-14T05:29:22.017 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_feed_pullee": "-1", 2024-09-14T05:29:22.017 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-14T05:29:22.017 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-14T05:29:22.017 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-14T05:29:22.017 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_misdirected_ops": "false", 2024-09-14T05:29:22.018 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_no_acting_change": "false", 2024-09-14T05:29:22.018 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_no_purge_strays": "false", 2024-09-14T05:29:22.018 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_op_order": "false", 2024-09-14T05:29:22.018 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-14T05:29:22.018 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-14T05:29:22.018 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-14T05:29:22.018 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-14T05:29:22.018 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_shutdown": "false", 2024-09-14T05:29:22.018 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-14T05:29:22.018 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-14T05:29:22.018 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_trim_objects": "false", 2024-09-14T05:29:22.018 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-14T05:29:22.018 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-14T05:29:22.018 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_verify_snaps": "false", 2024-09-14T05:29:22.019 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-14T05:29:22.019 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-14T05:29:22.019 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_deep_scrub_keys": "1024", 2024-09-14T05:29:22.019 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-14T05:29:22.019 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-14T05:29:22.019 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-14T05:29:22.019 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_deep_scrub_stride": "524288", 2024-09-14T05:29:22.019 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-14T05:29:22.019 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-14T05:29:22.019 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_default_notify_timeout": "30", 2024-09-14T05:29:22.019 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_delete_sleep": "0.000000", 2024-09-14T05:29:22.019 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-14T05:29:22.019 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-14T05:29:22.020 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-14T05:29:22.020 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_discard_disconnected_ops": "true", 2024-09-14T05:29:22.020 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_ec_partial_reads": "true", 2024-09-14T05:29:22.020 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_enable_op_tracker": "true", 2024-09-14T05:29:22.020 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-14T05:29:22.020 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-14T05:29:22.020 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-14T05:29:22.020 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-14T05:29:22.020 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_fast_info": "true", 2024-09-14T05:29:22.020 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_fast_shutdown": "true", 2024-09-14T05:29:22.020 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-14T05:29:22.020 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-14T05:29:22.020 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-14T05:29:22.021 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-14T05:29:22.021 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-14T05:29:22.021 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_function_tracing": "false", 2024-09-14T05:29:22.021 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_heartbeat_grace": "20", 2024-09-14T05:29:22.021 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_heartbeat_interval": "6", 2024-09-14T05:29:22.021 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-14T05:29:22.021 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_heartbeat_min_peers": "10", 2024-09-14T05:29:22.021 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_heartbeat_min_size": "2000", 2024-09-14T05:29:22.021 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_heartbeat_stale": "600", 2024-09-14T05:29:22.021 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-14T05:29:22.021 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_hit_set_max_size": "100000", 2024-09-14T05:29:22.021 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_hit_set_min_size": "1000", 2024-09-14T05:29:22.021 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-14T05:29:22.021 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-14T05:29:22.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-14T05:29:22.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-14T05:29:22.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-14T05:29:22.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-14T05:29:22.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_journal_size": "5120", 2024-09-14T05:29:22.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_kill_backfill_at": "0", 2024-09-14T05:29:22.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-14T05:29:22.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_map_cache_size": "50", 2024-09-14T05:29:22.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_map_dedup": "true", 2024-09-14T05:29:22.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_map_message_max": "40", 2024-09-14T05:29:22.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-14T05:29:22.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_map_share_max_epochs": "40", 2024-09-14T05:29:22.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_attr_name_len": "100", 2024-09-14T05:29:22.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_attr_size": "0", 2024-09-14T05:29:22.023 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_backfills": "1", 2024-09-14T05:29:22.023 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_markdown_count": "5", 2024-09-14T05:29:22.023 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_markdown_period": "600", 2024-09-14T05:29:22.023 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_object_name_len": "2048", 2024-09-14T05:29:22.023 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_object_namespace_len": "256", 2024-09-14T05:29:22.023 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_object_size": "134217728", 2024-09-14T05:29:22.023 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-14T05:29:22.023 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-14T05:29:22.023 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_pg_blocked_by": "16", 2024-09-14T05:29:22.023 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_pg_log_entries": "10000", 2024-09-14T05:29:22.023 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-14T05:29:22.023 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_pgls": "1024", 2024-09-14T05:29:22.023 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_push_cost": "8388608", 2024-09-14T05:29:22.023 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_push_objects": "10", 2024-09-14T05:29:22.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_scrubs": "3", 2024-09-14T05:29:22.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-14T05:29:22.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_trimming_pgs": "2", 2024-09-14T05:29:22.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_write_op_reply_len": "64", 2024-09-14T05:29:22.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_max_write_size": "90", 2024-09-14T05:29:22.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-14T05:29:22.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-14T05:29:22.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-14T05:29:22.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-14T05:29:22.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-14T05:29:22.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-14T05:29:22.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-14T05:29:22.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-14T05:29:22.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_profile": "balanced", 2024-09-14T05:29:22.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-14T05:29:22.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-14T05:29:22.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-14T05:29:22.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-14T05:29:22.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-14T05:29:22.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-14T05:29:22.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-14T05:29:22.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-14T05:29:22.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-14T05:29:22.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-14T05:29:22.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-14T05:29:22.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_memory_base": "805306368", 2024-09-14T05:29:22.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_memory_cache_min": "134217728", 2024-09-14T05:29:22.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-14T05:29:22.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-14T05:29:22.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_memory_target": "4294967296", 2024-09-14T05:29:22.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_memory_target_autotune": "false", 2024-09-14T05:29:22.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-14T05:29:22.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_min_pg_log_entries": "250", 2024-09-14T05:29:22.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_min_recovery_priority": "0", 2024-09-14T05:29:22.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-14T05:29:22.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-14T05:29:22.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mon_report_interval": "5", 2024-09-14T05:29:22.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-14T05:29:22.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-14T05:29:22.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_num_cache_shards": "32", 2024-09-14T05:29:22.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_num_op_tracker_shard": "32", 2024-09-14T05:29:22.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_numa_auto_affinity": "true", 2024-09-14T05:29:22.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_numa_node": "-1", 2024-09-14T05:29:22.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_numa_prefer_iface": "true", 2024-09-14T05:29:22.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-14T05:29:22.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_objecter_finishers": "1", 2024-09-14T05:29:22.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_objectstore": "bluestore", 2024-09-14T05:29:22.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_objectstore_fuse": "false", 2024-09-14T05:29:22.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-14T05:29:22.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_objectstore_tracing": "false", 2024-09-14T05:29:22.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_complaint_time": "30.000000", 2024-09-14T05:29:22.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_history_duration": "600", 2024-09-14T05:29:22.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_history_size": "20", 2024-09-14T05:29:22.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_history_slow_op_size": "20", 2024-09-14T05:29:22.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-14T05:29:22.028 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_log_threshold": "5", 2024-09-14T05:29:22.028 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_num_shards": "0", 2024-09-14T05:29:22.028 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_num_shards_hdd": "5", 2024-09-14T05:29:22.028 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_num_shards_ssd": "8", 2024-09-14T05:29:22.028 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-14T05:29:22.028 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-14T05:29:22.028 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-14T05:29:22.028 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-14T05:29:22.028 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_pq_min_cost": "65536", 2024-09-14T05:29:22.028 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-14T05:29:22.028 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_queue_cut_off": "high", 2024-09-14T05:29:22.028 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-14T05:29:22.028 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_op_thread_timeout": "15", 2024-09-14T05:29:22.028 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_open_classes_on_start": "true", 2024-09-14T05:29:22.028 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_os_flags": "0", 2024-09-14T05:29:22.029 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_peering_op_priority": "255", 2024-09-14T05:29:22.029 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_delete_cost": "1048576", 2024-09-14T05:29:22.029 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_delete_priority": "5", 2024-09-14T05:29:22.029 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-14T05:29:22.029 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-14T05:29:22.029 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-14T05:29:22.029 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_log_trim_max": "10000", 2024-09-14T05:29:22.029 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_log_trim_min": "100", 2024-09-14T05:29:22.029 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-14T05:29:22.029 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-14T05:29:22.029 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-14T05:29:22.029 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-14T05:29:22.029 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-14T05:29:22.030 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-14T05:29:22.030 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-14T05:29:22.030 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-14T05:29:22.030 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-14T05:29:22.030 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-14T05:29:22.030 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_crimson": "false", 2024-09-14T05:29:22.030 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-14T05:29:22.030 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-14T05:29:22.030 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-14T05:29:22.030 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-14T05:29:22.030 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-14T05:29:22.030 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-14T05:29:22.030 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-14T05:29:22.031 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-14T05:29:22.031 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_flags": "0", 2024-09-14T05:29:22.031 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-14T05:29:22.031 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_min_size": "0", 2024-09-14T05:29:22.031 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-14T05:29:22.031 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_pg_num": "32", 2024-09-14T05:29:22.031 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_pgp_num": "0", 2024-09-14T05:29:22.031 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-14T05:29:22.031 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_read_ratio": "70", 2024-09-14T05:29:22.031 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_size": "2", 2024-09-14T05:29:22.031 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_default_type": "replicated", 2024-09-14T05:29:22.031 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-14T05:29:22.031 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-14T05:29:22.031 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_push_per_object_cost": "1000", 2024-09-14T05:29:22.032 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-14T05:29:22.032 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recover_clone_overlap": "true", 2024-09-14T05:29:22.032 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-14T05:29:22.032 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_cost": "20971520", 2024-09-14T05:29:22.032 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-14T05:29:22.032 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_max_active": "0", 2024-09-14T05:29:22.032 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-14T05:29:22.032 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-14T05:29:22.032 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-14T05:29:22.032 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-14T05:29:22.032 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_max_single_start": "1", 2024-09-14T05:29:22.032 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_op_priority": "3", 2024-09-14T05:29:22.032 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-14T05:29:22.033 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_priority": "5", 2024-09-14T05:29:22.033 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-14T05:29:22.033 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_sleep": "0.000000", 2024-09-14T05:29:22.033 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-14T05:29:22.033 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-14T05:29:22.033 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-14T05:29:22.033 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_repair_during_recovery": "false", 2024-09-14T05:29:22.033 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_requested_scrub_priority": "120", 2024-09-14T05:29:22.033 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-14T05:29:22.033 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-14T05:29:22.033 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_auto_repair": "false", 2024-09-14T05:29:22.033 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-14T05:29:22.033 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-14T05:29:22.034 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_begin_hour": "0", 2024-09-14T05:29:22.034 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_begin_week_day": "0", 2024-09-14T05:29:22.034 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_chunk_max": "25", 2024-09-14T05:29:22.034 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_chunk_min": "5", 2024-09-14T05:29:22.034 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_cost": "52428800", 2024-09-14T05:29:22.034 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-14T05:29:22.034 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_during_recovery": "false", 2024-09-14T05:29:22.034 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_end_hour": "0", 2024-09-14T05:29:22.034 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_end_week_day": "0", 2024-09-14T05:29:22.034 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_event_cost": "4096", 2024-09-14T05:29:22.034 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-14T05:29:22.034 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-14T05:29:22.034 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_invalid_stats": "true", 2024-09-14T05:29:22.034 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-14T05:29:22.035 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-14T05:29:22.035 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_max_preemptions": "5", 2024-09-14T05:29:22.035 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-14T05:29:22.035 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_priority": "5", 2024-09-14T05:29:22.035 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_scrub_sleep": "0.000000", 2024-09-14T05:29:22.035 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-14T05:29:22.035 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-14T05:29:22.035 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-14T05:29:22.035 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-14T05:29:22.035 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_skip_data_digest": "false", 2024-09-14T05:29:22.035 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_smart_report_timeout": "5", 2024-09-14T05:29:22.035 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_snap_trim_cost": "1048576", 2024-09-14T05:29:22.035 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_snap_trim_priority": "5", 2024-09-14T05:29:22.036 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-14T05:29:22.036 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-14T05:29:22.036 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-14T05:29:22.036 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-14T05:29:22.036 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-14T05:29:22.036 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-14T05:29:22.036 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-14T05:29:22.036 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_target_transaction_size": "30", 2024-09-14T05:29:22.036 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-14T05:29:22.036 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-14T05:29:22.036 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-14T05:29:22.036 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-14T05:29:22.036 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-14T05:29:22.037 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-14T05:29:22.037 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-14T05:29:22.037 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-14T05:29:22.037 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-14T05:29:22.037 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-14T05:29:22.037 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_tracing": "false", 2024-09-14T05:29:22.037 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_use_stale_snap": "false", 2024-09-14T05:29:22.037 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-14T05:29:22.037 INFO:teuthology.orchestra.run.smithi071.stdout: "osdc_blkin_trace_all": "false", 2024-09-14T05:29:22.037 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_kill_at": "0", 2024-09-14T05:29:22.037 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_max_join_drift": "10", 2024-09-14T05:29:22.037 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_min": "500", 2024-09-14T05:29:22.037 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_min_wait": "0.050000", 2024-09-14T05:29:22.037 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_propose_interval": "1.000000", 2024-09-14T05:29:22.038 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_service_trim_max": "500", 2024-09-14T05:29:22.038 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-14T05:29:22.038 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_service_trim_min": "250", 2024-09-14T05:29:22.038 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_stash_full_interval": "25", 2024-09-14T05:29:22.038 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_trim_max": "500", 2024-09-14T05:29:22.038 INFO:teuthology.orchestra.run.smithi071.stdout: "paxos_trim_min": "250", 2024-09-14T05:29:22.038 INFO:teuthology.orchestra.run.smithi071.stdout: "perf": "true", 2024-09-14T05:29:22.038 INFO:teuthology.orchestra.run.smithi071.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-14T05:29:22.038 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-14T05:29:22.038 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-14T05:29:22.038 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "-", 2024-09-14T05:29:22.038 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrv": "", 2024-09-14T05:29:22.038 INFO:teuthology.orchestra.run.smithi071.stdout: "public_bind_addr": "-", 2024-09-14T05:29:22.039 INFO:teuthology.orchestra.run.smithi071.stdout: "public_network": "", 2024-09-14T05:29:22.039 INFO:teuthology.orchestra.run.smithi071.stdout: "public_network_interface": "", 2024-09-14T05:29:22.039 INFO:teuthology.orchestra.run.smithi071.stdout: "qat_compressor_busy_polling": "false", 2024-09-14T05:29:22.039 INFO:teuthology.orchestra.run.smithi071.stdout: "qat_compressor_enabled": "false", 2024-09-14T05:29:22.039 INFO:teuthology.orchestra.run.smithi071.stdout: "qat_compressor_session_max_number": "256", 2024-09-14T05:29:22.039 INFO:teuthology.orchestra.run.smithi071.stdout: "rados_mon_op_timeout": "0", 2024-09-14T05:29:22.039 INFO:teuthology.orchestra.run.smithi071.stdout: "rados_osd_op_timeout": "0", 2024-09-14T05:29:22.039 INFO:teuthology.orchestra.run.smithi071.stdout: "rados_tracing": "false", 2024-09-14T05:29:22.039 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_atime_update_interval": "60", 2024-09-14T05:29:22.039 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-14T05:29:22.039 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_balance_parent_reads": "false", 2024-09-14T05:29:22.039 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_balance_snap_reads": "false", 2024-09-14T05:29:22.039 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_blkin_trace_all": "false", 2024-09-14T05:29:22.040 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-14T05:29:22.040 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-14T05:29:22.040 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_cache": "true", 2024-09-14T05:29:22.040 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-14T05:29:22.040 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-14T05:29:22.040 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-14T05:29:22.040 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-14T05:29:22.040 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_cache_policy": "writearound", 2024-09-14T05:29:22.040 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_cache_size": "33554432", 2024-09-14T05:29:22.040 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-14T05:29:22.040 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-14T05:29:22.040 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_clone_copy_on_read": "false", 2024-09-14T05:29:22.040 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_compression_hint": "none", 2024-09-14T05:29:22.041 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_concurrent_management_ops": "10", 2024-09-14T05:29:22.041 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-14T05:29:22.041 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_clone_format": "auto", 2024-09-14T05:29:22.041 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_data_pool": "", 2024-09-14T05:29:22.041 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_features": "61", 2024-09-14T05:29:22.041 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_format": "2", 2024-09-14T05:29:22.041 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_map_options": "", 2024-09-14T05:29:22.041 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_order": "22", 2024-09-14T05:29:22.041 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_pool": "rbd", 2024-09-14T05:29:22.041 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-14T05:29:22.041 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_stripe_count": "0", 2024-09-14T05:29:22.041 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_default_stripe_unit": "0", 2024-09-14T05:29:22.041 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-14T05:29:22.042 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-14T05:29:22.042 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-14T05:29:22.042 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_enable_alloc_hint": "true", 2024-09-14T05:29:22.042 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-14T05:29:22.042 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_io_scheduler": "simple", 2024-09-14T05:29:22.042 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-14T05:29:22.042 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-14T05:29:22.042 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-14T05:29:22.042 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-14T05:29:22.042 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-14T05:29:22.042 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-14T05:29:22.042 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-14T05:29:22.042 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-14T05:29:22.042 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-14T05:29:22.042 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_order": "24", 2024-09-14T05:29:22.042 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_pool": "", 2024-09-14T05:29:22.043 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_journal_splay_width": "4", 2024-09-14T05:29:22.043 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_localize_parent_reads": "false", 2024-09-14T05:29:22.043 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_localize_snap_reads": "false", 2024-09-14T05:29:22.043 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-14T05:29:22.043 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-14T05:29:22.043 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-14T05:29:22.043 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-14T05:29:22.043 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-14T05:29:22.043 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-14T05:29:22.043 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-14T05:29:22.043 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-14T05:29:22.043 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-14T05:29:22.043 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-14T05:29:22.044 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-14T05:29:22.044 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-14T05:29:22.044 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-14T05:29:22.044 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-14T05:29:22.044 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-14T05:29:22.044 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-14T05:29:22.044 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-14T05:29:22.044 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-14T05:29:22.044 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-14T05:29:22.044 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-14T05:29:22.044 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-14T05:29:22.044 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-14T05:29:22.044 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-14T05:29:22.044 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-14T05:29:22.045 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-14T05:29:22.045 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-14T05:29:22.045 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-14T05:29:22.045 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-14T05:29:22.045 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-14T05:29:22.045 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-14T05:29:22.045 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-14T05:29:22.045 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_mtime_update_interval": "60", 2024-09-14T05:29:22.045 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_non_blocking_aio": "true", 2024-09-14T05:29:22.045 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_op_thread_timeout": "60", 2024-09-14T05:29:22.045 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_op_threads": "1", 2024-09-14T05:29:22.045 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_parent_cache_enabled": "false", 2024-09-14T05:29:22.045 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-14T05:29:22.046 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-14T05:29:22.046 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-14T05:29:22.046 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_plugins": "", 2024-09-14T05:29:22.046 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_bps_burst": "0", 2024-09-14T05:29:22.046 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-14T05:29:22.046 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_bps_limit": "0", 2024-09-14T05:29:22.046 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_exclude_ops": "0", 2024-09-14T05:29:22.046 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_iops_burst": "0", 2024-09-14T05:29:22.046 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-14T05:29:22.046 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_iops_limit": "0", 2024-09-14T05:29:22.046 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-14T05:29:22.046 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-14T05:29:22.046 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-14T05:29:22.046 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-14T05:29:22.046 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-14T05:29:22.047 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-14T05:29:22.047 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-14T05:29:22.047 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-14T05:29:22.047 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-14T05:29:22.047 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-14T05:29:22.047 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-14T05:29:22.047 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-14T05:29:22.047 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-14T05:29:22.047 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-14T05:29:22.047 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_read_from_replica_policy": "default", 2024-09-14T05:29:22.047 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-14T05:29:22.047 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-14T05:29:22.047 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-14T05:29:22.047 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-14T05:29:22.048 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_skip_partial_discard": "true", 2024-09-14T05:29:22.048 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-14T05:29:22.048 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_tracing": "false", 2024-09-14T05:29:22.048 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_validate_names": "true", 2024-09-14T05:29:22.048 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_validate_pool": "true", 2024-09-14T05:29:22.048 INFO:teuthology.orchestra.run.smithi071.stdout: "restapi_base_url": "", 2024-09-14T05:29:22.048 INFO:teuthology.orchestra.run.smithi071.stdout: "restapi_log_level": "", 2024-09-14T05:29:22.048 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-14T05:29:22.048 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-14T05:29:22.048 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_acl_grants_max_num": "100", 2024-09-14T05:29:22.048 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_admin_entry": "admin", 2024-09-14T05:29:22.048 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-14T05:29:22.048 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_asio_assert_yielding": "false", 2024-09-14T05:29:22.048 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_backend_store": "rados", 2024-09-14T05:29:22.049 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_barbican_url": "", 2024-09-14T05:29:22.049 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_beast_enable_async": "true", 2024-09-14T05:29:22.049 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_bucket_counters_cache": "false", 2024-09-14T05:29:22.049 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-14T05:29:22.049 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-14T05:29:22.049 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-14T05:29:22.049 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-14T05:29:22.049 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-14T05:29:22.049 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-14T05:29:22.049 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-14T05:29:22.049 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-14T05:29:22.049 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_cache_enabled": "true", 2024-09-14T05:29:22.049 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_cache_expiry_interval": "900", 2024-09-14T05:29:22.050 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_cache_lru_size": "25000", 2024-09-14T05:29:22.050 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_config_store": "rados", 2024-09-14T05:29:22.050 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_content_length_compat": "false", 2024-09-14T05:29:22.050 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_copy_obj_progress": "true", 2024-09-14T05:29:22.050 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-14T05:29:22.050 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_cors_rules_max_num": "100", 2024-09-14T05:29:22.050 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_cross_domain_policy": "", 2024-09-14T05:29:22.050 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-14T05:29:22.050 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_kmip_addr": "", 2024-09-14T05:29:22.050 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-14T05:29:22.050 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-14T05:29:22.050 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-14T05:29:22.050 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-14T05:29:22.051 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_kmip_password": "", 2024-09-14T05:29:22.051 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-14T05:29:22.051 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_kmip_username": "", 2024-09-14T05:29:22.051 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_require_ssl": "true", 2024-09-14T05:29:22.051 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-14T05:29:22.051 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-14T05:29:22.051 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-14T05:29:22.051 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-14T05:29:22.051 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-14T05:29:22.051 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-14T05:29:22.051 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-14T05:29:22.051 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-14T05:29:22.051 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-14T05:29:22.051 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-14T05:29:22.052 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-14T05:29:22.052 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-14T05:29:22.052 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-14T05:29:22.052 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-14T05:29:22.052 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-14T05:29:22.052 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_addr": "", 2024-09-14T05:29:22.052 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_auth": "token", 2024-09-14T05:29:22.052 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_namespace": "", 2024-09-14T05:29:22.052 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_prefix": "", 2024-09-14T05:29:22.052 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-14T05:29:22.052 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-14T05:29:22.052 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-14T05:29:22.052 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-14T05:29:22.052 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_token_file": "", 2024-09-14T05:29:22.053 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-14T05:29:22.053 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_curl_buffersize": "524288", 2024-09-14T05:29:22.053 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-14T05:29:22.053 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_curl_low_speed_time": "300", 2024-09-14T05:29:22.053 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-14T05:29:22.053 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-14T05:29:22.053 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-14T05:29:22.053 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-14T05:29:22.053 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-14T05:29:22.053 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-14T05:29:22.053 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-14T05:29:22.053 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-14T05:29:22.053 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-14T05:29:22.053 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-14T05:29:22.054 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-14T05:29:22.054 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-14T05:29:22.054 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-14T05:29:22.054 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-14T05:29:22.054 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-14T05:29:22.054 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-14T05:29:22.054 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-14T05:29:22.054 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-14T05:29:22.054 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-14T05:29:22.054 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_data_log_changes_size": "1000", 2024-09-14T05:29:22.054 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_data_log_num_shards": "128", 2024-09-14T05:29:22.054 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-14T05:29:22.054 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_data_log_window": "30", 2024-09-14T05:29:22.055 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-14T05:29:22.055 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-14T05:29:22.055 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-14T05:29:22.055 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-14T05:29:22.055 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-14T05:29:22.055 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-14T05:29:22.055 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-14T05:29:22.055 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-14T05:29:22.055 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-14T05:29:22.055 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-14T05:29:22.055 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-14T05:29:22.055 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-14T05:29:22.055 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-14T05:29:22.055 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_disable_s3select": "false", 2024-09-14T05:29:22.056 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-14T05:29:22.056 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-14T05:29:22.056 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-14T05:29:22.056 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-14T05:29:22.056 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-14T05:29:22.056 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-14T05:29:22.056 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-14T05:29:22.056 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-14T05:29:22.056 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-14T05:29:22.056 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-14T05:29:22.056 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-14T05:29:22.056 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-14T05:29:22.056 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dns_name": "", 2024-09-14T05:29:22.057 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dns_s3website_name": "", 2024-09-14T05:29:22.057 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dynamic_resharding": "true", 2024-09-14T05:29:22.057 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-14T05:29:22.057 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-14T05:29:22.057 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-14T05:29:22.057 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_enable_gc_threads": "true", 2024-09-14T05:29:22.057 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_enable_lc_threads": "true", 2024-09-14T05:29:22.057 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_enable_mdsearch": "true", 2024-09-14T05:29:22.057 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_enable_ops_log": "false", 2024-09-14T05:29:22.057 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_enable_quota_threads": "true", 2024-09-14T05:29:22.057 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_enable_static_website": "false", 2024-09-14T05:29:22.057 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_enable_usage_log": "false", 2024-09-14T05:29:22.057 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_enforce_swift_acls": "true", 2024-09-14T05:29:22.057 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_exit_timeout_secs": "120", 2024-09-14T05:29:22.058 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_expose_bucket": "false", 2024-09-14T05:29:22.058 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_extended_http_attrs": "", 2024-09-14T05:29:22.058 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_filter": "none", 2024-09-14T05:29:22.058 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-14T05:29:22.058 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_frontends": "beast port=7480", 2024-09-14T05:29:22.058 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-14T05:29:22.058 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_gc_max_deferred": "50", 2024-09-14T05:29:22.058 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-14T05:29:22.058 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_gc_max_objs": "32", 2024-09-14T05:29:22.058 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-14T05:29:22.058 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-14T05:29:22.058 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-14T05:29:22.058 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-14T05:29:22.058 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_gc_processor_period": "3600", 2024-09-14T05:29:22.059 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-14T05:29:22.059 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-14T05:29:22.059 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-14T05:29:22.059 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-14T05:29:22.059 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_init_timeout": "300", 2024-09-14T05:29:22.059 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-14T05:29:22.059 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-14T05:29:22.059 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_kafka_connection_idle": "300", 2024-09-14T05:29:22.059 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-14T05:29:22.059 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-14T05:29:22.059 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-14T05:29:22.059 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-14T05:29:22.059 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-14T05:29:22.060 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_admin_domain": "", 2024-09-14T05:29:22.060 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_admin_password": "", 2024-09-14T05:29:22.060 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_admin_password_path": "", 2024-09-14T05:29:22.060 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_admin_project": "", 2024-09-14T05:29:22.060 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_admin_tenant": "", 2024-09-14T05:29:22.060 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_admin_token": "", 2024-09-14T05:29:22.060 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_admin_token_path": "", 2024-09-14T05:29:22.060 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_admin_user": "", 2024-09-14T05:29:22.060 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_barbican_domain": "", 2024-09-14T05:29:22.060 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_barbican_password": "", 2024-09-14T05:29:22.060 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_barbican_project": "", 2024-09-14T05:29:22.060 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-14T05:29:22.060 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_barbican_user": "", 2024-09-14T05:29:22.061 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-14T05:29:22.061 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-14T05:29:22.061 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-14T05:29:22.061 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-14T05:29:22.061 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-14T05:29:22.061 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_url": "", 2024-09-14T05:29:22.061 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-14T05:29:22.061 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lc_debug_interval": "-1", 2024-09-14T05:29:22.061 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lc_lock_max_time": "90", 2024-09-14T05:29:22.061 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lc_max_objs": "32", 2024-09-14T05:29:22.061 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lc_max_rules": "1000", 2024-09-14T05:29:22.061 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lc_max_worker": "3", 2024-09-14T05:29:22.061 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-14T05:29:22.062 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lc_thread_delay": "0", 2024-09-14T05:29:22.062 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-14T05:29:22.062 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ldap_dnattr": "uid", 2024-09-14T05:29:22.062 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-14T05:29:22.062 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ldap_searchfilter": "", 2024-09-14T05:29:22.062 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-14T05:29:22.062 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ldap_uri": "", 2024-09-14T05:29:22.062 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-14T05:29:22.062 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-14T05:29:22.062 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-14T05:29:22.062 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-14T05:29:22.062 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_log_http_headers": "", 2024-09-14T05:29:22.062 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-14T05:29:22.062 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-14T05:29:22.063 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_log_object_name_utc": "false", 2024-09-14T05:29:22.063 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-14T05:29:22.063 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-14T05:29:22.063 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_attr_name_len": "0", 2024-09-14T05:29:22.063 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_attr_size": "0", 2024-09-14T05:29:22.063 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-14T05:29:22.063 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_chunk_size": "4194304", 2024-09-14T05:29:22.063 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-14T05:29:22.063 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-14T05:29:22.063 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-14T05:29:22.063 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_listing_results": "1000", 2024-09-14T05:29:22.063 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_notify_retries": "10", 2024-09-14T05:29:22.063 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-14T05:29:22.063 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_put_param_size": "1048576", 2024-09-14T05:29:22.063 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_put_size": "5368709120", 2024-09-14T05:29:22.064 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_max_slo_entries": "1000", 2024-09-14T05:29:22.064 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_md_log_max_shards": "64", 2024-09-14T05:29:22.064 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-14T05:29:22.064 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-14T05:29:22.064 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-14T05:29:22.064 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-14T05:29:22.064 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_mp_lock_max_time": "600", 2024-09-14T05:29:22.064 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-14T05:29:22.064 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-14T05:29:22.064 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-14T05:29:22.064 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-14T05:29:22.064 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-14T05:29:22.064 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-14T05:29:22.065 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-14T05:29:22.065 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-14T05:29:22.065 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_max_gc": "300", 2024-09-14T05:29:22.065 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-14T05:29:22.065 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-14T05:29:22.065 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-14T05:29:22.065 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-14T05:29:22.065 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-14T05:29:22.065 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-14T05:29:22.065 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-14T05:29:22.065 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_num_async_rados_threads": "32", 2024-09-14T05:29:22.066 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_num_control_oids": "8", 2024-09-14T05:29:22.066 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_numa_node": "-1", 2024-09-14T05:29:22.066 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-14T05:29:22.066 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-14T05:29:22.066 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_objexp_chunk_size": "100", 2024-09-14T05:29:22.066 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_objexp_gc_interval": "600", 2024-09-14T05:29:22.066 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-14T05:29:22.066 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-14T05:29:22.066 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-14T05:29:22.066 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_op_thread_timeout": "600", 2024-09-14T05:29:22.066 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_op_tracing": "false", 2024-09-14T05:29:22.066 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_opa_token": "", 2024-09-14T05:29:22.066 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_opa_url": "", 2024-09-14T05:29:22.066 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_opa_verify_ssl": "true", 2024-09-14T05:29:22.066 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-14T05:29:22.067 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-14T05:29:22.067 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ops_log_rados": "false", 2024-09-14T05:29:22.067 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_ops_log_socket_path": "", 2024-09-14T05:29:22.067 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-14T05:29:22.067 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-14T05:29:22.067 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-14T05:29:22.067 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_period_push_interval": "2.000000", 2024-09-14T05:29:22.067 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-14T05:29:22.067 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-14T05:29:22.067 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-14T05:29:22.067 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-14T05:29:22.067 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_posix_cache_lanes": "3", 2024-09-14T05:29:22.067 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-14T05:29:22.067 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-14T05:29:22.068 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_posix_cache_partitions": "3", 2024-09-14T05:29:22.068 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-14T05:29:22.068 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_print_continue": "true", 2024-09-14T05:29:22.068 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-14T05:29:22.068 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-14T05:29:22.068 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-14T05:29:22.068 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-14T05:29:22.068 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-14T05:29:22.068 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_rados_tracing": "false", 2024-09-14T05:29:22.068 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_realm": "", 2024-09-14T05:29:22.068 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_realm_id": "", 2024-09-14T05:29:22.068 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-14T05:29:22.069 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_region": "", 2024-09-14T05:29:22.069 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-14T05:29:22.069 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-14T05:29:22.069 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-14T05:29:22.069 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-14T05:29:22.069 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_request_uri": "", 2024-09-14T05:29:22.069 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_reshard_batch_size": "64", 2024-09-14T05:29:22.069 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-14T05:29:22.069 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-14T05:29:22.069 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_reshard_max_aio": "128", 2024-09-14T05:29:22.069 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_reshard_num_logs": "16", 2024-09-14T05:29:22.069 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-14T05:29:22.069 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-14T05:29:22.070 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_reshard_thread_interval": "600", 2024-09-14T05:29:22.070 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-14T05:29:22.070 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_resolve_cname": "false", 2024-09-14T05:29:22.070 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-14T05:29:22.070 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_run_sync_thread": "true", 2024-09-14T05:29:22.070 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-14T05:29:22.070 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-14T05:29:22.070 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-14T05:29:22.070 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-14T05:29:22.070 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-14T05:29:22.070 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-14T05:29:22.070 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-14T05:29:22.070 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-14T05:29:22.070 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-14T05:29:22.070 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_scheduler_type": "throttler", 2024-09-14T05:29:22.071 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_script_uri": "", 2024-09-14T05:29:22.071 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_service_provider_name": "", 2024-09-14T05:29:22.071 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-14T05:29:22.071 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sts_client_id": "", 2024-09-14T05:29:22.071 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sts_client_secret": "", 2024-09-14T05:29:22.071 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sts_entry": "sts", 2024-09-14T05:29:22.071 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sts_key": "sts", 2024-09-14T05:29:22.071 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-14T05:29:22.071 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sts_min_session_duration": "900", 2024-09-14T05:29:22.071 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sts_token_introspection_url": "", 2024-09-14T05:29:22.071 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_account_in_url": "false", 2024-09-14T05:29:22.071 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_auth_entry": "auth", 2024-09-14T05:29:22.071 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_auth_url": "", 2024-09-14T05:29:22.071 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_custom_header": "", 2024-09-14T05:29:22.071 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-14T05:29:22.071 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_need_stats": "true", 2024-09-14T05:29:22.072 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_tenant_name": "", 2024-09-14T05:29:22.072 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_token_expiration": "86400", 2024-09-14T05:29:22.072 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_url": "", 2024-09-14T05:29:22.072 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_url_prefix": "swift", 2024-09-14T05:29:22.072 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-14T05:29:22.072 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-14T05:29:22.072 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-14T05:29:22.072 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_lease_period": "120", 2024-09-14T05:29:22.072 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-14T05:29:22.072 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-14T05:29:22.072 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-14T05:29:22.072 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-14T05:29:22.072 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-14T05:29:22.073 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-14T05:29:22.073 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-14T05:29:22.073 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-14T05:29:22.073 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-14T05:29:22.073 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_thread_pool_size": "512", 2024-09-14T05:29:22.073 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-14T05:29:22.073 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-14T05:29:22.073 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-14T05:29:22.073 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-14T05:29:22.073 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_torrent_comment": "", 2024-09-14T05:29:22.073 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_torrent_createby": "", 2024-09-14T05:29:22.073 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_torrent_encoding": "", 2024-09-14T05:29:22.073 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_torrent_flag": "false", 2024-09-14T05:29:22.074 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-14T05:29:22.074 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_torrent_origin": "", 2024-09-14T05:29:22.074 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-14T05:29:22.074 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_torrent_tracker": "", 2024-09-14T05:29:22.074 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_trust_forwarded_https": "false", 2024-09-14T05:29:22.074 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-14T05:29:22.074 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-14T05:29:22.074 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_usage_max_shards": "32", 2024-09-14T05:29:22.074 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_usage_max_user_shards": "1", 2024-09-14T05:29:22.074 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_use_opa_authz": "false", 2024-09-14T05:29:22.074 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_counters_cache": "false", 2024-09-14T05:29:22.074 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-14T05:29:22.074 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-14T05:29:22.075 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-14T05:29:22.075 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_max_buckets": "1000", 2024-09-14T05:29:22.075 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_policies_max_num": "100", 2024-09-14T05:29:22.075 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-14T05:29:22.075 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-14T05:29:22.075 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-14T05:29:22.075 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-14T05:29:22.075 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_user_unique_email": "true", 2024-09-14T05:29:22.075 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_verify_ssl": "true", 2024-09-14T05:29:22.075 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-14T05:29:22.075 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_zone": "", 2024-09-14T05:29:22.075 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_zone_id": "", 2024-09-14T05:29:22.075 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-14T05:29:22.076 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_zonegroup": "", 2024-09-14T05:29:22.076 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_zonegroup_id": "", 2024-09-14T05:29:22.076 INFO:teuthology.orchestra.run.smithi071.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-14T05:29:22.076 INFO:teuthology.orchestra.run.smithi071.stdout: "rgwlc_auto_session_clear": "true", 2024-09-14T05:29:22.076 INFO:teuthology.orchestra.run.smithi071.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-14T05:29:22.076 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_block_size": "4096", 2024-09-14T05:29:22.076 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-14T05:29:22.076 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-14T05:29:22.076 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-14T05:29:22.076 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-14T05:29:22.076 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-14T05:29:22.076 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_cache_size": "536870912", 2024-09-14T05:29:22.076 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-14T05:29:22.076 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-14T05:29:22.077 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-14T05:29:22.077 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-14T05:29:22.077 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-14T05:29:22.077 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-14T05:29:22.077 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-14T05:29:22.077 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-14T05:29:22.077 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_index_type": "binary_search", 2024-09-14T05:29:22.077 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-14T05:29:22.077 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-14T05:29:22.077 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_partition_filters": "false", 2024-09-14T05:29:22.077 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_perf": "false", 2024-09-14T05:29:22.077 INFO:teuthology.orchestra.run.smithi071.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-14T05:29:22.077 INFO:teuthology.orchestra.run.smithi071.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-14T05:29:22.077 INFO:teuthology.orchestra.run.smithi071.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-14T05:29:22.078 INFO:teuthology.orchestra.run.smithi071.stdout: "run_dir": "/var/run/ceph", 2024-09-14T05:29:22.078 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_block_create": "true", 2024-09-14T05:29:22.078 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_cache_lru_size": "67108864", 2024-09-14T05:29:22.078 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-14T05:29:22.078 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-14T05:29:22.078 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_default_max_object_size": "16777216", 2024-09-14T05:29:22.078 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-14T05:29:22.078 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_device_size": "53687091200", 2024-09-14T05:29:22.078 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-14T05:29:22.078 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_full_integrity_check": "false", 2024-09-14T05:29:22.078 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_journal_batch_capacity": "16", 2024-09-14T05:29:22.078 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-14T05:29:22.078 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-14T05:29:22.078 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-14T05:29:22.078 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_main_device_type": "SSD", 2024-09-14T05:29:22.079 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-14T05:29:22.079 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-14T05:29:22.079 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-14T05:29:22.079 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-14T05:29:22.079 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-14T05:29:22.079 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-14T05:29:22.079 INFO:teuthology.orchestra.run.smithi071.stdout: "seastore_segment_size": "67108864", 2024-09-14T05:29:22.079 INFO:teuthology.orchestra.run.smithi071.stdout: "set_keepcaps": "false", 2024-09-14T05:29:22.079 INFO:teuthology.orchestra.run.smithi071.stdout: "setgroup": "", 2024-09-14T05:29:22.079 INFO:teuthology.orchestra.run.smithi071.stdout: "setuser": "", 2024-09-14T05:29:22.079 INFO:teuthology.orchestra.run.smithi071.stdout: "setuser_match_path": "", 2024-09-14T05:29:22.079 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-14T05:29:22.079 INFO:teuthology.orchestra.run.smithi071.stdout: "thp": "false", 2024-09-14T05:29:22.079 INFO:teuthology.orchestra.run.smithi071.stdout: "threadpool_default_timeout": "60", 2024-09-14T05:29:22.080 INFO:teuthology.orchestra.run.smithi071.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-14T05:29:22.080 INFO:teuthology.orchestra.run.smithi071.stdout: "throttler_perf_counter": "true", 2024-09-14T05:29:22.080 INFO:teuthology.orchestra.run.smithi071.stdout: "tmp_dir": "/tmp", 2024-09-14T05:29:22.080 INFO:teuthology.orchestra.run.smithi071.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-14T05:29:22.080 INFO:teuthology.orchestra.run.smithi071.stdout: "uadk_compressor_enabled": "false", 2024-09-14T05:29:22.080 INFO:teuthology.orchestra.run.smithi071.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-14T05:29:22.080 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:22.082 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-14T05:29:22.194 INFO:teuthology.orchestra.run.smithi071.stdout:mon.a: {} 2024-09-14T05:29:22.194 INFO:teuthology.orchestra.run.smithi071.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T05:29:22.203 INFO:teuthology.orchestra.run.smithi071.stdout:mon.b: {} 2024-09-14T05:29:22.204 INFO:teuthology.orchestra.run.smithi071.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T05:29:22.212 INFO:teuthology.orchestra.run.smithi071.stdout:mon.c: {} 2024-09-14T05:29:22.212 INFO:teuthology.orchestra.run.smithi071.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T05:29:22.222 DEBUG:teuthology.orchestra.run.smithi071:> type ceph-objectstore-tool 2024-09-14T05:29:22.236 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T05:29:22.237 DEBUG:teuthology.orchestra.run.smithi150:> type ceph-objectstore-tool 2024-09-14T05:29:22.254 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T05:29:22.254 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-14T05:29:22.266 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-14T05:29:22.266 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-14T05:29:22.266 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T05:29:22.266 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T05:29:22.267 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T05:29:22.267 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 1.000000 2024-09-14T05:29:22.267 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-14T05:29:22.267 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-14T05:29:22.267 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-14T05:29:22.268 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:29:22.268 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-14T05:29:22.268 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:29:22.268 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-14T05:29:22.268 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T05:29:22.269 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-14T05:29:22.278 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:22.277+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:22.314 INFO:teuthology.orchestra.run.smithi071.stdout:mapped writes DISABLED 2024-09-14T05:29:22.314 INFO:teuthology.orchestra.run.smithi071.stdout:mapped reads DISABLED 2024-09-14T05:29:22.314 INFO:teuthology.orchestra.run.smithi071.stdout:Seed set to 8920 2024-09-14T05:29:22.369 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:22.368+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:22.435 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:22.436 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:22.436 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:29:22.436 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:22.448 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:29:22.458 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:29:22.458 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:29:22.492 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:29:22.492 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:29:22.492 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:22.491+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:22.526 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:29:22.526 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:29:22.562 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:29:22.562 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:29:22.564 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:22.564 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:22.572 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:22.571+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:22.573 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:29:22.580 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:29:22.580 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:29:22.598 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:29:22.598 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:29:22.633 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:29:22.634 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:29:22.667 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:29:22.668 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:29:22.673 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:22.672+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:22.685 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:22.686 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:22.694 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:29:22.779 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:22.778+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:22.839 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:22.839 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:22.847 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:29:22.875 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:22.874+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:22.976 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:22.976+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:22.990 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:22.990 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:22.999 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:29:23.008 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-14T05:29:23.009 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-14T05:29:23.077 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:23.077+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:23.108 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:23.108 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:23.116 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:29:23.179 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:23.177+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:23.260 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:23.260 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:23.269 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:29:23.279 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:23.279+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:23.371 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:23.371 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:23.381 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:29:23.384 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:23.380+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:23.482 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:23.481+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:23.525 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:23.525 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:23.533 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:29:23.582 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:23.582+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:23.636 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:23.637 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:23.645 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:29:23.683 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:23.683+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:23.747 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:23.747 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:23.756 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:29:23.784 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:23.784+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:23.859 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:23.859 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:23.868 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:29:23.885 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:23.885+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:23.981 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:23.982 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:23.987 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:23.986+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:23.990 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:29:24.087 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:24.087+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:24.135 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:24.135 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:24.143 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:29:24.189 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:24.188+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:24.288 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:24.288 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:24.289 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:24.288+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:24.296 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:29:24.390 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:24.390+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:24.440 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:24.440 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:24.449 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:29:24.493 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:24.492+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:24.552 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:24.678 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:24.680 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:29:24.680 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:24.679+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:24.683 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:29:24.781 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:24.780+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:24.809 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:29:24.809 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:29:24.825 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:24.825 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:24.825 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:29:24.825 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:29:24.833 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:29:24.841 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:29:24.841 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:29:24.857 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:29:24.857 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:29:24.873 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:29:24.873 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:29:24.881 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:24.881+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:24.892 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:29:24.892 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:29:24.908 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:29:24.908 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:29:24.925 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:29:24.925 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:29:24.938 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:24.938 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:24.938 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:29:24.938 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:24.947 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:29:24.982 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:24.982+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:25.025 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T05:29:25.051 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:25.051 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:25.052 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:29:25.052 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:29:25.052 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:25.061 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:29:25.106 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:25.105+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:25.165 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:25.165 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:29:25.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:29:25.166 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:29:25.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.166 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4235.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T05:29:25.166 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:14.832333+0000", 2024-09-14T05:29:25.166 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.333082342000001, 2024-09-14T05:29:25.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002663332, 2024-09-14T05:29:25.166 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:25.166 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:25.166 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:25.166 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:25.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:25.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:14.832333+0000", 2024-09-14T05:29:25.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:25.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:25.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:14.832333+0000", 2024-09-14T05:29:25.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:25.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:25.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:14.832334+0000", 2024-09-14T05:29:25.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7300000000000002e-07 2024-09-14T05:29:25.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:25.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:14.832339+0000", 2024-09-14T05:29:25.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.215e-06 2024-09-14T05:29:25.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:25.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:14.832353+0000", 2024-09-14T05:29:25.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4402e-05 2024-09-14T05:29:25.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:25.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:14.832366+0000", 2024-09-14T05:29:25.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2277e-05 2024-09-14T05:29:25.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:25.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:14.832415+0000", 2024-09-14T05:29:25.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9852999999999999e-05 2024-09-14T05:29:25.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:25.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:14.832435+0000", 2024-09-14T05:29:25.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9959e-05 2024-09-14T05:29:25.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:25.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:14.834945+0000", 2024-09-14T05:29:25.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025096369999999999 2024-09-14T05:29:25.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:25.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:14.834997+0000", 2024-09-14T05:29:25.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1715999999999997e-05 2024-09-14T05:29:25.170 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:25.170 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:25.170 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:25.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.171 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4235.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-14T05:29:25.171 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:15.849793+0000", 2024-09-14T05:29:25.171 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.3156228779999992, 2024-09-14T05:29:25.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00101538, 2024-09-14T05:29:25.171 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:25.171 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:25.171 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:25.171 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:25.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:25.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:15.849793+0000", 2024-09-14T05:29:25.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:25.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:25.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:15.849793+0000", 2024-09-14T05:29:25.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:25.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:25.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:15.849793+0000", 2024-09-14T05:29:25.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2100000000000001e-07 2024-09-14T05:29:25.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:25.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:15.849801+0000", 2024-09-14T05:29:25.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4020000000000002e-06 2024-09-14T05:29:25.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:25.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:15.849812+0000", 2024-09-14T05:29:25.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0664e-05 2024-09-14T05:29:25.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:25.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:15.849821+0000", 2024-09-14T05:29:25.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4530000000000002e-06 2024-09-14T05:29:25.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:25.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:15.849891+0000", 2024-09-14T05:29:25.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999000000000005e-05 2024-09-14T05:29:25.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:25.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:15.849917+0000", 2024-09-14T05:29:25.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6162000000000001e-05 2024-09-14T05:29:25.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:25.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:15.850760+0000", 2024-09-14T05:29:25.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00084308899999999999 2024-09-14T05:29:25.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:25.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:25.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:25.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:15.850808+0000", 2024-09-14T05:29:25.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8390000000000003e-05 2024-09-14T05:29:25.175 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:25.175 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:25.175 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:25.175 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:25.175 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:25.175 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:25.177 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:29:25.184 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:25.183+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:25.286 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:25.285+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:25.321 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:25.322 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:25.322 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:29:25.322 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:25.330 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:29:25.386 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:25.386+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:25.473 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:25.474 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:25.474 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:29:25.474 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:29:25.474 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:25.482 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:29:25.527 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:25.526+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:25.585 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:25.586 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:29:25.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:29:25.586 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [] 2024-09-14T05:29:25.586 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:25.588 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:25.588+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:25.594 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:29:25.690 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:25.689+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:25.698 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:25.698 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:25.698 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:29:25.698 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:25.707 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:29:25.792 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:25.791+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:25.848 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:25.848 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:25.848 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:25.848 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:25.848 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:25.856 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:29:25.902 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:25.901+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:25.960 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:25.960 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [] 2024-09-14T05:29:25.960 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:25.968 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:29:25.993 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:25.992+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:26.015 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-14T05:29:26.093 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:26.093+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:26.116 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:26.116 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:26.116 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:29:26.117 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:26.126 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:29:26.195 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:26.194+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:26.232 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:26.233 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:26.233 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:26.233 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:26.233 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:26.241 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:29:26.295 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:26.295+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:26.347 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:26.347 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:26.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:26.347 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:26.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.347 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.348 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:29:26.348 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.568078231000001, 2024-09-14T05:29:26.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.24562984099999999, 2024-09-14T05:29:26.348 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.348 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.348 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.348 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.348 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.348 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.348 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1 2024-09-14T05:29:26.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.348 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:29:26.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:29:26.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776565+0000", 2024-09-14T05:29:26.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4780000000000001e-06 2024-09-14T05:29:26.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776571+0000", 2024-09-14T05:29:26.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7669999999999998e-06 2024-09-14T05:29:26.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776580+0000", 2024-09-14T05:29:26.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9830000000000002e-06 2024-09-14T05:29:26.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.351 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.351 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776589+0000", 2024-09-14T05:29:26.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7579999999999998e-06 2024-09-14T05:29:26.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.351 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.351 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776662+0000", 2024-09-14T05:29:26.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3332999999999997e-05 2024-09-14T05:29:26.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.351 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:29:26.351 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776670+0000", 2024-09-14T05:29:26.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9850000000000006e-06 2024-09-14T05:29:26.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.020867+0000", 2024-09-14T05:29:26.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.244196265 2024-09-14T05:29:26.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021022+0000", 2024-09-14T05:29:26.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000155699 2024-09-14T05:29:26.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021169+0000", 2024-09-14T05:29:26.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014661700000000001 2024-09-14T05:29:26.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021891+0000", 2024-09-14T05:29:26.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072146700000000005 2024-09-14T05:29:26.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022124+0000", 2024-09-14T05:29:26.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023354199999999999 2024-09-14T05:29:26.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022164+0000", 2024-09-14T05:29:26.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9897000000000001e-05 2024-09-14T05:29:26.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022193+0000", 2024-09-14T05:29:26.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9050000000000001e-05 2024-09-14T05:29:26.354 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.354 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.354 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.355 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 131072~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.355 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.038382+0000", 2024-09-14T05:29:26.355 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.306259568000002, 2024-09-14T05:29:26.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0078079539999999998, 2024-09-14T05:29:26.355 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.355 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.355 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.355 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.355 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.356 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.356 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 22 2024-09-14T05:29:26.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.356 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038382+0000", 2024-09-14T05:29:26.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038382+0000", 2024-09-14T05:29:26.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038382+0000", 2024-09-14T05:29:26.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1399999999999997e-07 2024-09-14T05:29:26.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038439+0000", 2024-09-14T05:29:26.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6638999999999998e-05 2024-09-14T05:29:26.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038445+0000", 2024-09-14T05:29:26.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8989999999999998e-06 2024-09-14T05:29:26.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038449+0000", 2024-09-14T05:29:26.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.743e-06 2024-09-14T05:29:26.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.039586+0000", 2024-09-14T05:29:26.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011374289999999999 2024-09-14T05:29:26.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.039672+0000", 2024-09-14T05:29:26.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5609e-05 2024-09-14T05:29:26.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.039806+0000", 2024-09-14T05:29:26.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013425000000000001 2024-09-14T05:29:26.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.041894+0000", 2024-09-14T05:29:26.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020875949999999998 2024-09-14T05:29:26.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.046094+0000", 2024-09-14T05:29:26.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042002799999999998 2024-09-14T05:29:26.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.046149+0000", 2024-09-14T05:29:26.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4950000000000001e-05 2024-09-14T05:29:26.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.046190+0000", 2024-09-14T05:29:26.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1046e-05 2024-09-14T05:29:26.361 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.361 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.361 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.361 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:23 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.361 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.038457+0000", 2024-09-14T05:29:26.362 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.306184732999998, 2024-09-14T05:29:26.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010089383, 2024-09-14T05:29:26.362 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.362 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.362 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.362 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.362 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.362 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.362 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-14T05:29:26.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.363 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.363 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.363 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038457+0000", 2024-09-14T05:29:26.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.363 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.363 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038457+0000", 2024-09-14T05:29:26.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038457+0000", 2024-09-14T05:29:26.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.75e-07 2024-09-14T05:29:26.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038509+0000", 2024-09-14T05:29:26.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2068999999999999e-05 2024-09-14T05:29:26.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038516+0000", 2024-09-14T05:29:26.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0929999999999997e-06 2024-09-14T05:29:26.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038521+0000", 2024-09-14T05:29:26.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4649999999999996e-06 2024-09-14T05:29:26.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.040259+0000", 2024-09-14T05:29:26.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017385370000000001 2024-09-14T05:29:26.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.040328+0000", 2024-09-14T05:29:26.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8756000000000004e-05 2024-09-14T05:29:26.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.367 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.040419+0000", 2024-09-14T05:29:26.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0521999999999998e-05 2024-09-14T05:29:26.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.367 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.041898+0000", 2024-09-14T05:29:26.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001479015 2024-09-14T05:29:26.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.048441+0000", 2024-09-14T05:29:26.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0065434930000000001 2024-09-14T05:29:26.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.048505+0000", 2024-09-14T05:29:26.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3826999999999997e-05 2024-09-14T05:29:26.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.048546+0000", 2024-09-14T05:29:26.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1131000000000003e-05 2024-09-14T05:29:26.369 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.369 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.369 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.369 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.369 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.038528+0000", 2024-09-14T05:29:26.369 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.306113577000001, 2024-09-14T05:29:26.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010138525000000001, 2024-09-14T05:29:26.369 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.369 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.369 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-14T05:29:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038528+0000", 2024-09-14T05:29:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038528+0000", 2024-09-14T05:29:26.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038528+0000", 2024-09-14T05:29:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.89e-07 2024-09-14T05:29:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038580+0000", 2024-09-14T05:29:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1436e-05 2024-09-14T05:29:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038585+0000", 2024-09-14T05:29:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3700000000000003e-06 2024-09-14T05:29:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038589+0000", 2024-09-14T05:29:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2339999999999996e-06 2024-09-14T05:29:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.040780+0000", 2024-09-14T05:29:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021907049999999998 2024-09-14T05:29:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.372 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.040860+0000", 2024-09-14T05:29:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.975e-05 2024-09-14T05:29:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.040994+0000", 2024-09-14T05:29:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013444499999999999 2024-09-14T05:29:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.041902+0000", 2024-09-14T05:29:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00090772499999999996 2024-09-14T05:29:26.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.048602+0000", 2024-09-14T05:29:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0067001040000000001 2024-09-14T05:29:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.048637+0000", 2024-09-14T05:29:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5255e-05 2024-09-14T05:29:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.048666+0000", 2024-09-14T05:29:26.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9111999999999998e-05 2024-09-14T05:29:26.375 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.375 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.375 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.375 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.375 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.038596+0000", 2024-09-14T05:29:26.375 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.306045259000001, 2024-09-14T05:29:26.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010202661, 2024-09-14T05:29:26.375 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.375 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.375 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.375 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.375 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.375 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.376 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-14T05:29:26.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.376 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038596+0000", 2024-09-14T05:29:26.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038596+0000", 2024-09-14T05:29:26.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.377 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038597+0000", 2024-09-14T05:29:26.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.01e-07 2024-09-14T05:29:26.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.377 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.377 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038599+0000", 2024-09-14T05:29:26.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4930000000000001e-06 2024-09-14T05:29:26.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038603+0000", 2024-09-14T05:29:26.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6179999999999999e-06 2024-09-14T05:29:26.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038606+0000", 2024-09-14T05:29:26.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0410000000000002e-06 2024-09-14T05:29:26.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.041421+0000", 2024-09-14T05:29:26.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002815031 2024-09-14T05:29:26.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.041505+0000", 2024-09-14T05:29:26.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4209999999999995e-05 2024-09-14T05:29:26.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.041661+0000", 2024-09-14T05:29:26.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000156243 2024-09-14T05:29:26.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.042187+0000", 2024-09-14T05:29:26.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052539600000000002 2024-09-14T05:29:26.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.048714+0000", 2024-09-14T05:29:26.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0065274549999999997 2024-09-14T05:29:26.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.048765+0000", 2024-09-14T05:29:26.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1269e-05 2024-09-14T05:29:26.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.048799+0000", 2024-09-14T05:29:26.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3504000000000003e-05 2024-09-14T05:29:26.381 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.381 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.381 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.381 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.381 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237163+0000", 2024-09-14T05:29:26.381 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.107478293, 2024-09-14T05:29:26.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064289880000000001, 2024-09-14T05:29:26.381 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.382 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.382 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.382 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.382 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.382 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.382 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 42 2024-09-14T05:29:26.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.382 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237163+0000", 2024-09-14T05:29:26.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237163+0000", 2024-09-14T05:29:26.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237164+0000", 2024-09-14T05:29:26.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3599999999999999e-07 2024-09-14T05:29:26.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237166+0000", 2024-09-14T05:29:26.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2689999999999998e-06 2024-09-14T05:29:26.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237172+0000", 2024-09-14T05:29:26.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0990000000000004e-06 2024-09-14T05:29:26.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237176+0000", 2024-09-14T05:29:26.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8859999999999997e-06 2024-09-14T05:29:26.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.238652+0000", 2024-09-14T05:29:26.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001476125 2024-09-14T05:29:26.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.238733+0000", 2024-09-14T05:29:26.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1456000000000001e-05 2024-09-14T05:29:26.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.238857+0000", 2024-09-14T05:29:26.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012333799999999999 2024-09-14T05:29:26.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.240901+0000", 2024-09-14T05:29:26.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020440409999999999 2024-09-14T05:29:26.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243502+0000", 2024-09-14T05:29:26.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026007869999999998 2024-09-14T05:29:26.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243552+0000", 2024-09-14T05:29:26.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0302000000000001e-05 2024-09-14T05:29:26.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243592+0000", 2024-09-14T05:29:26.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0249e-05 2024-09-14T05:29:26.387 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.387 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.387 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.387 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.387 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237459+0000", 2024-09-14T05:29:26.387 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.107181936, 2024-09-14T05:29:26.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064058400000000003, 2024-09-14T05:29:26.387 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.387 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.387 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.387 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.387 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.388 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.388 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 44 2024-09-14T05:29:26.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.388 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237459+0000", 2024-09-14T05:29:26.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237459+0000", 2024-09-14T05:29:26.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237460+0000", 2024-09-14T05:29:26.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0100000000000005e-07 2024-09-14T05:29:26.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237500+0000", 2024-09-14T05:29:26.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9549999999999999e-05 2024-09-14T05:29:26.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237507+0000", 2024-09-14T05:29:26.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3610000000000003e-06 2024-09-14T05:29:26.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237512+0000", 2024-09-14T05:29:26.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4450000000000004e-06 2024-09-14T05:29:26.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.239830+0000", 2024-09-14T05:29:26.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023173249999999999 2024-09-14T05:29:26.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.239872+0000", 2024-09-14T05:29:26.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2101999999999998e-05 2024-09-14T05:29:26.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.239959+0000", 2024-09-14T05:29:26.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6997999999999996e-05 2024-09-14T05:29:26.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242053+0000", 2024-09-14T05:29:26.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020942220000000002 2024-09-14T05:29:26.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243781+0000", 2024-09-14T05:29:26.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001728388 2024-09-14T05:29:26.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243831+0000", 2024-09-14T05:29:26.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9552000000000003e-05 2024-09-14T05:29:26.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243865+0000", 2024-09-14T05:29:26.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4396000000000001e-05 2024-09-14T05:29:26.393 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.393 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.393 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.394 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.394 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237547+0000", 2024-09-14T05:29:26.394 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.107094855, 2024-09-14T05:29:26.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0076309530000000002, 2024-09-14T05:29:26.394 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.394 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.394 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.394 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.394 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.394 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.394 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 45 2024-09-14T05:29:26.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.394 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237547+0000", 2024-09-14T05:29:26.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237547+0000", 2024-09-14T05:29:26.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237547+0000", 2024-09-14T05:29:26.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0200000000000002e-07 2024-09-14T05:29:26.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237551+0000", 2024-09-14T05:29:26.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8229999999999997e-06 2024-09-14T05:29:26.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237563+0000", 2024-09-14T05:29:26.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2041999999999999e-05 2024-09-14T05:29:26.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237568+0000", 2024-09-14T05:29:26.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4709999999999998e-06 2024-09-14T05:29:26.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.397 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.240349+0000", 2024-09-14T05:29:26.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027806060000000001 2024-09-14T05:29:26.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.397 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.240423+0000", 2024-09-14T05:29:26.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4197999999999997e-05 2024-09-14T05:29:26.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.240546+0000", 2024-09-14T05:29:26.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012227399999999999 2024-09-14T05:29:26.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242951+0000", 2024-09-14T05:29:26.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002405866 2024-09-14T05:29:26.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.245091+0000", 2024-09-14T05:29:26.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021393829999999999 2024-09-14T05:29:26.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.245142+0000", 2024-09-14T05:29:26.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0782999999999998e-05 2024-09-14T05:29:26.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.245177+0000", 2024-09-14T05:29:26.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5905000000000003e-05 2024-09-14T05:29:26.399 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.399 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.400 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.400 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.400 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237579+0000", 2024-09-14T05:29:26.400 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.107062359, 2024-09-14T05:29:26.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010719608, 2024-09-14T05:29:26.400 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.400 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.400 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.401 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.401 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.401 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.401 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 46 2024-09-14T05:29:26.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.401 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237579+0000", 2024-09-14T05:29:26.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237579+0000", 2024-09-14T05:29:26.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.403 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237580+0000", 2024-09-14T05:29:26.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3499999999999996e-07 2024-09-14T05:29:26.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.403 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237583+0000", 2024-09-14T05:29:26.404 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.895e-06 2024-09-14T05:29:26.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.404 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.404 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237591+0000", 2024-09-14T05:29:26.404 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.17e-06 2024-09-14T05:29:26.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.404 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.404 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237598+0000", 2024-09-14T05:29:26.404 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0119999999999999e-06 2024-09-14T05:29:26.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.240949+0000", 2024-09-14T05:29:26.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033510929999999999 2024-09-14T05:29:26.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.241002+0000", 2024-09-14T05:29:26.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2837999999999998e-05 2024-09-14T05:29:26.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.241086+0000", 2024-09-14T05:29:26.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4530999999999994e-05 2024-09-14T05:29:26.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242957+0000", 2024-09-14T05:29:26.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001871239 2024-09-14T05:29:26.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248141+0000", 2024-09-14T05:29:26.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0051832040000000003 2024-09-14T05:29:26.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248262+0000", 2024-09-14T05:29:26.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121884 2024-09-14T05:29:26.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248299+0000", 2024-09-14T05:29:26.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6207000000000001e-05 2024-09-14T05:29:26.407 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.407 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.407 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.407 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.407 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237608+0000", 2024-09-14T05:29:26.408 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.107033806, 2024-09-14T05:29:26.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010804924, 2024-09-14T05:29:26.408 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.408 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.408 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.408 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.408 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.408 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.408 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 47 2024-09-14T05:29:26.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.408 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.409 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.409 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237608+0000", 2024-09-14T05:29:26.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.409 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.409 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237608+0000", 2024-09-14T05:29:26.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.409 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.409 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237608+0000", 2024-09-14T05:29:26.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6499999999999999e-07 2024-09-14T05:29:26.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237654+0000", 2024-09-14T05:29:26.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6326000000000003e-05 2024-09-14T05:29:26.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237665+0000", 2024-09-14T05:29:26.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0056e-05 2024-09-14T05:29:26.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237671+0000", 2024-09-14T05:29:26.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9819999999999996e-06 2024-09-14T05:29:26.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.241343+0000", 2024-09-14T05:29:26.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036724610000000001 2024-09-14T05:29:26.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.241419+0000", 2024-09-14T05:29:26.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6089999999999998e-05 2024-09-14T05:29:26.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.241540+0000", 2024-09-14T05:29:26.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000120493 2024-09-14T05:29:26.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243603+0000", 2024-09-14T05:29:26.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020638649999999998 2024-09-14T05:29:26.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248351+0000", 2024-09-14T05:29:26.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0047479030000000004 2024-09-14T05:29:26.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248381+0000", 2024-09-14T05:29:26.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0136000000000001e-05 2024-09-14T05:29:26.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248413+0000", 2024-09-14T05:29:26.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1047000000000003e-05 2024-09-14T05:29:26.414 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.414 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.414 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.414 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.414 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237682+0000", 2024-09-14T05:29:26.414 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.106958941999999, 2024-09-14T05:29:26.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010869477000000001, 2024-09-14T05:29:26.414 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.414 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.414 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.414 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.415 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.415 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.415 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 48 2024-09-14T05:29:26.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.415 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.415 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237682+0000", 2024-09-14T05:29:26.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.415 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237682+0000", 2024-09-14T05:29:26.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237683+0000", 2024-09-14T05:29:26.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8400000000000004e-07 2024-09-14T05:29:26.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237688+0000", 2024-09-14T05:29:26.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5519999999999998e-06 2024-09-14T05:29:26.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237696+0000", 2024-09-14T05:29:26.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4689999999999997e-06 2024-09-14T05:29:26.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237701+0000", 2024-09-14T05:29:26.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5950000000000001e-06 2024-09-14T05:29:26.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242083+0000", 2024-09-14T05:29:26.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004382782 2024-09-14T05:29:26.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242132+0000", 2024-09-14T05:29:26.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8171999999999999e-05 2024-09-14T05:29:26.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242205+0000", 2024-09-14T05:29:26.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3165999999999997e-05 2024-09-14T05:29:26.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243610+0000", 2024-09-14T05:29:26.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001405199 2024-09-14T05:29:26.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248464+0000", 2024-09-14T05:29:26.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048539079999999997 2024-09-14T05:29:26.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248514+0000", 2024-09-14T05:29:26.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9639999999999999e-05 2024-09-14T05:29:26.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248552+0000", 2024-09-14T05:29:26.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.841e-05 2024-09-14T05:29:26.420 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.420 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.420 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.421 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.421 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237709+0000", 2024-09-14T05:29:26.421 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.106932869000001, 2024-09-14T05:29:26.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010940642, 2024-09-14T05:29:26.421 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.421 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.421 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.421 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.421 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.421 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.421 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 49 2024-09-14T05:29:26.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.422 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237709+0000", 2024-09-14T05:29:26.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237709+0000", 2024-09-14T05:29:26.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237709+0000", 2024-09-14T05:29:26.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.27e-07 2024-09-14T05:29:26.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237712+0000", 2024-09-14T05:29:26.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3299999999999999e-06 2024-09-14T05:29:26.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237717+0000", 2024-09-14T05:29:26.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8890000000000001e-06 2024-09-14T05:29:26.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237720+0000", 2024-09-14T05:29:26.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.881e-06 2024-09-14T05:29:26.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242470+0000", 2024-09-14T05:29:26.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0047503780000000004 2024-09-14T05:29:26.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242549+0000", 2024-09-14T05:29:26.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8564000000000002e-05 2024-09-14T05:29:26.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242672+0000", 2024-09-14T05:29:26.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122769 2024-09-14T05:29:26.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.426 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243876+0000", 2024-09-14T05:29:26.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012039119999999999 2024-09-14T05:29:26.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.426 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248592+0000", 2024-09-14T05:29:26.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0047165619999999997 2024-09-14T05:29:26.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248620+0000", 2024-09-14T05:29:26.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7591999999999999e-05 2024-09-14T05:29:26.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248649+0000", 2024-09-14T05:29:26.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9337999999999999e-05 2024-09-14T05:29:26.427 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.427 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.427 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.428 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.428 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237731+0000", 2024-09-14T05:29:26.428 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.10691083, 2024-09-14T05:29:26.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.011041749, 2024-09-14T05:29:26.428 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.428 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.428 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.428 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.428 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.428 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.429 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 50 2024-09-14T05:29:26.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.429 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237731+0000", 2024-09-14T05:29:26.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237731+0000", 2024-09-14T05:29:26.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237731+0000", 2024-09-14T05:29:26.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1699999999999998e-07 2024-09-14T05:29:26.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237790+0000", 2024-09-14T05:29:26.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8440000000000003e-05 2024-09-14T05:29:26.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237800+0000", 2024-09-14T05:29:26.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0431000000000001e-05 2024-09-14T05:29:26.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237806+0000", 2024-09-14T05:29:26.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8889999999999999e-06 2024-09-14T05:29:26.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242992+0000", 2024-09-14T05:29:26.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0051858349999999998 2024-09-14T05:29:26.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243039+0000", 2024-09-14T05:29:26.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7574000000000002e-05 2024-09-14T05:29:26.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243118+0000", 2024-09-14T05:29:26.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8409000000000001e-05 2024-09-14T05:29:26.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.244673+0000", 2024-09-14T05:29:26.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015554189999999999 2024-09-14T05:29:26.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248687+0000", 2024-09-14T05:29:26.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040139670000000002 2024-09-14T05:29:26.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248736+0000", 2024-09-14T05:29:26.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9168000000000001e-05 2024-09-14T05:29:26.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248772+0000", 2024-09-14T05:29:26.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000003e-05 2024-09-14T05:29:26.434 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.434 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.434 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.434 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.434 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237820+0000", 2024-09-14T05:29:26.434 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.106821843999999, 2024-09-14T05:29:26.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.011040212000000001, 2024-09-14T05:29:26.434 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.434 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.435 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.435 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.435 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.435 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.435 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 51 2024-09-14T05:29:26.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.435 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237820+0000", 2024-09-14T05:29:26.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.436 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237820+0000", 2024-09-14T05:29:26.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.436 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237820+0000", 2024-09-14T05:29:26.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0500000000000003e-07 2024-09-14T05:29:26.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.436 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237824+0000", 2024-09-14T05:29:26.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9890000000000003e-06 2024-09-14T05:29:26.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237830+0000", 2024-09-14T05:29:26.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1179999999999997e-06 2024-09-14T05:29:26.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237835+0000", 2024-09-14T05:29:26.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3420000000000001e-06 2024-09-14T05:29:26.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243906+0000", 2024-09-14T05:29:26.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060709819999999999 2024-09-14T05:29:26.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243955+0000", 2024-09-14T05:29:26.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9431000000000002e-05 2024-09-14T05:29:26.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.244031+0000", 2024-09-14T05:29:26.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5765000000000004e-05 2024-09-14T05:29:26.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.244680+0000", 2024-09-14T05:29:26.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00064928100000000003 2024-09-14T05:29:26.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.439 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248801+0000", 2024-09-14T05:29:26.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041206020000000001 2024-09-14T05:29:26.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.439 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248829+0000", 2024-09-14T05:29:26.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7971999999999999e-05 2024-09-14T05:29:26.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.439 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248860+0000", 2024-09-14T05:29:26.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1124999999999998e-05 2024-09-14T05:29:26.439 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.440 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.440 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.440 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.440 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237844+0000", 2024-09-14T05:29:26.440 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.106797873000001, 2024-09-14T05:29:26.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.012929487999999999, 2024-09-14T05:29:26.440 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.440 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.440 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.440 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.440 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.440 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.441 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 52 2024-09-14T05:29:26.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.441 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.441 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237844+0000", 2024-09-14T05:29:26.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.441 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237844+0000", 2024-09-14T05:29:26.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237844+0000", 2024-09-14T05:29:26.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3099999999999998e-07 2024-09-14T05:29:26.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237848+0000", 2024-09-14T05:29:26.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2220000000000004e-06 2024-09-14T05:29:26.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237853+0000", 2024-09-14T05:29:26.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1760000000000001e-06 2024-09-14T05:29:26.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.443 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.443 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237858+0000", 2024-09-14T05:29:26.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8450000000000002e-06 2024-09-14T05:29:26.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.443 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.443 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.244269+0000", 2024-09-14T05:29:26.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064110970000000001 2024-09-14T05:29:26.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.443 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.443 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.244345+0000", 2024-09-14T05:29:26.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6013999999999994e-05 2024-09-14T05:29:26.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.244472+0000", 2024-09-14T05:29:26.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012630200000000001 2024-09-14T05:29:26.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.244871+0000", 2024-09-14T05:29:26.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00039959099999999998 2024-09-14T05:29:26.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.250638+0000", 2024-09-14T05:29:26.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057671759999999997 2024-09-14T05:29:26.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.250713+0000", 2024-09-14T05:29:26.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4762e-05 2024-09-14T05:29:26.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.250773+0000", 2024-09-14T05:29:26.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9772000000000003e-05 2024-09-14T05:29:26.446 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.446 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.446 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.446 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.446 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.527756+0000", 2024-09-14T05:29:26.446 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.816885032999998, 2024-09-14T05:29:26.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0071073860000000003, 2024-09-14T05:29:26.446 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.446 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.446 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.446 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.447 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.447 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.447 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 72 2024-09-14T05:29:26.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.447 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.447 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.447 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527756+0000", 2024-09-14T05:29:26.447 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.447 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.448 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527756+0000", 2024-09-14T05:29:26.448 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.448 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.448 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527757+0000", 2024-09-14T05:29:26.448 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2700000000000001e-07 2024-09-14T05:29:26.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.448 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.448 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527797+0000", 2024-09-14T05:29:26.448 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9915e-05 2024-09-14T05:29:26.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527801+0000", 2024-09-14T05:29:26.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0239999999999999e-06 2024-09-14T05:29:26.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527805+0000", 2024-09-14T05:29:26.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0559999999999998e-06 2024-09-14T05:29:26.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.532244+0000", 2024-09-14T05:29:26.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0044392149999999998 2024-09-14T05:29:26.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.532293+0000", 2024-09-14T05:29:26.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8971999999999998e-05 2024-09-14T05:29:26.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.532359+0000", 2024-09-14T05:29:26.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5786999999999999e-05 2024-09-14T05:29:26.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.533307+0000", 2024-09-14T05:29:26.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00094854300000000004 2024-09-14T05:29:26.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.534813+0000", 2024-09-14T05:29:26.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015063699999999999 2024-09-14T05:29:26.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.534841+0000", 2024-09-14T05:29:26.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7412000000000001e-05 2024-09-14T05:29:26.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.452 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.452 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.534864+0000", 2024-09-14T05:29:26.452 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2864999999999999e-05 2024-09-14T05:29:26.452 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.452 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.452 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.452 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.452 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.527810+0000", 2024-09-14T05:29:26.452 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.816831684, 2024-09-14T05:29:26.452 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0072560050000000003, 2024-09-14T05:29:26.453 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.453 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.453 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.453 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.453 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.453 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.453 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73 2024-09-14T05:29:26.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.453 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.453 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527810+0000", 2024-09-14T05:29:26.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527810+0000", 2024-09-14T05:29:26.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527810+0000", 2024-09-14T05:29:26.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1600000000000002e-07 2024-09-14T05:29:26.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527812+0000", 2024-09-14T05:29:26.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0870000000000001e-06 2024-09-14T05:29:26.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527815+0000", 2024-09-14T05:29:26.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.219e-06 2024-09-14T05:29:26.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527819+0000", 2024-09-14T05:29:26.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.783e-06 2024-09-14T05:29:26.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.532774+0000", 2024-09-14T05:29:26.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0049551150000000004 2024-09-14T05:29:26.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.456 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.456 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.532815+0000", 2024-09-14T05:29:26.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0268999999999997e-05 2024-09-14T05:29:26.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.456 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.456 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.532876+0000", 2024-09-14T05:29:26.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1643999999999995e-05 2024-09-14T05:29:26.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.534907+0000", 2024-09-14T05:29:26.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020311180000000002 2024-09-14T05:29:26.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.534979+0000", 2024-09-14T05:29:26.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1422999999999997e-05 2024-09-14T05:29:26.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535027+0000", 2024-09-14T05:29:26.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8401e-05 2024-09-14T05:29:26.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535066+0000", 2024-09-14T05:29:26.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8630000000000001e-05 2024-09-14T05:29:26.458 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.458 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.458 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.458 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.458 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.527837+0000", 2024-09-14T05:29:26.458 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.816804267999999, 2024-09-14T05:29:26.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.007313127, 2024-09-14T05:29:26.459 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.459 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.459 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.459 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.459 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.459 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.459 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74 2024-09-14T05:29:26.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.459 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527837+0000", 2024-09-14T05:29:26.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527837+0000", 2024-09-14T05:29:26.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527837+0000", 2024-09-14T05:29:26.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3599999999999999e-07 2024-09-14T05:29:26.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527841+0000", 2024-09-14T05:29:26.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7179999999999998e-06 2024-09-14T05:29:26.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527847+0000", 2024-09-14T05:29:26.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3310000000000003e-06 2024-09-14T05:29:26.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527850+0000", 2024-09-14T05:29:26.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2550000000000002e-06 2024-09-14T05:29:26.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.533411+0000", 2024-09-14T05:29:26.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005561646 2024-09-14T05:29:26.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.533455+0000", 2024-09-14T05:29:26.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3476999999999997e-05 2024-09-14T05:29:26.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.533521+0000", 2024-09-14T05:29:26.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6482999999999993e-05 2024-09-14T05:29:26.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.534963+0000", 2024-09-14T05:29:26.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001442079 2024-09-14T05:29:26.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535074+0000", 2024-09-14T05:29:26.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000110454 2024-09-14T05:29:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535115+0000", 2024-09-14T05:29:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1430000000000001e-05 2024-09-14T05:29:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535150+0000", 2024-09-14T05:29:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4918e-05 2024-09-14T05:29:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.527884+0000", 2024-09-14T05:29:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.816757003999999, 2024-09-14T05:29:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0077308359999999996, 2024-09-14T05:29:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.465 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 75 2024-09-14T05:29:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527884+0000", 2024-09-14T05:29:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527884+0000", 2024-09-14T05:29:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527885+0000", 2024-09-14T05:29:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3099999999999998e-07 2024-09-14T05:29:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527925+0000", 2024-09-14T05:29:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9823000000000001e-05 2024-09-14T05:29:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527931+0000", 2024-09-14T05:29:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8159999999999996e-06 2024-09-14T05:29:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527935+0000", 2024-09-14T05:29:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.213e-06 2024-09-14T05:29:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.533763+0000", 2024-09-14T05:29:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0058281449999999999 2024-09-14T05:29:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.533806+0000", 2024-09-14T05:29:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3523999999999999e-05 2024-09-14T05:29:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.533903+0000", 2024-09-14T05:29:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6849999999999996e-05 2024-09-14T05:29:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535518+0000", 2024-09-14T05:29:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0016150959999999999 2024-09-14T05:29:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535533+0000", 2024-09-14T05:29:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5003e-05 2024-09-14T05:29:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535576+0000", 2024-09-14T05:29:26.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2540000000000003e-05 2024-09-14T05:29:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535615+0000", 2024-09-14T05:29:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9394999999999998e-05 2024-09-14T05:29:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.471 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.527940+0000", 2024-09-14T05:29:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.816701213000002, 2024-09-14T05:29:26.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010608569999999999, 2024-09-14T05:29:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:26.473 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 76 2024-09-14T05:29:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527940+0000", 2024-09-14T05:29:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527940+0000", 2024-09-14T05:29:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527940+0000", 2024-09-14T05:29:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0400000000000002e-07 2024-09-14T05:29:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527944+0000", 2024-09-14T05:29:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5700000000000001e-06 2024-09-14T05:29:26.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527948+0000", 2024-09-14T05:29:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9029999999999997e-06 2024-09-14T05:29:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527951+0000", 2024-09-14T05:29:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6309999999999999e-06 2024-09-14T05:29:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535166+0000", 2024-09-14T05:29:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0072155479999999996 2024-09-14T05:29:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.479 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535211+0000", 2024-09-14T05:29:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5108999999999998e-05 2024-09-14T05:29:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535282+0000", 2024-09-14T05:29:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0364999999999995e-05 2024-09-14T05:29:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.536073+0000", 2024-09-14T05:29:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00079145699999999997 2024-09-14T05:29:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.538464+0000", 2024-09-14T05:29:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023912429999999999 2024-09-14T05:29:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.538511+0000", 2024-09-14T05:29:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6328000000000001e-05 2024-09-14T05:29:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.538549+0000", 2024-09-14T05:29:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8111999999999997e-05 2024-09-14T05:29:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.482 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.482 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:26.484 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:29:26.484 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:26.482+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:26.585 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:26.584+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:26.628 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:26.629 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:26.629 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:29:26.629 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:26.637 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:29:26.686 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:26.685+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:26.740 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:26.741 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:26.741 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:26.741 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:26.741 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:26.750 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:29:26.791 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:26.790+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:26.887 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:26.887+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:26.893 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:26.893 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:26.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:26.893 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:26.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.894 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4235.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-14T05:29:26.894 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:13.836551+0000", 2024-09-14T05:29:26.894 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 13.056739113000001, 2024-09-14T05:29:26.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00065322999999999998, 2024-09-14T05:29:26.894 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.894 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.894 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.894 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.836551+0000", 2024-09-14T05:29:26.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.836551+0000", 2024-09-14T05:29:26.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.836551+0000", 2024-09-14T05:29:26.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9400000000000001e-07 2024-09-14T05:29:26.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.836559+0000", 2024-09-14T05:29:26.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1629999999999996e-06 2024-09-14T05:29:26.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.836573+0000", 2024-09-14T05:29:26.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3397000000000001e-05 2024-09-14T05:29:26.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.836584+0000", 2024-09-14T05:29:26.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1484000000000001e-05 2024-09-14T05:29:26.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.836639+0000", 2024-09-14T05:29:26.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5031999999999998e-05 2024-09-14T05:29:26.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.836666+0000", 2024-09-14T05:29:26.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6588e-05 2024-09-14T05:29:26.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.837165+0000", 2024-09-14T05:29:26.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049923800000000005 2024-09-14T05:29:26.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.837204+0000", 2024-09-14T05:29:26.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8933999999999997e-05 2024-09-14T05:29:26.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.898 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.898 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:13 3.1e e27/24 3:7a918c66:::rbd_object_map.112d8b048597:head v 27'1, mlcod=0'0)", 2024-09-14T05:29:26.898 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:26.026611+0000", 2024-09-14T05:29:26.898 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.86667896, 2024-09-14T05:29:26.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00088820700000000002, 2024-09-14T05:29:26.899 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:26.899 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:26.899 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:26.899 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:26.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:26.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.026611+0000", 2024-09-14T05:29:26.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:26.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.026611+0000", 2024-09-14T05:29:26.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:26.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:26.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.026611+0000", 2024-09-14T05:29:26.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2e-07 2024-09-14T05:29:26.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:26.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.026617+0000", 2024-09-14T05:29:26.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5910000000000003e-06 2024-09-14T05:29:26.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:26.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.026637+0000", 2024-09-14T05:29:26.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.95e-05 2024-09-14T05:29:26.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:26.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.026651+0000", 2024-09-14T05:29:26.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.418e-05 2024-09-14T05:29:26.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:26.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.026721+0000", 2024-09-14T05:29:26.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0508999999999999e-05 2024-09-14T05:29:26.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:26.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.026745+0000", 2024-09-14T05:29:26.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3305000000000001e-05 2024-09-14T05:29:26.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:26.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.027452+0000", 2024-09-14T05:29:26.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070784299999999999 2024-09-14T05:29:26.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:26.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:26.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:26.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.027499+0000", 2024-09-14T05:29:26.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6859e-05 2024-09-14T05:29:26.902 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.902 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.903 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.903 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:26.903 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:26.903 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:26.904 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:29:26.937 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:29:26.988 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:26.988+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:27.010 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:27.010 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:27.010 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:29:27.010 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:27.020 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:29:27.073 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:29:27.073 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:29:27.089 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:27.088+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:27.091 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:29:27.091 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:29:27.108 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:29:27.108 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:29:27.125 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:29:27.126 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:29:27.128 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:27.128 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:27.137 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:29:27.143 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:29:27.143 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:29:27.160 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:29:27.160 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:29:27.177 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:29:27.178 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:29:27.190 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:27.189+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:27.194 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:29:27.195 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:29:27.249 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:27.249 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:27.258 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:29:27.303 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:27.302+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:27.374 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:27.374 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:27.383 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:29:27.391 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:27.391+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:27.492 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:27.492+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:27.527 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:27.527 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:27.536 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:29:27.593 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:27.593+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:27.639 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:27.639 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:27.648 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:29:27.695 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:27.694+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:27.741 INFO:teuthology.orchestra.run.smithi071.stdout:2 writesame 0x7fc377c thru 0x7fd2d52 data_size 0x29(0xf5d7 bytes) 2024-09-14T05:29:27.741 INFO:teuthology.orchestra.run.smithi071.stdout:truncating to largest ever: 0x631b966 2024-09-14T05:29:27.743 INFO:teuthology.orchestra.run.smithi071.stdout:3 trunc from 0x7fd2d53 to 0x631b966 2024-09-14T05:29:27.755 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:27.756 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:27.765 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:29:27.766 INFO:teuthology.orchestra.run.smithi071.stdout:4 write 0xe899f69 thru 0xe8a3358 (0x93f0 bytes) 2024-09-14T05:29:27.769 INFO:teuthology.orchestra.run.smithi071.stdout:5 write 0x4c9617c thru 0x4ca422a (0xe0af bytes) 2024-09-14T05:29:27.773 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:27.778 INFO:teuthology.orchestra.run.smithi071.stdout:9 writesame 0xeda04b4 thru 0xedabdc3 data_size 0x5e(0xb910 bytes) 2024-09-14T05:29:27.778 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:27.779 INFO:teuthology.orchestra.run.smithi071.stdout:11 punch from 0xb3bd0bc to 0xb3c1f19, (0x4e5d bytes) 2024-09-14T05:29:27.792 INFO:teuthology.orchestra.run.smithi071.stdout:12 trunc from 0xedabdc4 to 0x3065595 2024-09-14T05:29:27.795 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:27.794+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:27.802 INFO:teuthology.orchestra.run.smithi071.stdout:13 read 0x577ea0 thru 0x585b69 (0xdcca bytes) 2024-09-14T05:29:27.822 INFO:teuthology.orchestra.run.smithi071.stdout:15 write 0xe9d90b3 thru 0xe9deb5e (0x5aac bytes) 2024-09-14T05:29:27.825 INFO:teuthology.orchestra.run.smithi071.stdout:17 write 0x3bb974 thru 0x3c135a (0x59e7 bytes) 2024-09-14T05:29:27.829 INFO:teuthology.orchestra.run.smithi071.stdout:18 read 0xc08445e thru 0xc092c88 (0xe82b bytes) 2024-09-14T05:29:27.829 INFO:teuthology.orchestra.run.smithi071.stdout:21 writesame 0x52db9e4 thru 0x52de5d7 data_size 0x1d(0x2bf4 bytes) 2024-09-14T05:29:27.834 INFO:teuthology.orchestra.run.smithi071.stdout:23 punch from 0x9e1750e to 0x9e26187, (0xec79 bytes) 2024-09-14T05:29:27.834 INFO:teuthology.orchestra.run.smithi071.stdout:24 punch from 0x77639eb to 0x777254c, (0xeb61 bytes) 2024-09-14T05:29:27.834 INFO:teuthology.orchestra.run.smithi071.stdout:25 writesame 0x451191a thru 0x451604a data_size 0x51(0x4731 bytes) 2024-09-14T05:29:27.837 INFO:teuthology.orchestra.run.smithi071.stdout:26 read 0x8729960 thru 0x8737fa5 (0xe646 bytes) 2024-09-14T05:29:27.837 INFO:teuthology.orchestra.run.smithi071.stdout:27 read 0x9c202cf thru 0x9c24fc0 (0x4cf2 bytes) 2024-09-14T05:29:27.837 INFO:teuthology.orchestra.run.smithi071.stdout:28 write 0x5453389 thru 0x5461f34 (0xebac bytes) 2024-09-14T05:29:27.841 INFO:teuthology.orchestra.run.smithi071.stdout:29 write 0x4e3ea50 thru 0x4e4683c (0x7ded bytes) 2024-09-14T05:29:27.844 INFO:teuthology.orchestra.run.smithi071.stdout:30 punch from 0x83e5b55 to 0x83e90f1, (0x359c bytes) 2024-09-14T05:29:27.844 INFO:teuthology.orchestra.run.smithi071.stdout:31 writesame 0x97947b4 thru 0x97a29d1 data_size 0x67(0xe21e bytes) 2024-09-14T05:29:27.847 INFO:teuthology.orchestra.run.smithi071.stdout:truncating to largest ever: 0xdcc5cd1 2024-09-14T05:29:27.848 INFO:teuthology.orchestra.run.smithi071.stdout:32 trunc from 0xe9deb5f to 0xdcc5cd1 2024-09-14T05:29:27.863 INFO:teuthology.orchestra.run.smithi071.stdout:33 write 0x34e8642 thru 0x34f4e90 (0xc84f bytes) 2024-09-14T05:29:27.867 INFO:teuthology.orchestra.run.smithi071.stdout:34 trunc from 0xdcc5cd1 to 0xdc93f9a 2024-09-14T05:29:27.874 INFO:teuthology.orchestra.run.smithi071.stdout:35 write 0x101e0ef thru 0x1020b15 (0x2a27 bytes) 2024-09-14T05:29:27.878 INFO:teuthology.orchestra.run.smithi071.stdout:36 read 0x384ee95 thru 0x38503c9 (0x1535 bytes) 2024-09-14T05:29:27.880 INFO:teuthology.orchestra.run.smithi071.stdout:37 trunc from 0xdc93f9a to 0xbcfce6a 2024-09-14T05:29:27.886 INFO:teuthology.orchestra.run.smithi071.stdout:38 punch from 0x8122f7c to 0x812c945, (0x99c9 bytes) 2024-09-14T05:29:27.886 INFO:teuthology.orchestra.run.smithi071.stdout:39 read 0x9f56d3a thru 0x9f5bf13 (0x51da bytes) 2024-09-14T05:29:27.886 INFO:teuthology.orchestra.run.smithi071.stdout:40 write 0x8225296 thru 0x822bad2 (0x683d bytes) 2024-09-14T05:29:27.889 INFO:teuthology.orchestra.run.smithi071.stdout:41 read 0x522b3c1 thru 0x52399f6 (0xe636 bytes) 2024-09-14T05:29:27.895 INFO:teuthology.orchestra.run.smithi071.stdout:42 write 0xbd89c14 thru 0xbd98d32 (0xf11f bytes) 2024-09-14T05:29:27.896 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:27.895+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:27.900 INFO:teuthology.orchestra.run.smithi071.stdout:43 punch from 0x7cfea17 to 0x7d0c510, (0xdaf9 bytes) 2024-09-14T05:29:27.900 INFO:teuthology.orchestra.run.smithi071.stdout:44 read 0x14041e1 thru 0x14138c6 (0xf6e6 bytes) 2024-09-14T05:29:27.900 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:27.900 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:27.900 INFO:teuthology.orchestra.run.smithi071.stdout:48 write 0x9265151 thru 0x9269177 (0x4027 bytes) 2024-09-14T05:29:27.905 INFO:teuthology.orchestra.run.smithi071.stdout:49 write 0xb4bad44 thru 0xb4c8302 (0xd5bf bytes) 2024-09-14T05:29:27.910 INFO:teuthology.orchestra.run.smithi071.stdout:50 punch from 0x149b9fb to 0x14ab957, (0xff5c bytes) 2024-09-14T05:29:27.910 INFO:teuthology.orchestra.run.smithi071.stdout:51 write 0x42d500c thru 0x42dccb1 (0x7ca6 bytes) 2024-09-14T05:29:27.915 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:27.915 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:27.924 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:29:27.925 INFO:teuthology.orchestra.run.smithi071.stdout:52 trunc from 0xbd98d33 to 0x2b789eb 2024-09-14T05:29:27.959 INFO:teuthology.orchestra.run.smithi071.stdout:53 writesame 0xccbb925 thru 0xccbf9d0 data_size 0x1(0x40ac bytes) 2024-09-14T05:29:27.959 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:27.959 INFO:teuthology.orchestra.run.smithi071.stdout:56 read 0x1f15888 thru 0x1f158e4 (0x5d bytes) 2024-09-14T05:29:27.959 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:27.973 INFO:teuthology.orchestra.run.smithi071.stdout:58 trunc from 0xccbf9d1 to 0xd14d94 2024-09-14T05:29:27.983 INFO:teuthology.orchestra.run.smithi071.stdout:59 clone 1 order 22 su 131072 sc 6 2024-09-14T05:29:27.996 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:27.996+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:28.025 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T05:29:28.025 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T05:29:28.025 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:29:28.031 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T05:29:28.032 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:28.032 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:28.041 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:29:28.111 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:28.111+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:28.114 INFO:teuthology.orchestra.run.smithi071.stdout:60 write 0x7571469 thru 0x757a9fa (0x9592 bytes) 2024-09-14T05:29:28.117 INFO:teuthology.orchestra.run.smithi071.stdout:61 write 0x1df6419 thru 0x1df747e (0x1066 bytes) 2024-09-14T05:29:28.125 INFO:teuthology.orchestra.run.smithi071.stdout:62 trunc from 0x757a9fb to 0x3b6742b 2024-09-14T05:29:28.134 INFO:teuthology.orchestra.run.smithi071.stdout:63 writesame 0x1debc35 thru 0x1dee2b9 data_size 0x13(0x2685 bytes) 2024-09-14T05:29:28.143 INFO:teuthology.orchestra.run.smithi071.stdout:64 trunc from 0x3b6742b to 0x972a3c8 2024-09-14T05:29:28.148 INFO:teuthology.orchestra.run.smithi071.stdout:66 write 0x175b3ec thru 0x175bbbb (0x7d0 bytes) 2024-09-14T05:29:28.152 INFO:teuthology.orchestra.run.smithi071.stdout:67 trunc from 0x972a3c8 to 0x9354970 2024-09-14T05:29:28.158 INFO:teuthology.orchestra.run.smithi071.stdout:69 read 0x5452337 thru 0x545eda1 (0xca6b bytes) 2024-09-14T05:29:28.158 INFO:teuthology.orchestra.run.smithi071.stdout:71 write 0x6cb9c8b thru 0x6cc41d0 (0xa546 bytes) 2024-09-14T05:29:28.161 INFO:teuthology.orchestra.run.smithi071.stdout:72 write 0x8cd68ff thru 0x8ce29b7 (0xc0b9 bytes) 2024-09-14T05:29:28.168 INFO:teuthology.orchestra.run.smithi071.stdout:73 write 0x8b4eb48 thru 0x8b5266a (0x3b23 bytes) 2024-09-14T05:29:28.170 INFO:teuthology.orchestra.run.smithi071.stdout:74 punch from 0x5c22505 to 0x5c2ff08, (0xda03 bytes) 2024-09-14T05:29:28.170 INFO:teuthology.orchestra.run.smithi071.stdout:75 punch from 0x7902635 to 0x7909b1f, (0x74ea bytes) 2024-09-14T05:29:28.170 INFO:teuthology.orchestra.run.smithi071.stdout:76 writesame 0x70db56b thru 0x70e01ad data_size 0x1(0x4c43 bytes) 2024-09-14T05:29:28.182 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:28.182 INFO:teuthology.orchestra.run.smithi071.stdout:78 write 0x8b120e5 thru 0x8b15ed4 (0x3df0 bytes) 2024-09-14T05:29:28.185 INFO:teuthology.orchestra.run.smithi071.stdout:79 read 0x57fc129 thru 0x58014b5 (0x538d bytes) 2024-09-14T05:29:28.185 INFO:teuthology.orchestra.run.smithi071.stdout:80 read 0x6fb0477 thru 0x6fbe6b4 (0xe23e bytes) 2024-09-14T05:29:28.186 INFO:teuthology.orchestra.run.smithi071.stdout:83 writesame 0x7ed5aad thru 0x7ee0743 data_size 0x17(0xac97 bytes) 2024-09-14T05:29:28.191 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:28.191 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:28.192 INFO:teuthology.orchestra.run.smithi071.stdout:84 read 0xc775b6 thru 0xc80d4c (0x9797 bytes) 2024-09-14T05:29:28.198 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:28.197+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:28.199 INFO:teuthology.orchestra.run.smithi071.stdout:85 trunc from 0x9354970 to 0x2cdf1e8 2024-09-14T05:29:28.200 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:29:28.222 INFO:teuthology.orchestra.run.smithi071.stdout:88 trunc from 0x2cdf1e8 to 0xc0b26c3 2024-09-14T05:29:28.227 INFO:teuthology.orchestra.run.smithi071.stdout:89 read 0x8518db9 thru 0x851fbf4 (0x6e3c bytes) 2024-09-14T05:29:28.228 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:28.228 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:28.228 INFO:teuthology.orchestra.run.smithi071.stdout:92 writesame 0x7f1a2c8 thru 0x7f1ed6a data_size 0x21(0x4aa3 bytes) 2024-09-14T05:29:28.230 INFO:teuthology.orchestra.run.smithi071.stdout:truncating to largest ever: 0xe104af0 2024-09-14T05:29:28.233 INFO:teuthology.orchestra.run.smithi071.stdout:93 trunc from 0xc0b26c3 to 0xe104af0 2024-09-14T05:29:28.237 INFO:teuthology.orchestra.run.smithi071.stdout:94 writesame 0x93df936 thru 0x93e6fe6 data_size 0x3b(0x76b1 bytes) 2024-09-14T05:29:28.242 INFO:teuthology.orchestra.run.smithi071.stdout:95 read 0xc1946b4 thru 0xc1a0795 (0xc0e2 bytes) 2024-09-14T05:29:28.242 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:28.242 INFO:teuthology.orchestra.run.smithi071.stdout:98 write 0xd2c4877 thru 0xd2cc351 (0x7adb bytes) 2024-09-14T05:29:28.245 INFO:teuthology.orchestra.run.smithi071.stdout:101 read 0x2b8d07a thru 0x2b9acfa (0xdc81 bytes) 2024-09-14T05:29:28.245 INFO:teuthology.orchestra.run.smithi071.stdout:102 write 0xd2b067 thru 0xd3647f (0xb419 bytes) 2024-09-14T05:29:28.259 INFO:teuthology.orchestra.run.smithi071.stdout:104 trunc from 0xe104af0 to 0x66a3696 2024-09-14T05:29:28.271 INFO:teuthology.orchestra.run.smithi071.stdout:105 punch from 0x517abf4 to 0x518359e, (0x89aa bytes) 2024-09-14T05:29:28.271 INFO:teuthology.orchestra.run.smithi071.stdout:107 read 0x631a837 thru 0x6322613 (0x7ddd bytes) 2024-09-14T05:29:28.280 INFO:teuthology.orchestra.run.smithi071.stdout:108 write 0x99e3887 thru 0x99f0c1d (0xd397 bytes) 2024-09-14T05:29:28.286 INFO:teuthology.orchestra.run.smithi071.stdout:109 trunc from 0x99f0c1e to 0xc311b49 2024-09-14T05:29:28.299 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:28.298+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:28.303 INFO:teuthology.orchestra.run.smithi071.stdout:110 trunc from 0xc311b49 to 0x371cda6 2024-09-14T05:29:28.313 INFO:teuthology.orchestra.run.smithi071.stdout:116 punch from 0x3004593 to 0x300e5e8, (0xa055 bytes) 2024-09-14T05:29:28.313 INFO:teuthology.orchestra.run.smithi071.stdout:117 read 0x831b69 thru 0x8401c5 (0xe65d bytes) 2024-09-14T05:29:28.313 INFO:teuthology.orchestra.run.smithi071.stdout:118 punch from 0x13fbcd0 to 0x13fc00a, (0x33a bytes) 2024-09-14T05:29:28.314 INFO:teuthology.orchestra.run.smithi071.stdout:120 punch from 0xf7c585 to 0xf85709, (0x9184 bytes) 2024-09-14T05:29:28.314 INFO:teuthology.orchestra.run.smithi071.stdout:121 punch from 0x22fdd1 to 0x23bb73, (0xbda2 bytes) 2024-09-14T05:29:28.314 INFO:teuthology.orchestra.run.smithi071.stdout:123 read 0x545a4c thru 0x54feaa (0xa45f bytes) 2024-09-14T05:29:28.333 INFO:teuthology.orchestra.run.smithi071.stdout:124 writesame 0xecf38a4 thru 0xed02b28 data_size 0x5(0xf285 bytes) 2024-09-14T05:29:28.333 INFO:teuthology.orchestra.run.smithi071.stdout:125 write 0x7fec532 thru 0x7ff4ac4 (0x8593 bytes) 2024-09-14T05:29:28.336 INFO:teuthology.orchestra.run.smithi071.stdout:126 punch from 0xdd9da13 to 0xdda525d, (0x784a bytes) 2024-09-14T05:29:28.336 INFO:teuthology.orchestra.run.smithi071.stdout:128 write 0xa40482d thru 0xa405056 (0x82a bytes) 2024-09-14T05:29:28.339 INFO:teuthology.orchestra.run.smithi071.stdout:131 punch from 0xbd1a769 to 0xbd1aec0, (0x757 bytes) 2024-09-14T05:29:28.339 INFO:teuthology.orchestra.run.smithi071.stdout:132 write 0x893e5ab thru 0x8942ac1 (0x4517 bytes) 2024-09-14T05:29:28.346 INFO:teuthology.orchestra.run.smithi071.stdout:133 write 0xa592749 thru 0xa5a0ebb (0xe773 bytes) 2024-09-14T05:29:28.349 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:28.349 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:28.351 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:28.351 INFO:teuthology.orchestra.run.smithi071.stdout:135 writesame 0xad251cd thru 0xad2e563 data_size 0x1(0x9397 bytes) 2024-09-14T05:29:28.358 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:29:28.373 INFO:teuthology.orchestra.run.smithi071.stdout:136 read 0xe06cd5a thru 0xe06e747 (0x19ee bytes) 2024-09-14T05:29:28.373 INFO:teuthology.orchestra.run.smithi071.stdout:truncating to largest ever: 0xe4b1591 2024-09-14T05:29:28.373 INFO:teuthology.orchestra.run.smithi071.stdout:138 trunc from 0xed02b29 to 0xe4b1591 2024-09-14T05:29:28.379 INFO:teuthology.orchestra.run.smithi071.stdout:140 punch from 0xa600aa0 to 0xa6023b5, (0x1915 bytes) 2024-09-14T05:29:28.379 INFO:teuthology.orchestra.run.smithi071.stdout:141 read 0x13fd1fa thru 0x1403412 (0x6219 bytes) 2024-09-14T05:29:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:29:28.381 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:29:28.381 INFO:teuthology.orchestra.run.smithi071.stdout:142 read 0xdd08b15 thru 0xdd0fd90 (0x727c bytes) 2024-09-14T05:29:28.381 INFO:teuthology.orchestra.run.smithi071.stdout:143 punch from 0xa9ad82e to 0xa9b5522, (0x7cf4 bytes) 2024-09-14T05:29:28.382 INFO:teuthology.orchestra.run.smithi071.stdout:144 clone 2 order 18 su 131072 sc 13 2024-09-14T05:29:28.392 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-14T05:29:28.018246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590452,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":100,"num_read_kb":81,"num_write":109,"num_write_kb":1636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219620,"kb_used_data":3368,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755100,"statfs":{"total":773094113280,"available":772869222400,"internally_reserved":0,"allocated":3448832,"data_stored":2006529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.015831"},"pg_stats":[{"pgid":"3.1f","version":"27'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.032008+0000","last_change":"2024-09-14T05:29:26.019843+0000","last_active":"2024-09-14T05:29:26.032008+0000","last_peered":"2024-09-14T05:29:26.032008+0000","last_clean":"2024-09-14T05:29:26.032008+0000","last_became_active":"2024-09-14T05:29:24.018892+0000","last_became_peered":"2024-09-14T05:29:24.018892+0000","last_unstale":"2024-09-14T05:29:26.032008+0000","last_undegraded":"2024-09-14T05:29:26.032008+0000","last_fullsized":"2024-09-14T05:29:26.032008+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:33:43.537596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011130109999999999,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"27'1","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.027639+0000","last_change":"2024-09-14T05:29:26.019068+0000","last_active":"2024-09-14T05:29:26.027639+0000","last_peered":"2024-09-14T05:29:26.027639+0000","last_clean":"2024-09-14T05:29:26.027639+0000","last_became_active":"2024-09-14T05:29:24.014468+0000","last_became_peered":"2024-09-14T05:29:24.014468+0000","last_unstale":"2024-09-14T05:29:26.027639+0000","last_undegraded":"2024-09-14T05:29:26.027639+0000","last_fullsized":"2024-09-14T05:29:26.027639+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:51:11.696879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000399031,"stat_sum":{"num_bytes":30,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.165031+0000","last_change":"2024-09-14T05:29:26.165031+0000","last_active":"2024-09-14T05:29:26.165031+0000","last_peered":"2024-09-14T05:29:26.165031+0000","last_clean":"2024-09-14T05:29:26.165031+0000","last_became_active":"2024-09-14T05:29:24.014251+0000","last_became_peered":"2024-09-14T05:29:24.014251+0000","last_unstale":"2024-09-14T05:29:26.165031+0000","last_undegraded":"2024-09-14T05:29:26.165031+0000","last_fullsized":"2024-09-14T05:29:26.165031+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:04:28.028774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010261580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"27'1","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.018757+0000","last_change":"2024-09-14T05:29:26.014243+0000","last_active":"2024-09-14T05:29:26.018757+0000","last_peered":"2024-09-14T05:29:26.018757+0000","last_clean":"2024-09-14T05:29:26.018757+0000","last_became_active":"2024-09-14T05:29:24.012427+0000","last_became_peered":"2024-09-14T05:29:24.012427+0000","last_unstale":"2024-09-14T05:29:26.018757+0000","last_undegraded":"2024-09-14T05:29:26.018757+0000","last_fullsized":"2024-09-14T05:29:26.018757+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:38:02.144339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045797999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.014462+0000","last_change":"2024-09-14T05:29:26.014462+0000","last_active":"2024-09-14T05:29:26.014462+0000","last_peered":"2024-09-14T05:29:26.014462+0000","last_clean":"2024-09-14T05:29:26.014462+0000","last_became_active":"2024-09-14T05:29:24.012079+0000","last_became_peered":"2024-09-14T05:29:24.012079+0000","last_unstale":"2024-09-14T05:29:26.014462+0000","last_undegraded":"2024-09-14T05:29:26.014462+0000","last_fullsized":"2024-09-14T05:29:26.014462+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:03:03.882387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070589999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"27'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.034908+0000","last_change":"2024-09-14T05:29:26.014453+0000","last_active":"2024-09-14T05:29:26.034908+0000","last_peered":"2024-09-14T05:29:26.034908+0000","last_clean":"2024-09-14T05:29:26.034908+0000","last_became_active":"2024-09-14T05:29:24.012461+0000","last_became_peered":"2024-09-14T05:29:24.012461+0000","last_unstale":"2024-09-14T05:29:26.034908+0000","last_undegraded":"2024-09-14T05:29:26.034908+0000","last_fullsized":"2024-09-14T05:29:26.034908+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:03:38.326321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061290100000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.014685+0000","last_change":"2024-09-14T05:29:26.014685+0000","last_active":"2024-09-14T05:29:26.014685+0000","last_peered":"2024-09-14T05:29:26.014685+0000","last_clean":"2024-09-14T05:29:26.014685+0000","last_became_active":"2024-09-14T05:29:24.018753+0000","last_became_peered":"2024-09-14T05:29:24.018753+0000","last_unstale":"2024-09-14T05:29:26.014685+0000","last_undegraded":"2024-09-14T05:29:26.014685+0000","last_fullsized":"2024-09-14T05:29:26.014685+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:53:55.480798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089943399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.019928+0000","last_change":"2024-09-14T05:29:26.019928+0000","last_active":"2024-09-14T05:29:26.019928+0000","last_peered":"2024-09-14T05:29:26.019928+0000","last_clean":"2024-09-14T05:29:26.019928+0000","last_became_active":"2024-09-14T05:29:24.011176+0000","last_became_peered":"2024-09-14T05:29:24.011176+0000","last_unstale":"2024-09-14T05:29:26.019928+0000","last_undegraded":"2024-09-14T05:29:26.019928+0000","last_fullsized":"2024-09-14T05:29:26.019928+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:20:38.544244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011097850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.019132+0000","last_change":"2024-09-14T05:29:26.019132+0000","last_active":"2024-09-14T05:29:26.019132+0000","last_peered":"2024-09-14T05:29:26.019132+0000","last_clean":"2024-09-14T05:29:26.019132+0000","last_became_active":"2024-09-14T05:29:24.014005+0000","last_became_peered":"2024-09-14T05:29:24.014005+0000","last_unstale":"2024-09-14T05:29:26.019132+0000","last_undegraded":"2024-09-14T05:29:26.019132+0000","last_fullsized":"2024-09-14T05:29:26.019132+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:02:03.515010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045225300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.028571+0000","last_change":"2024-09-14T05:29:26.028571+0000","last_active":"2024-09-14T05:29:26.028571+0000","last_peered":"2024-09-14T05:29:26.028571+0000","last_clean":"2024-09-14T05:29:26.028571+0000","last_became_active":"2024-09-14T05:29:24.013434+0000","last_became_peered":"2024-09-14T05:29:24.013434+0000","last_unstale":"2024-09-14T05:29:26.028571+0000","last_undegraded":"2024-09-14T05:29:26.028571+0000","last_fullsized":"2024-09-14T05:29:26.028571+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:23:56.619936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038301200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.020364+0000","last_change":"2024-09-14T05:29:26.020364+0000","last_active":"2024-09-14T05:29:26.020364+0000","last_peered":"2024-09-14T05:29:26.020364+0000","last_clean":"2024-09-14T05:29:26.020364+0000","last_became_active":"2024-09-14T05:29:24.014457+0000","last_became_peered":"2024-09-14T05:29:24.014457+0000","last_unstale":"2024-09-14T05:29:26.020364+0000","last_undegraded":"2024-09-14T05:29:26.020364+0000","last_fullsized":"2024-09-14T05:29:26.020364+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:17:58.899443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010888930000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.164291+0000","last_change":"2024-09-14T05:29:26.164291+0000","last_active":"2024-09-14T05:29:26.164291+0000","last_peered":"2024-09-14T05:29:26.164291+0000","last_clean":"2024-09-14T05:29:26.164291+0000","last_became_active":"2024-09-14T05:29:24.013266+0000","last_became_peered":"2024-09-14T05:29:24.013266+0000","last_unstale":"2024-09-14T05:29:26.164291+0000","last_undegraded":"2024-09-14T05:29:26.164291+0000","last_fullsized":"2024-09-14T05:29:26.164291+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:44:06.474105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060129699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.163919+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:29:26.163919+0000","last_peered":"2024-09-14T05:29:26.163919+0000","last_clean":"2024-09-14T05:29:26.163919+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:29:26.163919+0000","last_undegraded":"2024-09-14T05:29:26.163919+0000","last_fullsized":"2024-09-14T05:29:26.163919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.014725+0000","last_change":"2024-09-14T05:29:26.014725+0000","last_active":"2024-09-14T05:29:26.014725+0000","last_peered":"2024-09-14T05:29:26.014725+0000","last_clean":"2024-09-14T05:29:26.014725+0000","last_became_active":"2024-09-14T05:29:24.012099+0000","last_became_peered":"2024-09-14T05:29:24.012099+0000","last_unstale":"2024-09-14T05:29:26.014725+0000","last_undegraded":"2024-09-14T05:29:26.014725+0000","last_fullsized":"2024-09-14T05:29:26.014725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:10:42.487138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000794599,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.014085+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:29:26.014085+0000","last_peered":"2024-09-14T05:29:26.014085+0000","last_clean":"2024-09-14T05:29:26.014085+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:29:26.014085+0000","last_undegraded":"2024-09-14T05:29:26.014085+0000","last_fullsized":"2024-09-14T05:29:26.014085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.036391+0000","last_change":"2024-09-14T05:29:26.015082+0000","last_active":"2024-09-14T05:29:26.036391+0000","last_peered":"2024-09-14T05:29:26.036391+0000","last_clean":"2024-09-14T05:29:26.036391+0000","last_became_active":"2024-09-14T05:29:24.011481+0000","last_became_peered":"2024-09-14T05:29:24.011481+0000","last_unstale":"2024-09-14T05:29:26.036391+0000","last_undegraded":"2024-09-14T05:29:26.036391+0000","last_fullsized":"2024-09-14T05:29:26.036391+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:00.893195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011470110000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":25,"num_read_kb":22,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.164105+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:29:26.164105+0000","last_peered":"2024-09-14T05:29:26.164105+0000","last_clean":"2024-09-14T05:29:26.164105+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:29:26.164105+0000","last_undegraded":"2024-09-14T05:29:26.164105+0000","last_fullsized":"2024-09-14T05:29:26.164105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.014605+0000","last_change":"2024-09-14T05:29:26.014605+0000","last_active":"2024-09-14T05:29:26.014605+0000","last_peered":"2024-09-14T05:29:26.014605+0000","last_clean":"2024-09-14T05:29:26.014605+0000","last_became_active":"2024-09-14T05:29:24.011590+0000","last_became_peered":"2024-09-14T05:29:24.011590+0000","last_unstale":"2024-09-14T05:29:26.014605+0000","last_undegraded":"2024-09-14T05:29:26.014605+0000","last_fullsized":"2024-09-14T05:29:26.014605+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:44:49.685022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053582499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.018628+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:29:26.018628+0000","last_peered":"2024-09-14T05:29:26.018628+0000","last_clean":"2024-09-14T05:29:26.018628+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:29:26.018628+0000","last_undegraded":"2024-09-14T05:29:26.018628+0000","last_fullsized":"2024-09-14T05:29:26.018628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.028547+0000","last_change":"2024-09-14T05:29:26.028547+0000","last_active":"2024-09-14T05:29:26.028547+0000","last_peered":"2024-09-14T05:29:26.028547+0000","last_clean":"2024-09-14T05:29:26.028547+0000","last_became_active":"2024-09-14T05:29:24.016940+0000","last_became_peered":"2024-09-14T05:29:24.016940+0000","last_unstale":"2024-09-14T05:29:26.028547+0000","last_undegraded":"2024-09-14T05:29:26.028547+0000","last_fullsized":"2024-09-14T05:29:26.028547+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:18:01.479228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042396399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.019829+0000","last_change":"2024-09-14T05:29:26.019829+0000","last_active":"2024-09-14T05:29:26.019829+0000","last_peered":"2024-09-14T05:29:26.019829+0000","last_clean":"2024-09-14T05:29:26.019829+0000","last_became_active":"2024-09-14T05:29:24.015905+0000","last_became_peered":"2024-09-14T05:29:24.015905+0000","last_unstale":"2024-09-14T05:29:26.019829+0000","last_undegraded":"2024-09-14T05:29:26.019829+0000","last_fullsized":"2024-09-14T05:29:26.019829+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:22:08.483880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049652099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.164291+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:29:26.164291+0000","last_peered":"2024-09-14T05:29:26.164291+0000","last_clean":"2024-09-14T05:29:26.164291+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:29:26.164291+0000","last_undegraded":"2024-09-14T05:29:26.164291+0000","last_fullsized":"2024-09-14T05:29:26.164291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.019261+0000","last_change":"2024-09-14T05:29:26.019261+0000","last_active":"2024-09-14T05:29:26.019261+0000","last_peered":"2024-09-14T05:29:26.019261+0000","last_clean":"2024-09-14T05:29:26.019261+0000","last_became_active":"2024-09-14T05:29:24.011764+0000","last_became_peered":"2024-09-14T05:29:24.011764+0000","last_unstale":"2024-09-14T05:29:26.019261+0000","last_undegraded":"2024-09-14T05:29:26.019261+0000","last_fullsized":"2024-09-14T05:29:26.019261+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:28:07.933000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055502299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.163389+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:29:26.163389+0000","last_peered":"2024-09-14T05:29:26.163389+0000","last_clean":"2024-09-14T05:29:26.163389+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:29:26.163389+0000","last_undegraded":"2024-09-14T05:29:26.163389+0000","last_fullsized":"2024-09-14T05:29:26.163389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.014570+0000","last_change":"2024-09-14T05:29:26.014570+0000","last_active":"2024-09-14T05:29:26.014570+0000","last_peered":"2024-09-14T05:29:26.014570+0000","last_clean":"2024-09-14T05:29:26.014570+0000","last_became_active":"2024-09-14T05:29:24.013141+0000","last_became_peered":"2024-09-14T05:29:24.013141+0000","last_unstale":"2024-09-14T05:29:26.014570+0000","last_undegraded":"2024-09-14T05:29:26.014570+0000","last_fullsized":"2024-09-14T05:29:26.014570+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:36:33.124976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077857800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.163575+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:29:26.163575+0000","last_peered":"2024-09-14T05:29:26.163575+0000","last_clean":"2024-09-14T05:29:26.163575+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:29:26.163575+0000","last_undegraded":"2024-09-14T05:29:26.163575+0000","last_fullsized":"2024-09-14T05:29:26.163575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.015124+0000","last_change":"2024-09-14T05:29:26.015124+0000","last_active":"2024-09-14T05:29:26.015124+0000","last_peered":"2024-09-14T05:29:26.015124+0000","last_clean":"2024-09-14T05:29:26.015124+0000","last_became_active":"2024-09-14T05:29:24.015792+0000","last_became_peered":"2024-09-14T05:29:24.015792+0000","last_unstale":"2024-09-14T05:29:26.015124+0000","last_undegraded":"2024-09-14T05:29:26.015124+0000","last_fullsized":"2024-09-14T05:29:26.015124+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:26:00.237671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001049964,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.163721+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:29:26.163721+0000","last_peered":"2024-09-14T05:29:26.163721+0000","last_clean":"2024-09-14T05:29:26.163721+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:29:26.163721+0000","last_undegraded":"2024-09-14T05:29:26.163721+0000","last_fullsized":"2024-09-14T05:29:26.163721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":121,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.013872+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:29:26.013872+0000","last_peered":"2024-09-14T05:29:26.013872+0000","last_clean":"2024-09-14T05:29:26.013872+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:29:26.013872+0000","last_undegraded":"2024-09-14T05:29:26.013872+0000","last_fullsized":"2024-09-14T05:29:26.013872+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.014430+0000","last_change":"2024-09-14T05:29:26.014430+0000","last_active":"2024-09-14T05:29:26.014430+0000","last_peered":"2024-09-14T05:29:26.014430+0000","last_clean":"2024-09-14T05:29:26.014430+0000","last_became_active":"2024-09-14T05:29:24.012222+0000","last_became_peered":"2024-09-14T05:29:24.012222+0000","last_unstale":"2024-09-14T05:29:26.014430+0000","last_undegraded":"2024-09-14T05:29:26.014430+0000","last_fullsized":"2024-09-14T05:29:26.014430+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:48:42.754931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076580099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.164542+0000","last_change":"2024-09-14T05:29:26.164542+0000","last_active":"2024-09-14T05:29:26.164542+0000","last_peered":"2024-09-14T05:29:26.164542+0000","last_clean":"2024-09-14T05:29:26.164542+0000","last_became_active":"2024-09-14T05:29:24.012041+0000","last_became_peered":"2024-09-14T05:29:24.012041+0000","last_unstale":"2024-09-14T05:29:26.164542+0000","last_undegraded":"2024-09-14T05:29:26.164542+0000","last_fullsized":"2024-09-14T05:29:26.164542+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:22:04.213254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078334300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.165164+0000","last_change":"2024-09-14T05:29:26.165164+0000","last_active":"2024-09-14T05:29:26.165164+0000","last_peered":"2024-09-14T05:29:26.165164+0000","last_clean":"2024-09-14T05:29:26.165164+0000","last_became_active":"2024-09-14T05:29:24.011403+0000","last_became_peered":"2024-09-14T05:29:24.011403+0000","last_unstale":"2024-09-14T05:29:26.165164+0000","last_undegraded":"2024-09-14T05:29:26.165164+0000","last_fullsized":"2024-09-14T05:29:26.165164+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:58:20.789385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083114500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.163783+0000","last_change":"2024-09-14T05:29:26.163783+0000","last_active":"2024-09-14T05:29:26.163783+0000","last_peered":"2024-09-14T05:29:26.163783+0000","last_clean":"2024-09-14T05:29:26.163783+0000","last_became_active":"2024-09-14T05:29:24.006836+0000","last_became_peered":"2024-09-14T05:29:24.006836+0000","last_unstale":"2024-09-14T05:29:26.163783+0000","last_undegraded":"2024-09-14T05:29:26.163783+0000","last_fullsized":"2024-09-14T05:29:26.163783+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:22:35.022069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031188400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.020113+0000","last_change":"2024-09-14T05:29:26.020113+0000","last_active":"2024-09-14T05:29:26.020113+0000","last_peered":"2024-09-14T05:29:26.020113+0000","last_clean":"2024-09-14T05:29:26.020113+0000","last_became_active":"2024-09-14T05:29:24.012574+0000","last_became_peered":"2024-09-14T05:29:24.012574+0000","last_unstale":"2024-09-14T05:29:26.020113+0000","last_undegraded":"2024-09-14T05:29:26.020113+0000","last_fullsized":"2024-09-14T05:29:26.020113+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:51:10.774451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072680599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.164736+0000","last_change":"2024-09-14T05:29:26.164736+0000","last_active":"2024-09-14T05:29:26.164736+0000","last_peered":"2024-09-14T05:29:26.164736+0000","last_clean":"2024-09-14T05:29:26.164736+0000","last_became_active":"2024-09-14T05:29:24.012401+0000","last_became_peered":"2024-09-14T05:29:24.012401+0000","last_unstale":"2024-09-14T05:29:26.164736+0000","last_undegraded":"2024-09-14T05:29:26.164736+0000","last_fullsized":"2024-09-14T05:29:26.164736+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:24:04.139737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085260200000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.028811+0000","last_change":"2024-09-14T05:29:26.028811+0000","last_active":"2024-09-14T05:29:26.028811+0000","last_peered":"2024-09-14T05:29:26.028811+0000","last_clean":"2024-09-14T05:29:26.028811+0000","last_became_active":"2024-09-14T05:29:24.015315+0000","last_became_peered":"2024-09-14T05:29:24.015315+0000","last_unstale":"2024-09-14T05:29:26.028811+0000","last_undegraded":"2024-09-14T05:29:26.028811+0000","last_fullsized":"2024-09-14T05:29:26.028811+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:20:09.442666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046485199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.020027+0000","last_change":"2024-09-14T05:29:26.020027+0000","last_active":"2024-09-14T05:29:26.020027+0000","last_peered":"2024-09-14T05:29:26.020027+0000","last_clean":"2024-09-14T05:29:26.020027+0000","last_became_active":"2024-09-14T05:29:24.018854+0000","last_became_peered":"2024-09-14T05:29:24.018854+0000","last_unstale":"2024-09-14T05:29:26.020027+0000","last_undegraded":"2024-09-14T05:29:26.020027+0000","last_fullsized":"2024-09-14T05:29:26.020027+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:54.086814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063672000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.014200+0000","last_change":"2024-09-14T05:29:26.014200+0000","last_active":"2024-09-14T05:29:26.014200+0000","last_peered":"2024-09-14T05:29:26.014200+0000","last_clean":"2024-09-14T05:29:26.014200+0000","last_became_active":"2024-09-14T05:29:24.010897+0000","last_became_peered":"2024-09-14T05:29:24.010897+0000","last_unstale":"2024-09-14T05:29:26.014200+0000","last_undegraded":"2024-09-14T05:29:26.014200+0000","last_fullsized":"2024-09-14T05:29:26.014200+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:24:43.129494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051531799999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.164788+0000","last_change":"2024-09-14T05:29:26.164788+0000","last_active":"2024-09-14T05:29:26.164788+0000","last_peered":"2024-09-14T05:29:26.164788+0000","last_clean":"2024-09-14T05:29:26.164788+0000","last_became_active":"2024-09-14T05:29:24.013792+0000","last_became_peered":"2024-09-14T05:29:24.013792+0000","last_unstale":"2024-09-14T05:29:26.164788+0000","last_undegraded":"2024-09-14T05:29:26.164788+0000","last_fullsized":"2024-09-14T05:29:26.164788+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:43:12.662503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090671299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.014845+0000","last_change":"2024-09-14T05:29:26.014845+0000","last_active":"2024-09-14T05:29:26.014845+0000","last_peered":"2024-09-14T05:29:26.014845+0000","last_clean":"2024-09-14T05:29:26.014845+0000","last_became_active":"2024-09-14T05:29:24.013294+0000","last_became_peered":"2024-09-14T05:29:24.013294+0000","last_unstale":"2024-09-14T05:29:26.014845+0000","last_undegraded":"2024-09-14T05:29:26.014845+0000","last_fullsized":"2024-09-14T05:29:26.014845+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:10:33.377796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000774981,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T05:29:26.164295+0000","last_change":"2024-09-14T05:29:26.164295+0000","last_active":"2024-09-14T05:29:26.164295+0000","last_peered":"2024-09-14T05:29:26.164295+0000","last_clean":"2024-09-14T05:29:26.164295+0000","last_became_active":"2024-09-14T05:29:24.014371+0000","last_became_peered":"2024-09-14T05:29:24.014371+0000","last_unstale":"2024-09-14T05:29:26.164295+0000","last_undegraded":"2024-09-14T05:29:26.164295+0000","last_fullsized":"2024-09-14T05:29:26.164295+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:24:41.025031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050570599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476744,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":282624,"data_stored":104767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":98540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":286720,"data_stored":104786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":688916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":104767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":104786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":104797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":884736,"data_stored":695170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":35,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":30,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":65,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:29:28.433 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:28.433+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:28.462 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:28.462 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:28.470 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:29:28.501 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:28.501+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:28.574 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:28.574 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:28.582 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:29:28.601 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:28.601+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:28.685 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:28.686 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:28.694 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:29:28.703 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:28.702+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:28.797 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:28.797 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:28.803 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:28.803+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:28.806 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:29:28.905 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:28.904+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:28.908 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:28.909 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:28.917 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:29:29.005 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:29.004+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:29.020 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:29.020 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:29.029 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:29:29.034 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-14T05:29:29.097 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone1 from 0xe4b1591 (overlap 0xd14d94) to 0xbc5f6b 2024-09-14T05:29:29.106 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:29.105+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:29.142 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:29.142 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:29.151 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:29:29.164 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-14T05:29:29.182 INFO:teuthology.orchestra.run.smithi071.stdout:146 punch from 0x77c936d to 0x77d924a, (0xfedd bytes) 2024-09-14T05:29:29.186 INFO:teuthology.orchestra.run.smithi071.stdout:147 read 0x6cfb8b4 thru 0x6cfd6eb (0x1e38 bytes) 2024-09-14T05:29:29.186 INFO:teuthology.orchestra.run.smithi071.stdout:148 read 0xcc6a045 thru 0xcc6f1bd (0x5179 bytes) 2024-09-14T05:29:29.195 INFO:teuthology.orchestra.run.smithi071.stdout:149 trunc from 0xe4b1591 to 0x6958000 2024-09-14T05:29:29.206 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:29:29.207 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:29.206+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:29.215 INFO:teuthology.orchestra.run.smithi071.stdout:151 punch from 0x1f9ce10 to 0x1faa815, (0xda05 bytes) 2024-09-14T05:29:29.215 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:29.225 INFO:teuthology.orchestra.run.smithi071.stdout:153 write 0x9a4e233 thru 0x9a4eed3 (0xca1 bytes) 2024-09-14T05:29:29.228 INFO:teuthology.orchestra.run.smithi071.stdout:154 write 0x20a8aec thru 0x20ab3b7 (0x28cc bytes) 2024-09-14T05:29:29.231 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:29.231 INFO:teuthology.orchestra.run.smithi071.stdout:156 write 0x5d147e3 thru 0x5d19ef0 (0x570e bytes) 2024-09-14T05:29:29.234 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:29.234 INFO:teuthology.orchestra.run.smithi071.stdout:158 trunc from 0x9a4eed4 to 0xa0174d2 2024-09-14T05:29:29.240 INFO:teuthology.orchestra.run.smithi071.stdout:160 read 0x6e11df2 thru 0x6e1379a (0x19a9 bytes) 2024-09-14T05:29:29.240 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:29.240 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:29.241 INFO:teuthology.orchestra.run.smithi071.stdout:163 read 0x9977292 thru 0x997f490 (0x81ff bytes) 2024-09-14T05:29:29.241 INFO:teuthology.orchestra.run.smithi071.stdout:166 read 0x40db9dd thru 0x40dedd8 (0x33fc bytes) 2024-09-14T05:29:29.250 INFO:teuthology.orchestra.run.smithi071.stdout:167 writesame 0xd523f87 thru 0xd5291db data_size 0x1(0x5255 bytes) 2024-09-14T05:29:29.250 INFO:teuthology.orchestra.run.smithi071.stdout:168 read 0x886f183 thru 0x8875589 (0x6407 bytes) 2024-09-14T05:29:29.250 INFO:teuthology.orchestra.run.smithi071.stdout:169 write 0xa1712a thru 0xa20990 (0x9867 bytes) 2024-09-14T05:29:29.255 INFO:teuthology.orchestra.run.smithi071.stdout:171 trunc from 0xd5291dc to 0xdeb5f13 2024-09-14T05:29:29.259 INFO:teuthology.orchestra.run.smithi071.stdout:172 punch from 0xb092074 to 0xb093393, (0x131f bytes) 2024-09-14T05:29:29.269 INFO:teuthology.orchestra.run.smithi071.stdout:173 trunc from 0xdeb5f13 to 0x777b71c 2024-09-14T05:29:29.270 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:29.270 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:29.280 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:29:29.281 INFO:teuthology.orchestra.run.smithi071.stdout:174 read 0x2ed8ed5 thru 0x2edff1d (0x7049 bytes) 2024-09-14T05:29:29.286 INFO:teuthology.orchestra.run.smithi071.stdout:175 write 0x7a5e0bd thru 0x7a6b280 (0xd1c4 bytes) 2024-09-14T05:29:29.290 INFO:teuthology.orchestra.run.smithi071.stdout:176 write 0x177c72 thru 0x18212f (0xa4be bytes) 2024-09-14T05:29:29.295 INFO:teuthology.orchestra.run.smithi071.stdout:178 write 0x1b45775 thru 0x1b473c3 (0x1c4f bytes) 2024-09-14T05:29:29.298 INFO:teuthology.orchestra.run.smithi071.stdout:179 writesame 0x7005cbb thru 0x701186a data_size 0xb6(0xbbb0 bytes) 2024-09-14T05:29:29.307 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:29.307+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:29.313 INFO:teuthology.orchestra.run.smithi071.stdout:181 writesame 0x8fa1f51 thru 0x8fa45d4 data_size 0x11(0x2684 bytes) 2024-09-14T05:29:29.313 INFO:teuthology.orchestra.run.smithi071.stdout:182 clone 3 order 25 su 8388608 sc 6 2024-09-14T05:29:29.352 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:29:29.352 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:29:29.369 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:29:29.369 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:29:29.385 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:29:29.385 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:29:29.394 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:29.394 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:29.395 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:29.395 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:29.395 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:29.401 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:29:29.402 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:29:29.403 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:29:29.419 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:29:29.420 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:29:29.420 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:29.418+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:29.436 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:29:29.436 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:29:29.453 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:29:29.453 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:29:29.470 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:29:29.470 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:29:29.510 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:29.509+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:29.526 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:29.526 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:29.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:29.526 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:29.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.526 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.526 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:29:29.526 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.747478756, 2024-09-14T05:29:29.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.24562984099999999, 2024-09-14T05:29:29.526 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.527 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.527 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.527 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.527 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.527 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.527 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1 2024-09-14T05:29:29.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.527 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:29:29.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:29:29.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.529 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.529 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776565+0000", 2024-09-14T05:29:29.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4780000000000001e-06 2024-09-14T05:29:29.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776571+0000", 2024-09-14T05:29:29.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7669999999999998e-06 2024-09-14T05:29:29.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.531 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776580+0000", 2024-09-14T05:29:29.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9830000000000002e-06 2024-09-14T05:29:29.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776589+0000", 2024-09-14T05:29:29.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7579999999999998e-06 2024-09-14T05:29:29.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776662+0000", 2024-09-14T05:29:29.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3332999999999997e-05 2024-09-14T05:29:29.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:29:29.534 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776670+0000", 2024-09-14T05:29:29.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9850000000000006e-06 2024-09-14T05:29:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.020867+0000", 2024-09-14T05:29:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.244196265 2024-09-14T05:29:29.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.536 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021022+0000", 2024-09-14T05:29:29.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000155699 2024-09-14T05:29:29.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.537 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021169+0000", 2024-09-14T05:29:29.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014661700000000001 2024-09-14T05:29:29.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021891+0000", 2024-09-14T05:29:29.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072146700000000005 2024-09-14T05:29:29.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022124+0000", 2024-09-14T05:29:29.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023354199999999999 2024-09-14T05:29:29.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022164+0000", 2024-09-14T05:29:29.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9897000000000001e-05 2024-09-14T05:29:29.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022193+0000", 2024-09-14T05:29:29.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9050000000000001e-05 2024-09-14T05:29:29.541 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.542 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.542 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.542 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 131072~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.543 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.038382+0000", 2024-09-14T05:29:29.543 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.485660093, 2024-09-14T05:29:29.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0078079539999999998, 2024-09-14T05:29:29.543 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.543 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.544 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.544 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.544 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.544 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.544 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 22 2024-09-14T05:29:29.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.545 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038382+0000", 2024-09-14T05:29:29.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038382+0000", 2024-09-14T05:29:29.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038382+0000", 2024-09-14T05:29:29.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1399999999999997e-07 2024-09-14T05:29:29.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.548 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.548 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038439+0000", 2024-09-14T05:29:29.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6638999999999998e-05 2024-09-14T05:29:29.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038445+0000", 2024-09-14T05:29:29.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8989999999999998e-06 2024-09-14T05:29:29.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038449+0000", 2024-09-14T05:29:29.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.743e-06 2024-09-14T05:29:29.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.039586+0000", 2024-09-14T05:29:29.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0011374289999999999 2024-09-14T05:29:29.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.039672+0000", 2024-09-14T05:29:29.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5609e-05 2024-09-14T05:29:29.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.039806+0000", 2024-09-14T05:29:29.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013425000000000001 2024-09-14T05:29:29.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.041894+0000", 2024-09-14T05:29:29.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020875949999999998 2024-09-14T05:29:29.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.046094+0000", 2024-09-14T05:29:29.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042002799999999998 2024-09-14T05:29:29.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.046149+0000", 2024-09-14T05:29:29.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4950000000000001e-05 2024-09-14T05:29:29.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.046190+0000", 2024-09-14T05:29:29.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1046e-05 2024-09-14T05:29:29.557 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.557 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.557 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.558 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:23 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.558 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.038457+0000", 2024-09-14T05:29:29.558 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.485585258, 2024-09-14T05:29:29.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010089383, 2024-09-14T05:29:29.559 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.559 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.559 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.559 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.560 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.560 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.560 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23 2024-09-14T05:29:29.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.560 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038457+0000", 2024-09-14T05:29:29.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038457+0000", 2024-09-14T05:29:29.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038457+0000", 2024-09-14T05:29:29.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.75e-07 2024-09-14T05:29:29.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.564 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038509+0000", 2024-09-14T05:29:29.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2068999999999999e-05 2024-09-14T05:29:29.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.565 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038516+0000", 2024-09-14T05:29:29.565 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0929999999999997e-06 2024-09-14T05:29:29.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.566 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038521+0000", 2024-09-14T05:29:29.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4649999999999996e-06 2024-09-14T05:29:29.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.040259+0000", 2024-09-14T05:29:29.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0017385370000000001 2024-09-14T05:29:29.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.040328+0000", 2024-09-14T05:29:29.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8756000000000004e-05 2024-09-14T05:29:29.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.040419+0000", 2024-09-14T05:29:29.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0521999999999998e-05 2024-09-14T05:29:29.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.570 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.570 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.041898+0000", 2024-09-14T05:29:29.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001479015 2024-09-14T05:29:29.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.048441+0000", 2024-09-14T05:29:29.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0065434930000000001 2024-09-14T05:29:29.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.048505+0000", 2024-09-14T05:29:29.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3826999999999997e-05 2024-09-14T05:29:29.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.048546+0000", 2024-09-14T05:29:29.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1131000000000003e-05 2024-09-14T05:29:29.573 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.573 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.573 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.574 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.574 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.038528+0000", 2024-09-14T05:29:29.574 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.485514102, 2024-09-14T05:29:29.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010138525000000001, 2024-09-14T05:29:29.575 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.575 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.575 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.575 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.575 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.576 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.576 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 24 2024-09-14T05:29:29.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.576 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038528+0000", 2024-09-14T05:29:29.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038528+0000", 2024-09-14T05:29:29.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.579 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038528+0000", 2024-09-14T05:29:29.579 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.89e-07 2024-09-14T05:29:29.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.580 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038580+0000", 2024-09-14T05:29:29.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1436e-05 2024-09-14T05:29:29.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.580 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038585+0000", 2024-09-14T05:29:29.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3700000000000003e-06 2024-09-14T05:29:29.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038589+0000", 2024-09-14T05:29:29.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2339999999999996e-06 2024-09-14T05:29:29.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.583 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.583 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.040780+0000", 2024-09-14T05:29:29.583 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021907049999999998 2024-09-14T05:29:29.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.584 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.584 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.584 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.040860+0000", 2024-09-14T05:29:29.584 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.975e-05 2024-09-14T05:29:29.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.584 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.585 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.040994+0000", 2024-09-14T05:29:29.585 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013444499999999999 2024-09-14T05:29:29.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.586 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.586 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.041902+0000", 2024-09-14T05:29:29.586 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00090772499999999996 2024-09-14T05:29:29.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.048602+0000", 2024-09-14T05:29:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0067001040000000001 2024-09-14T05:29:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.048637+0000", 2024-09-14T05:29:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5255e-05 2024-09-14T05:29:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.048666+0000", 2024-09-14T05:29:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9111999999999998e-05 2024-09-14T05:29:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.038596+0000", 2024-09-14T05:29:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.485445783999999, 2024-09-14T05:29:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010202661, 2024-09-14T05:29:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 25 2024-09-14T05:29:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038596+0000", 2024-09-14T05:29:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.594 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038596+0000", 2024-09-14T05:29:29.594 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.594 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.594 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038597+0000", 2024-09-14T05:29:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.01e-07 2024-09-14T05:29:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038599+0000", 2024-09-14T05:29:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4930000000000001e-06 2024-09-14T05:29:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038603+0000", 2024-09-14T05:29:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6179999999999999e-06 2024-09-14T05:29:29.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.596 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.596 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.596 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.038606+0000", 2024-09-14T05:29:29.596 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0410000000000002e-06 2024-09-14T05:29:29.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.596 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.596 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.596 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.041421+0000", 2024-09-14T05:29:29.596 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002815031 2024-09-14T05:29:29.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.596 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.596 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.596 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.041505+0000", 2024-09-14T05:29:29.597 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4209999999999995e-05 2024-09-14T05:29:29.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.597 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.597 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.597 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.041661+0000", 2024-09-14T05:29:29.597 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000156243 2024-09-14T05:29:29.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.597 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.597 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.597 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.042187+0000", 2024-09-14T05:29:29.597 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00052539600000000002 2024-09-14T05:29:29.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.597 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.598 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.598 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.048714+0000", 2024-09-14T05:29:29.598 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0065274549999999997 2024-09-14T05:29:29.598 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.598 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.598 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.598 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.048765+0000", 2024-09-14T05:29:29.598 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1269e-05 2024-09-14T05:29:29.598 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.598 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.598 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.598 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.048799+0000", 2024-09-14T05:29:29.598 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3504000000000003e-05 2024-09-14T05:29:29.599 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.599 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.599 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.599 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.599 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.599 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.599 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237163+0000", 2024-09-14T05:29:29.599 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.286878818000002, 2024-09-14T05:29:29.599 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064289880000000001, 2024-09-14T05:29:29.599 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.599 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.599 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.599 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.600 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.600 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.600 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 42 2024-09-14T05:29:29.600 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.600 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.600 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.600 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.600 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237163+0000", 2024-09-14T05:29:29.600 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.600 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.600 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.600 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.600 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237163+0000", 2024-09-14T05:29:29.601 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.601 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.601 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.601 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.601 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237164+0000", 2024-09-14T05:29:29.601 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3599999999999999e-07 2024-09-14T05:29:29.601 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.601 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.601 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.601 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237166+0000", 2024-09-14T05:29:29.601 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2689999999999998e-06 2024-09-14T05:29:29.601 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.601 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.601 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.602 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237172+0000", 2024-09-14T05:29:29.602 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0990000000000004e-06 2024-09-14T05:29:29.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.602 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.602 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.602 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237176+0000", 2024-09-14T05:29:29.602 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8859999999999997e-06 2024-09-14T05:29:29.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.602 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.602 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.602 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.238652+0000", 2024-09-14T05:29:29.602 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001476125 2024-09-14T05:29:29.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.602 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.603 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.603 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.238733+0000", 2024-09-14T05:29:29.603 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1456000000000001e-05 2024-09-14T05:29:29.603 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.603 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.603 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.603 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.238857+0000", 2024-09-14T05:29:29.603 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012333799999999999 2024-09-14T05:29:29.603 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.603 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.603 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.603 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.240901+0000", 2024-09-14T05:29:29.603 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020440409999999999 2024-09-14T05:29:29.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.604 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.604 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.604 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243502+0000", 2024-09-14T05:29:29.604 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0026007869999999998 2024-09-14T05:29:29.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.604 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.604 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.604 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243552+0000", 2024-09-14T05:29:29.604 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0302000000000001e-05 2024-09-14T05:29:29.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.604 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.604 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.605 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243592+0000", 2024-09-14T05:29:29.605 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0249e-05 2024-09-14T05:29:29.605 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.605 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.605 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.605 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.605 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237459+0000", 2024-09-14T05:29:29.605 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.286582460999998, 2024-09-14T05:29:29.605 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064058400000000003, 2024-09-14T05:29:29.605 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.605 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.605 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.606 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.606 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.606 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.606 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 44 2024-09-14T05:29:29.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.606 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.606 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237459+0000", 2024-09-14T05:29:29.606 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.606 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.607 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237459+0000", 2024-09-14T05:29:29.607 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.607 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.607 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237460+0000", 2024-09-14T05:29:29.607 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0100000000000005e-07 2024-09-14T05:29:29.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.607 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.607 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237500+0000", 2024-09-14T05:29:29.607 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9549999999999999e-05 2024-09-14T05:29:29.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.607 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.608 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237507+0000", 2024-09-14T05:29:29.608 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3610000000000003e-06 2024-09-14T05:29:29.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.608 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.608 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.608 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237512+0000", 2024-09-14T05:29:29.608 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4450000000000004e-06 2024-09-14T05:29:29.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.608 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.608 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.608 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.239830+0000", 2024-09-14T05:29:29.608 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023173249999999999 2024-09-14T05:29:29.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.239872+0000", 2024-09-14T05:29:29.609 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2101999999999998e-05 2024-09-14T05:29:29.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.239959+0000", 2024-09-14T05:29:29.609 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6997999999999996e-05 2024-09-14T05:29:29.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242053+0000", 2024-09-14T05:29:29.610 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020942220000000002 2024-09-14T05:29:29.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.610 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.610 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.610 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243781+0000", 2024-09-14T05:29:29.610 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001728388 2024-09-14T05:29:29.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.610 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.610 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.610 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243831+0000", 2024-09-14T05:29:29.610 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9552000000000003e-05 2024-09-14T05:29:29.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.610 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.610 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.611 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243865+0000", 2024-09-14T05:29:29.611 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4396000000000001e-05 2024-09-14T05:29:29.611 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.611 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.611 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.611 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.611 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237547+0000", 2024-09-14T05:29:29.611 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.286495380000002, 2024-09-14T05:29:29.611 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0076309530000000002, 2024-09-14T05:29:29.611 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.612 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.612 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.612 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.612 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.612 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.612 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 45 2024-09-14T05:29:29.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.612 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.613 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.613 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237547+0000", 2024-09-14T05:29:29.613 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.613 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.613 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.613 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.613 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237547+0000", 2024-09-14T05:29:29.613 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.614 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.614 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.614 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237547+0000", 2024-09-14T05:29:29.614 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0200000000000002e-07 2024-09-14T05:29:29.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.614 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.614 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.614 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237551+0000", 2024-09-14T05:29:29.614 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8229999999999997e-06 2024-09-14T05:29:29.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.614 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.614 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.614 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237563+0000", 2024-09-14T05:29:29.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2041999999999999e-05 2024-09-14T05:29:29.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.615 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237568+0000", 2024-09-14T05:29:29.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4709999999999998e-06 2024-09-14T05:29:29.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.615 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.240349+0000", 2024-09-14T05:29:29.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027806060000000001 2024-09-14T05:29:29.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.616 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.240423+0000", 2024-09-14T05:29:29.616 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4197999999999997e-05 2024-09-14T05:29:29.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.616 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.616 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.240546+0000", 2024-09-14T05:29:29.616 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012227399999999999 2024-09-14T05:29:29.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.616 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.616 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242951+0000", 2024-09-14T05:29:29.616 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002405866 2024-09-14T05:29:29.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.617 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.617 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.245091+0000", 2024-09-14T05:29:29.617 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0021393829999999999 2024-09-14T05:29:29.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.617 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.617 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.245142+0000", 2024-09-14T05:29:29.617 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0782999999999998e-05 2024-09-14T05:29:29.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.617 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.617 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.245177+0000", 2024-09-14T05:29:29.618 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5905000000000003e-05 2024-09-14T05:29:29.618 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.618 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.618 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.618 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.618 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237579+0000", 2024-09-14T05:29:29.618 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.286462883999999, 2024-09-14T05:29:29.618 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010719608, 2024-09-14T05:29:29.618 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.618 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.618 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.618 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.619 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.619 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.619 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 46 2024-09-14T05:29:29.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.619 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.619 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.619 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237579+0000", 2024-09-14T05:29:29.619 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.619 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.619 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237579+0000", 2024-09-14T05:29:29.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.620 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.620 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237580+0000", 2024-09-14T05:29:29.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3499999999999996e-07 2024-09-14T05:29:29.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.620 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.620 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237583+0000", 2024-09-14T05:29:29.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.895e-06 2024-09-14T05:29:29.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237591+0000", 2024-09-14T05:29:29.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.17e-06 2024-09-14T05:29:29.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237598+0000", 2024-09-14T05:29:29.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0119999999999999e-06 2024-09-14T05:29:29.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.240949+0000", 2024-09-14T05:29:29.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033510929999999999 2024-09-14T05:29:29.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.622 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.622 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.241002+0000", 2024-09-14T05:29:29.622 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2837999999999998e-05 2024-09-14T05:29:29.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.622 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.622 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.241086+0000", 2024-09-14T05:29:29.622 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4530999999999994e-05 2024-09-14T05:29:29.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.622 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.622 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242957+0000", 2024-09-14T05:29:29.623 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001871239 2024-09-14T05:29:29.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.623 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.623 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.623 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248141+0000", 2024-09-14T05:29:29.623 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0051832040000000003 2024-09-14T05:29:29.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.623 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.623 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.623 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248262+0000", 2024-09-14T05:29:29.623 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000121884 2024-09-14T05:29:29.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.623 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.623 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.624 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248299+0000", 2024-09-14T05:29:29.624 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6207000000000001e-05 2024-09-14T05:29:29.624 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.624 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.624 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.624 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.624 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237608+0000", 2024-09-14T05:29:29.624 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.286434330999999, 2024-09-14T05:29:29.624 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010804924, 2024-09-14T05:29:29.624 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.625 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.625 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.625 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.625 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.625 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.625 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 47 2024-09-14T05:29:29.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.625 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.625 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.625 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237608+0000", 2024-09-14T05:29:29.625 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237608+0000", 2024-09-14T05:29:29.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237608+0000", 2024-09-14T05:29:29.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6499999999999999e-07 2024-09-14T05:29:29.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237654+0000", 2024-09-14T05:29:29.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6326000000000003e-05 2024-09-14T05:29:29.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237665+0000", 2024-09-14T05:29:29.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0056e-05 2024-09-14T05:29:29.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237671+0000", 2024-09-14T05:29:29.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9819999999999996e-06 2024-09-14T05:29:29.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.241343+0000", 2024-09-14T05:29:29.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036724610000000001 2024-09-14T05:29:29.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.628 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.628 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.241419+0000", 2024-09-14T05:29:29.628 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6089999999999998e-05 2024-09-14T05:29:29.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.628 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.628 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.241540+0000", 2024-09-14T05:29:29.628 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000120493 2024-09-14T05:29:29.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.628 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.629 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243603+0000", 2024-09-14T05:29:29.629 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020638649999999998 2024-09-14T05:29:29.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.629 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.629 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248351+0000", 2024-09-14T05:29:29.629 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0047479030000000004 2024-09-14T05:29:29.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.629 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.629 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248381+0000", 2024-09-14T05:29:29.629 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0136000000000001e-05 2024-09-14T05:29:29.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.630 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.630 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248413+0000", 2024-09-14T05:29:29.630 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1047000000000003e-05 2024-09-14T05:29:29.630 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.630 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.630 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.630 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.630 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237682+0000", 2024-09-14T05:29:29.630 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.286359467, 2024-09-14T05:29:29.630 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010869477000000001, 2024-09-14T05:29:29.630 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.631 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.631 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.631 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.631 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.631 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.631 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 48 2024-09-14T05:29:29.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.631 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.631 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.631 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237682+0000", 2024-09-14T05:29:29.631 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.632 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.632 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237682+0000", 2024-09-14T05:29:29.632 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.632 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.632 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237683+0000", 2024-09-14T05:29:29.632 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8400000000000004e-07 2024-09-14T05:29:29.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.632 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.632 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237688+0000", 2024-09-14T05:29:29.632 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5519999999999998e-06 2024-09-14T05:29:29.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237696+0000", 2024-09-14T05:29:29.633 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4689999999999997e-06 2024-09-14T05:29:29.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237701+0000", 2024-09-14T05:29:29.633 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5950000000000001e-06 2024-09-14T05:29:29.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242083+0000", 2024-09-14T05:29:29.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004382782 2024-09-14T05:29:29.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.634 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.634 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242132+0000", 2024-09-14T05:29:29.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8171999999999999e-05 2024-09-14T05:29:29.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.634 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.634 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242205+0000", 2024-09-14T05:29:29.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3165999999999997e-05 2024-09-14T05:29:29.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.634 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.635 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243610+0000", 2024-09-14T05:29:29.635 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001405199 2024-09-14T05:29:29.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.635 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.635 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248464+0000", 2024-09-14T05:29:29.635 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048539079999999997 2024-09-14T05:29:29.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.635 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.635 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248514+0000", 2024-09-14T05:29:29.635 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9639999999999999e-05 2024-09-14T05:29:29.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.636 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.636 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248552+0000", 2024-09-14T05:29:29.636 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.841e-05 2024-09-14T05:29:29.636 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.636 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.636 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.636 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.636 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.636 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237709+0000", 2024-09-14T05:29:29.636 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.286333394, 2024-09-14T05:29:29.636 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010940642, 2024-09-14T05:29:29.636 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.637 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.637 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.637 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.637 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.637 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.637 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 49 2024-09-14T05:29:29.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.637 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.637 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.637 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237709+0000", 2024-09-14T05:29:29.637 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.638 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.638 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237709+0000", 2024-09-14T05:29:29.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.638 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.638 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237709+0000", 2024-09-14T05:29:29.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.27e-07 2024-09-14T05:29:29.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.638 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.638 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237712+0000", 2024-09-14T05:29:29.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3299999999999999e-06 2024-09-14T05:29:29.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237717+0000", 2024-09-14T05:29:29.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8890000000000001e-06 2024-09-14T05:29:29.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237720+0000", 2024-09-14T05:29:29.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.881e-06 2024-09-14T05:29:29.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242470+0000", 2024-09-14T05:29:29.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0047503780000000004 2024-09-14T05:29:29.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.640 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242549+0000", 2024-09-14T05:29:29.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8564000000000002e-05 2024-09-14T05:29:29.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.640 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242672+0000", 2024-09-14T05:29:29.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122769 2024-09-14T05:29:29.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.641 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.641 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243876+0000", 2024-09-14T05:29:29.641 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0012039119999999999 2024-09-14T05:29:29.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.641 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.641 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248592+0000", 2024-09-14T05:29:29.641 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0047165619999999997 2024-09-14T05:29:29.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.641 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.641 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248620+0000", 2024-09-14T05:29:29.641 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7591999999999999e-05 2024-09-14T05:29:29.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.642 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.642 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.642 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248649+0000", 2024-09-14T05:29:29.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9337999999999999e-05 2024-09-14T05:29:29.642 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.642 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.642 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.642 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.642 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.642 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237731+0000", 2024-09-14T05:29:29.642 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.286311354999999, 2024-09-14T05:29:29.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.011041749, 2024-09-14T05:29:29.642 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.643 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.643 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.643 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.643 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.643 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.643 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 50 2024-09-14T05:29:29.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.643 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.643 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.643 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237731+0000", 2024-09-14T05:29:29.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237731+0000", 2024-09-14T05:29:29.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237731+0000", 2024-09-14T05:29:29.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1699999999999998e-07 2024-09-14T05:29:29.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237790+0000", 2024-09-14T05:29:29.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8440000000000003e-05 2024-09-14T05:29:29.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237800+0000", 2024-09-14T05:29:29.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0431000000000001e-05 2024-09-14T05:29:29.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237806+0000", 2024-09-14T05:29:29.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8889999999999999e-06 2024-09-14T05:29:29.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.242992+0000", 2024-09-14T05:29:29.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0051858349999999998 2024-09-14T05:29:29.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.646 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243039+0000", 2024-09-14T05:29:29.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7574000000000002e-05 2024-09-14T05:29:29.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.646 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243118+0000", 2024-09-14T05:29:29.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8409000000000001e-05 2024-09-14T05:29:29.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.244673+0000", 2024-09-14T05:29:29.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015554189999999999 2024-09-14T05:29:29.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.647 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248687+0000", 2024-09-14T05:29:29.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0040139670000000002 2024-09-14T05:29:29.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.647 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248736+0000", 2024-09-14T05:29:29.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9168000000000001e-05 2024-09-14T05:29:29.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.648 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.648 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248772+0000", 2024-09-14T05:29:29.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000003e-05 2024-09-14T05:29:29.648 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.648 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.648 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.648 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.648 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237820+0000", 2024-09-14T05:29:29.648 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.286222369000001, 2024-09-14T05:29:29.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.011040212000000001, 2024-09-14T05:29:29.648 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.649 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.649 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.649 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.649 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.649 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.649 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 51 2024-09-14T05:29:29.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.649 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.649 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.649 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237820+0000", 2024-09-14T05:29:29.649 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237820+0000", 2024-09-14T05:29:29.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237820+0000", 2024-09-14T05:29:29.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0500000000000003e-07 2024-09-14T05:29:29.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237824+0000", 2024-09-14T05:29:29.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9890000000000003e-06 2024-09-14T05:29:29.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.651 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.651 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237830+0000", 2024-09-14T05:29:29.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1179999999999997e-06 2024-09-14T05:29:29.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.651 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.651 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237835+0000", 2024-09-14T05:29:29.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3420000000000001e-06 2024-09-14T05:29:29.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.651 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243906+0000", 2024-09-14T05:29:29.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0060709819999999999 2024-09-14T05:29:29.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.652 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.243955+0000", 2024-09-14T05:29:29.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9431000000000002e-05 2024-09-14T05:29:29.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.652 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.244031+0000", 2024-09-14T05:29:29.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5765000000000004e-05 2024-09-14T05:29:29.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.653 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.653 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.244680+0000", 2024-09-14T05:29:29.653 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00064928100000000003 2024-09-14T05:29:29.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.653 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.653 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248801+0000", 2024-09-14T05:29:29.653 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0041206020000000001 2024-09-14T05:29:29.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.653 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.653 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248829+0000", 2024-09-14T05:29:29.654 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7971999999999999e-05 2024-09-14T05:29:29.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.654 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.654 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.654 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.248860+0000", 2024-09-14T05:29:29.654 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1124999999999998e-05 2024-09-14T05:29:29.654 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.654 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.654 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.654 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.654 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.654 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.237844+0000", 2024-09-14T05:29:29.655 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.286198398, 2024-09-14T05:29:29.655 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.012929487999999999, 2024-09-14T05:29:29.655 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.655 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.655 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.655 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.655 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.655 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.655 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 52 2024-09-14T05:29:29.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.655 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.655 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.655 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.655 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237844+0000", 2024-09-14T05:29:29.656 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.656 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.656 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237844+0000", 2024-09-14T05:29:29.656 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.656 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.656 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237844+0000", 2024-09-14T05:29:29.656 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3099999999999998e-07 2024-09-14T05:29:29.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.656 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.657 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237848+0000", 2024-09-14T05:29:29.657 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2220000000000004e-06 2024-09-14T05:29:29.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.657 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.657 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.657 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237853+0000", 2024-09-14T05:29:29.657 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1760000000000001e-06 2024-09-14T05:29:29.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.657 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.657 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.657 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.237858+0000", 2024-09-14T05:29:29.657 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8450000000000002e-06 2024-09-14T05:29:29.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.657 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.657 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.658 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.244269+0000", 2024-09-14T05:29:29.658 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0064110970000000001 2024-09-14T05:29:29.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.658 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.658 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.658 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.244345+0000", 2024-09-14T05:29:29.658 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6013999999999994e-05 2024-09-14T05:29:29.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.658 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.658 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.658 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.244472+0000", 2024-09-14T05:29:29.658 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012630200000000001 2024-09-14T05:29:29.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.658 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.658 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.659 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.244871+0000", 2024-09-14T05:29:29.659 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00039959099999999998 2024-09-14T05:29:29.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.659 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.659 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.659 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.250638+0000", 2024-09-14T05:29:29.659 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0057671759999999997 2024-09-14T05:29:29.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.659 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.659 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.659 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.250713+0000", 2024-09-14T05:29:29.659 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4762e-05 2024-09-14T05:29:29.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.660 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.660 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.660 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.250773+0000", 2024-09-14T05:29:29.660 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9772000000000003e-05 2024-09-14T05:29:29.660 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.660 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.660 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.660 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.660 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.660 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.527756+0000", 2024-09-14T05:29:29.660 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 20.996285558, 2024-09-14T05:29:29.660 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0071073860000000003, 2024-09-14T05:29:29.660 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.661 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.661 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.661 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.661 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.661 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.661 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 72 2024-09-14T05:29:29.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.661 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.661 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.661 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527756+0000", 2024-09-14T05:29:29.661 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.662 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.662 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527756+0000", 2024-09-14T05:29:29.662 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.662 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.662 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527757+0000", 2024-09-14T05:29:29.662 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2700000000000001e-07 2024-09-14T05:29:29.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.662 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.662 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527797+0000", 2024-09-14T05:29:29.663 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9915e-05 2024-09-14T05:29:29.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.663 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.663 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527801+0000", 2024-09-14T05:29:29.663 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0239999999999999e-06 2024-09-14T05:29:29.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.663 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.663 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527805+0000", 2024-09-14T05:29:29.663 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0559999999999998e-06 2024-09-14T05:29:29.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.663 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.664 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.532244+0000", 2024-09-14T05:29:29.664 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0044392149999999998 2024-09-14T05:29:29.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.664 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.664 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.532293+0000", 2024-09-14T05:29:29.664 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8971999999999998e-05 2024-09-14T05:29:29.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.664 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.664 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.532359+0000", 2024-09-14T05:29:29.664 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5786999999999999e-05 2024-09-14T05:29:29.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.665 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.665 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.533307+0000", 2024-09-14T05:29:29.665 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00094854300000000004 2024-09-14T05:29:29.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.665 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.665 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.534813+0000", 2024-09-14T05:29:29.665 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015063699999999999 2024-09-14T05:29:29.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.665 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.665 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.534841+0000", 2024-09-14T05:29:29.666 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7412000000000001e-05 2024-09-14T05:29:29.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.666 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.666 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.534864+0000", 2024-09-14T05:29:29.666 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2864999999999999e-05 2024-09-14T05:29:29.666 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.666 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.666 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.666 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.666 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.527810+0000", 2024-09-14T05:29:29.666 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 20.996232208999999, 2024-09-14T05:29:29.666 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0072560050000000003, 2024-09-14T05:29:29.667 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.667 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.667 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.667 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.667 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.667 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.667 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73 2024-09-14T05:29:29.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.667 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.667 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.667 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527810+0000", 2024-09-14T05:29:29.667 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.668 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.668 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.668 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527810+0000", 2024-09-14T05:29:29.668 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.668 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.668 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.668 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527810+0000", 2024-09-14T05:29:29.668 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1600000000000002e-07 2024-09-14T05:29:29.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.668 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.668 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.668 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527812+0000", 2024-09-14T05:29:29.669 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0870000000000001e-06 2024-09-14T05:29:29.669 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.669 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.669 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.669 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527815+0000", 2024-09-14T05:29:29.669 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.219e-06 2024-09-14T05:29:29.669 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.669 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.669 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.669 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527819+0000", 2024-09-14T05:29:29.669 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.783e-06 2024-09-14T05:29:29.669 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.669 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.669 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.532774+0000", 2024-09-14T05:29:29.670 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0049551150000000004 2024-09-14T05:29:29.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.670 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.532815+0000", 2024-09-14T05:29:29.670 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0268999999999997e-05 2024-09-14T05:29:29.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.670 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.532876+0000", 2024-09-14T05:29:29.670 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1643999999999995e-05 2024-09-14T05:29:29.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.671 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.671 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.671 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.534907+0000", 2024-09-14T05:29:29.671 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0020311180000000002 2024-09-14T05:29:29.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.671 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.671 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.671 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.534979+0000", 2024-09-14T05:29:29.671 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1422999999999997e-05 2024-09-14T05:29:29.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.671 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.671 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.671 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535027+0000", 2024-09-14T05:29:29.671 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8401e-05 2024-09-14T05:29:29.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.672 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.672 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535066+0000", 2024-09-14T05:29:29.672 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8630000000000001e-05 2024-09-14T05:29:29.672 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.672 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.672 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.672 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.672 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.527837+0000", 2024-09-14T05:29:29.673 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 20.996204793, 2024-09-14T05:29:29.673 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.007313127, 2024-09-14T05:29:29.673 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.673 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.673 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.673 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.673 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.673 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.673 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74 2024-09-14T05:29:29.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.673 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.673 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.673 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.673 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527837+0000", 2024-09-14T05:29:29.674 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.674 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.674 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527837+0000", 2024-09-14T05:29:29.674 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.674 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.674 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527837+0000", 2024-09-14T05:29:29.674 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3599999999999999e-07 2024-09-14T05:29:29.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.675 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.675 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527841+0000", 2024-09-14T05:29:29.675 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7179999999999998e-06 2024-09-14T05:29:29.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.675 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.675 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527847+0000", 2024-09-14T05:29:29.675 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3310000000000003e-06 2024-09-14T05:29:29.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.675 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.675 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527850+0000", 2024-09-14T05:29:29.675 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2550000000000002e-06 2024-09-14T05:29:29.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.676 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.676 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.676 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.533411+0000", 2024-09-14T05:29:29.676 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.005561646 2024-09-14T05:29:29.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.676 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.676 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.676 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.533455+0000", 2024-09-14T05:29:29.676 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3476999999999997e-05 2024-09-14T05:29:29.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.676 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.676 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.676 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.533521+0000", 2024-09-14T05:29:29.677 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6482999999999993e-05 2024-09-14T05:29:29.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.677 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.677 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.534963+0000", 2024-09-14T05:29:29.677 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001442079 2024-09-14T05:29:29.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.677 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.677 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535074+0000", 2024-09-14T05:29:29.677 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000110454 2024-09-14T05:29:29.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.677 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.678 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535115+0000", 2024-09-14T05:29:29.678 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1430000000000001e-05 2024-09-14T05:29:29.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.678 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.678 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.678 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535150+0000", 2024-09-14T05:29:29.678 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4918e-05 2024-09-14T05:29:29.678 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.678 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.678 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.678 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.678 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.679 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.527884+0000", 2024-09-14T05:29:29.679 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 20.996157529000001, 2024-09-14T05:29:29.679 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0077308359999999996, 2024-09-14T05:29:29.679 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.679 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.679 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.679 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.679 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.679 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.679 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 75 2024-09-14T05:29:29.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.679 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.679 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.680 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527884+0000", 2024-09-14T05:29:29.680 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.680 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.680 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.680 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527884+0000", 2024-09-14T05:29:29.680 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.680 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.680 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.680 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527885+0000", 2024-09-14T05:29:29.680 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3099999999999998e-07 2024-09-14T05:29:29.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.680 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.681 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.681 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527925+0000", 2024-09-14T05:29:29.681 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9823000000000001e-05 2024-09-14T05:29:29.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.681 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.681 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527931+0000", 2024-09-14T05:29:29.681 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8159999999999996e-06 2024-09-14T05:29:29.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.681 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.681 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527935+0000", 2024-09-14T05:29:29.682 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.213e-06 2024-09-14T05:29:29.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.682 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.682 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.682 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.533763+0000", 2024-09-14T05:29:29.682 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0058281449999999999 2024-09-14T05:29:29.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.682 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.682 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.682 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.533806+0000", 2024-09-14T05:29:29.682 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3523999999999999e-05 2024-09-14T05:29:29.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.682 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.682 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.683 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.533903+0000", 2024-09-14T05:29:29.683 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6849999999999996e-05 2024-09-14T05:29:29.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.683 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.683 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.683 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535518+0000", 2024-09-14T05:29:29.683 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0016150959999999999 2024-09-14T05:29:29.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.683 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.683 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.683 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535533+0000", 2024-09-14T05:29:29.683 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5003e-05 2024-09-14T05:29:29.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.684 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.684 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.684 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535576+0000", 2024-09-14T05:29:29.684 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2540000000000003e-05 2024-09-14T05:29:29.684 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.684 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.684 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.684 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535615+0000", 2024-09-14T05:29:29.684 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9394999999999998e-05 2024-09-14T05:29:29.684 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.684 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.684 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.684 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.685 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.685 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:29.685 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:08.527940+0000", 2024-09-14T05:29:29.685 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 20.996101738, 2024-09-14T05:29:29.685 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010608569999999999, 2024-09-14T05:29:29.685 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:29.685 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:29.685 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:29.685 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:29.685 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:29.685 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:29.685 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 76 2024-09-14T05:29:29.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.685 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:29.686 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.686 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:29.686 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527940+0000", 2024-09-14T05:29:29.686 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.686 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.686 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:29.686 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527940+0000", 2024-09-14T05:29:29.686 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:29.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.686 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.686 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:29.686 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527940+0000", 2024-09-14T05:29:29.687 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0400000000000002e-07 2024-09-14T05:29:29.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.687 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.687 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:29.687 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527944+0000", 2024-09-14T05:29:29.687 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5700000000000001e-06 2024-09-14T05:29:29.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.687 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.687 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:29.687 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527948+0000", 2024-09-14T05:29:29.687 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9029999999999997e-06 2024-09-14T05:29:29.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.687 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.687 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:29.688 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.527951+0000", 2024-09-14T05:29:29.688 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6309999999999999e-06 2024-09-14T05:29:29.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.688 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:29.688 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535166+0000", 2024-09-14T05:29:29.688 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0072155479999999996 2024-09-14T05:29:29.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.688 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:29.688 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535211+0000", 2024-09-14T05:29:29.688 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5108999999999998e-05 2024-09-14T05:29:29.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.689 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:29.689 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.535282+0000", 2024-09-14T05:29:29.689 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0364999999999995e-05 2024-09-14T05:29:29.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.689 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:29.689 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.536073+0000", 2024-09-14T05:29:29.689 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00079145699999999997 2024-09-14T05:29:29.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.689 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:29.689 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.538464+0000", 2024-09-14T05:29:29.689 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023912429999999999 2024-09-14T05:29:29.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.690 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.690 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:29.690 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.538511+0000", 2024-09-14T05:29:29.690 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6328000000000001e-05 2024-09-14T05:29:29.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:29.690 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:29.690 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:29.690 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.538549+0000", 2024-09-14T05:29:29.690 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8111999999999997e-05 2024-09-14T05:29:29.690 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.690 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.690 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.690 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:29.691 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:29.691 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:29.692 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:29:29.693 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:29.692+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:29.792 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:29.792+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:29.836 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:29.836 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:29.836 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:29:29.836 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:29.845 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:29:29.893 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:29.893+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:29.946 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:29.947 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:29.947 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:29.947 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:29.947 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:29.955 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:29:29.994 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:29.994+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:30.062 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:30.062 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:30.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:30.062 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:30.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.062 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4235.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-14T05:29:30.063 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:13.836551+0000", 2024-09-14T05:29:30.063 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 16.225543511000001, 2024-09-14T05:29:30.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00065322999999999998, 2024-09-14T05:29:30.063 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.063 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.063 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:30.064 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.836551+0000", 2024-09-14T05:29:30.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.836551+0000", 2024-09-14T05:29:30.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.836551+0000", 2024-09-14T05:29:30.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9400000000000001e-07 2024-09-14T05:29:30.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.836559+0000", 2024-09-14T05:29:30.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1629999999999996e-06 2024-09-14T05:29:30.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.836573+0000", 2024-09-14T05:29:30.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3397000000000001e-05 2024-09-14T05:29:30.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.836584+0000", 2024-09-14T05:29:30.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1484000000000001e-05 2024-09-14T05:29:30.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.836639+0000", 2024-09-14T05:29:30.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5031999999999998e-05 2024-09-14T05:29:30.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.067 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.067 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.836666+0000", 2024-09-14T05:29:30.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6588e-05 2024-09-14T05:29:30.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.067 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:30.067 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.837165+0000", 2024-09-14T05:29:30.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00049923800000000005 2024-09-14T05:29:30.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.837204+0000", 2024-09-14T05:29:30.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8933999999999997e-05 2024-09-14T05:29:30.068 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.068 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.068 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.068 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:13 3.1e e27/24 3:7a918c66:::rbd_object_map.112d8b048597:head v 27'1, mlcod=0'0)", 2024-09-14T05:29:30.068 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:26.026611+0000", 2024-09-14T05:29:30.069 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.0354833579999996, 2024-09-14T05:29:30.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00088820700000000002, 2024-09-14T05:29:30.069 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.069 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.069 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:30.069 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.069 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.069 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.026611+0000", 2024-09-14T05:29:30.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.069 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.026611+0000", 2024-09-14T05:29:30.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.026611+0000", 2024-09-14T05:29:30.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2e-07 2024-09-14T05:29:30.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.026617+0000", 2024-09-14T05:29:30.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5910000000000003e-06 2024-09-14T05:29:30.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.026637+0000", 2024-09-14T05:29:30.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.95e-05 2024-09-14T05:29:30.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.026651+0000", 2024-09-14T05:29:30.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.418e-05 2024-09-14T05:29:30.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.026721+0000", 2024-09-14T05:29:30.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0508999999999999e-05 2024-09-14T05:29:30.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.026745+0000", 2024-09-14T05:29:30.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3305000000000001e-05 2024-09-14T05:29:30.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:30.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.027452+0000", 2024-09-14T05:29:30.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00070784299999999999 2024-09-14T05:29:30.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.073 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.073 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:26.027499+0000", 2024-09-14T05:29:30.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6859e-05 2024-09-14T05:29:30.073 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.073 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.073 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.073 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.073 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.074 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:30.075 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:29:30.117 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone2 from 0x8fa45d5 (overlap 0x6958000) to 0xa88ccc 2024-09-14T05:29:30.118 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:30.117+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:30.196 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:30.195+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:30.203 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:30.203 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:30.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.203 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1199 3.13 3:ce6c3324:::rbd_data.112dd25fb4c.0000000000000085:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.204 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.194776+0000", 2024-09-14T05:29:30.204 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.00703377, 2024-09-14T05:29:30.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0070515350000000003, 2024-09-14T05:29:30.204 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.204 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.204 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:30.204 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.205 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.205 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.205 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1199 2024-09-14T05:29:30.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.205 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194776+0000", 2024-09-14T05:29:30.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194776+0000", 2024-09-14T05:29:30.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.206 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.206 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194777+0000", 2024-09-14T05:29:30.206 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0729999999999999e-06 2024-09-14T05:29:30.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.206 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.206 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194781+0000", 2024-09-14T05:29:30.206 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.675e-06 2024-09-14T05:29:30.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.206 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.206 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194792+0000", 2024-09-14T05:29:30.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1596e-05 2024-09-14T05:29:30.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194802+0000", 2024-09-14T05:29:30.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9769999999999995e-06 2024-09-14T05:29:30.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194857+0000", 2024-09-14T05:29:30.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4698999999999999e-05 2024-09-14T05:29:30.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194973+0000", 2024-09-14T05:29:30.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011576 2024-09-14T05:29:30.208 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.208 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.208 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.208 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:1198 3.16 e30/24 3:692cbb64:::rbd_data.112dd25fb4c.000000000000007e:head v 30'17, mlcod=30'15)", 2024-09-14T05:29:30.209 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.195106+0000", 2024-09-14T05:29:30.209 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0067036350000000003, 2024-09-14T05:29:30.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.006763921, 2024-09-14T05:29:30.209 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.209 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.209 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:30.209 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.195106+0000", 2024-09-14T05:29:30.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.210 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.195106+0000", 2024-09-14T05:29:30.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.210 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.210 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.195107+0000", 2024-09-14T05:29:30.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0100000000000005e-07 2024-09-14T05:29:30.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.210 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.210 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.195111+0000", 2024-09-14T05:29:30.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0589999999999996e-06 2024-09-14T05:29:30.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.195118+0000", 2024-09-14T05:29:30.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4070000000000004e-06 2024-09-14T05:29:30.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.195122+0000", 2024-09-14T05:29:30.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2629999999999997e-06 2024-09-14T05:29:30.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.195163+0000", 2024-09-14T05:29:30.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0868000000000002e-05 2024-09-14T05:29:30.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.195189+0000", 2024-09-14T05:29:30.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6173000000000002e-05 2024-09-14T05:29:30.212 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.212 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.212 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.212 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1200 3.d 3:b1bffb8d:::rbd_data.112dd25fb4c.0000000000000080:head [call rbd.sparse_copyup in=24b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.212 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.195387+0000", 2024-09-14T05:29:30.213 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0064231360000000003, 2024-09-14T05:29:30.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0065229909999999997, 2024-09-14T05:29:30.213 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.213 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.213 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "waiting for sub ops", 2024-09-14T05:29:30.213 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.213 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.213 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.213 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1200 2024-09-14T05:29:30.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.213 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.195387+0000", 2024-09-14T05:29:30.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.195387+0000", 2024-09-14T05:29:30.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.195388+0000", 2024-09-14T05:29:30.215 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5799999999999998e-07 2024-09-14T05:29:30.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.215 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.215 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.195393+0000", 2024-09-14T05:29:30.215 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7880000000000003e-06 2024-09-14T05:29:30.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.215 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.195406+0000", 2024-09-14T05:29:30.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2242000000000001e-05 2024-09-14T05:29:30.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.195415+0000", 2024-09-14T05:29:30.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0000000000000002e-06 2024-09-14T05:29:30.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.195455+0000", 2024-09-14T05:29:30.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0574999999999997e-05 2024-09-14T05:29:30.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.195501+0000", 2024-09-14T05:29:30.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5330999999999997e-05 2024-09-14T05:29:30.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-14T05:29:30.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.195604+0000", 2024-09-14T05:29:30.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000103906 2024-09-14T05:29:30.218 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.218 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.218 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.218 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1201 3.9 3:93781545:::rbd_data.112dd25fb4c.000000000000007a:head [call rbd.sparse_copyup in=14115b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.218 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.197062+0000", 2024-09-14T05:29:30.218 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0047481329999999999, 2024-09-14T05:29:30.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0048833349999999999, 2024-09-14T05:29:30.218 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.219 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.219 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "waiting for sub ops", 2024-09-14T05:29:30.219 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.219 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.219 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.219 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1201 2024-09-14T05:29:30.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.219 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.219 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.219 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197062+0000", 2024-09-14T05:29:30.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197062+0000", 2024-09-14T05:29:30.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197062+0000", 2024-09-14T05:29:30.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9400000000000005e-07 2024-09-14T05:29:30.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197079+0000", 2024-09-14T05:29:30.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6118999999999999e-05 2024-09-14T05:29:30.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197087+0000", 2024-09-14T05:29:30.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3170000000000002e-06 2024-09-14T05:29:30.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197093+0000", 2024-09-14T05:29:30.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.246e-06 2024-09-14T05:29:30.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197136+0000", 2024-09-14T05:29:30.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3362000000000003e-05 2024-09-14T05:29:30.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197185+0000", 2024-09-14T05:29:30.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8374000000000001e-05 2024-09-14T05:29:30.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-14T05:29:30.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197305+0000", 2024-09-14T05:29:30.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119782 2024-09-14T05:29:30.222 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.222 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.223 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.223 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1202 3.9 3:93781545:::rbd_data.112dd25fb4c.000000000000007a:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.223 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.197108+0000", 2024-09-14T05:29:30.223 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0047018490000000001, 2024-09-14T05:29:30.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.004876465, 2024-09-14T05:29:30.223 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.223 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.223 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "waiting for sub ops", 2024-09-14T05:29:30.223 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.223 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.224 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.224 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1202 2024-09-14T05:29:30.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.224 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.224 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.224 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197108+0000", 2024-09-14T05:29:30.224 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.224 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.224 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197108+0000", 2024-09-14T05:29:30.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197109+0000", 2024-09-14T05:29:30.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.13e-07 2024-09-14T05:29:30.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197111+0000", 2024-09-14T05:29:30.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.711e-06 2024-09-14T05:29:30.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197120+0000", 2024-09-14T05:29:30.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7409999999999998e-06 2024-09-14T05:29:30.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197125+0000", 2024-09-14T05:29:30.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4990000000000002e-06 2024-09-14T05:29:30.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197993+0000", 2024-09-14T05:29:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000867307 2024-09-14T05:29:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.198063+0000", 2024-09-14T05:29:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0652000000000001e-05 2024-09-14T05:29:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-14T05:29:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.198256+0000", 2024-09-14T05:29:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019209399999999999 2024-09-14T05:29:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop_reply(client.4397.0:1201 3.9 e30/24)", 2024-09-14T05:29:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.199134+0000", 2024-09-14T05:29:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0026753829999999999, 2024-09-14T05:29:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028926770000000002, 2024-09-14T05:29:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.199134+0000", 2024-09-14T05:29:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.199134+0000", 2024-09-14T05:29:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.199136+0000", 2024-09-14T05:29:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.046e-06 2024-09-14T05:29:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.199146+0000", 2024-09-14T05:29:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0379e-05 2024-09-14T05:29:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.199158+0000", 2024-09-14T05:29:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1729000000000001e-05 2024-09-14T05:29:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.199164+0000", 2024-09-14T05:29:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4359999999999999e-06 2024-09-14T05:29:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop_reply(client.4397.0:1202 3.9 e30/24)", 2024-09-14T05:29:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.199685+0000", 2024-09-14T05:29:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0021251759999999999, 2024-09-14T05:29:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0023680009999999998, 2024-09-14T05:29:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.199685+0000", 2024-09-14T05:29:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.199685+0000", 2024-09-14T05:29:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.199685+0000", 2024-09-14T05:29:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3799999999999998e-07 2024-09-14T05:29:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.199688+0000", 2024-09-14T05:29:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6110000000000001e-06 2024-09-14T05:29:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.199695+0000", 2024-09-14T05:29:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9779999999999999e-06 2024-09-14T05:29:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.199698+0000", 2024-09-14T05:29:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4989999999999999e-06 2024-09-14T05:29:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1208 3.d 3:b1b1b80b:::rbd_data.112dd25fb4c.000000000000008a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.200746+0000", 2024-09-14T05:29:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.001063872, 2024-09-14T05:29:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0013472270000000001, 2024-09-14T05:29:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1208 2024-09-14T05:29:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200746+0000", 2024-09-14T05:29:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200746+0000", 2024-09-14T05:29:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200747+0000", 2024-09-14T05:29:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7400000000000002e-07 2024-09-14T05:29:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200757+0000", 2024-09-14T05:29:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9299999999999998e-06 2024-09-14T05:29:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200768+0000", 2024-09-14T05:29:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-05 2024-09-14T05:29:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200772+0000", 2024-09-14T05:29:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9400000000000001e-06 2024-09-14T05:29:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200816+0000", 2024-09-14T05:29:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3730999999999999e-05 2024-09-14T05:29:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200902+0000", 2024-09-14T05:29:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5317999999999999e-05 2024-09-14T05:29:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.239 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:29:30.239 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 8 2024-09-14T05:29:30.239 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:30.240 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:29:30.296 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:30.296+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:30.397 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:30.396+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:30.416 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:30.416 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:30.416 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:30.416 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:30.417 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:30.428 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:29:30.498 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:30.497+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:30.527 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-14T05:29:30.544 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:30.544 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:30.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:30.544 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:30.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.544 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4235.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-14T05:29:30.544 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:29:30.545 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 16.702795990999999, 2024-09-14T05:29:30.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12887758399999999, 2024-09-14T05:29:30.545 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.545 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.545 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:29:30.545 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.545 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4235", 2024-09-14T05:29:30.545 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/3211559243", 2024-09-14T05:29:30.545 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 2 2024-09-14T05:29:30.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.545 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:29:30.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:29:30.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839833+0000", 2024-09-14T05:29:30.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:29:30.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839838+0000", 2024-09-14T05:29:30.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2440000000000001e-06 2024-09-14T05:29:30.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839848+0000", 2024-09-14T05:29:30.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0030000000000001e-05 2024-09-14T05:29:30.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839854+0000", 2024-09-14T05:29:30.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3750000000000002e-06 2024-09-14T05:29:30.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.548 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.548 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839908+0000", 2024-09-14T05:29:30.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3965999999999998e-05 2024-09-14T05:29:30.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.548 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:29:30.548 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839914+0000", 2024-09-14T05:29:30.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6139999999999997e-06 2024-09-14T05:29:30.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968558+0000", 2024-09-14T05:29:30.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12864361199999999 2024-09-14T05:29:30.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968710+0000", 2024-09-14T05:29:30.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015177300000000001 2024-09-14T05:29:30.549 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.549 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.549 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.550 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1088 3.11 3:8fa4ce7b:::rbd_data.112dd25fb4c.0000000000000059:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.550 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.146612+0000", 2024-09-14T05:29:30.550 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.39601639599999999, 2024-09-14T05:29:30.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0076673100000000001, 2024-09-14T05:29:30.550 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.550 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.550 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:30.550 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.550 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.550 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.550 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1088 2024-09-14T05:29:30.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.551 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.146612+0000", 2024-09-14T05:29:30.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.146612+0000", 2024-09-14T05:29:30.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.146612+0000", 2024-09-14T05:29:30.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0299999999999999e-07 2024-09-14T05:29:30.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.146615+0000", 2024-09-14T05:29:30.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.118e-06 2024-09-14T05:29:30.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.146620+0000", 2024-09-14T05:29:30.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6789999999999996e-06 2024-09-14T05:29:30.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.146625+0000", 2024-09-14T05:29:30.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1390000000000003e-06 2024-09-14T05:29:30.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.146950+0000", 2024-09-14T05:29:30.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000325349 2024-09-14T05:29:30.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.147046+0000", 2024-09-14T05:29:30.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5536000000000001e-05 2024-09-14T05:29:30.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.154279+0000", 2024-09-14T05:29:30.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0072329860000000003 2024-09-14T05:29:30.554 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.554 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.554 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.554 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1091 3.11 3:8904a053:::rbd_data.112dd25fb4c.000000000000005c:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.554 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.148300+0000", 2024-09-14T05:29:30.555 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.39432824399999999, 2024-09-14T05:29:30.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.008774347, 2024-09-14T05:29:30.555 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.555 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.555 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:30.555 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.555 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.555 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.555 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1091 2024-09-14T05:29:30.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.555 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.148300+0000", 2024-09-14T05:29:30.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.148300+0000", 2024-09-14T05:29:30.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.148301+0000", 2024-09-14T05:29:30.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2399999999999997e-07 2024-09-14T05:29:30.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.148304+0000", 2024-09-14T05:29:30.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.597e-06 2024-09-14T05:29:30.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.148313+0000", 2024-09-14T05:29:30.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4579999999999993e-06 2024-09-14T05:29:30.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.148319+0000", 2024-09-14T05:29:30.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2110000000000003e-06 2024-09-14T05:29:30.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.148425+0000", 2024-09-14T05:29:30.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000105963 2024-09-14T05:29:30.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.148505+0000", 2024-09-14T05:29:30.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0552999999999999e-05 2024-09-14T05:29:30.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.559 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.157074+0000", 2024-09-14T05:29:30.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0085687409999999995 2024-09-14T05:29:30.559 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.559 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.559 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.560 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1099 3.11 3:8cc65777:::rbd_data.112dd25fb4c.0000000000000058:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.560 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.149585+0000", 2024-09-14T05:29:30.560 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.39304257300000001, 2024-09-14T05:29:30.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0076382430000000003, 2024-09-14T05:29:30.560 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.560 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.560 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:30.560 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.560 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.560 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.560 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1099 2024-09-14T05:29:30.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.561 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.149585+0000", 2024-09-14T05:29:30.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.149585+0000", 2024-09-14T05:29:30.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.149586+0000", 2024-09-14T05:29:30.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7599999999999997e-07 2024-09-14T05:29:30.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.149590+0000", 2024-09-14T05:29:30.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5939999999999998e-06 2024-09-14T05:29:30.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.149598+0000", 2024-09-14T05:29:30.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8490000000000006e-06 2024-09-14T05:29:30.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.149603+0000", 2024-09-14T05:29:30.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7010000000000001e-06 2024-09-14T05:29:30.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.149652+0000", 2024-09-14T05:29:30.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9157999999999999e-05 2024-09-14T05:29:30.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.149723+0000", 2024-09-14T05:29:30.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0553e-05 2024-09-14T05:29:30.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:30.564 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.149924+0000", 2024-09-14T05:29:30.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020064200000000001 2024-09-14T05:29:30.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:30.564 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.154290+0000", 2024-09-14T05:29:30.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0043660690000000002 2024-09-14T05:29:30.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:30.565 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.157120+0000", 2024-09-14T05:29:30.565 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028306859999999998 2024-09-14T05:29:30.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:30.565 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.157180+0000", 2024-09-14T05:29:30.565 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9936999999999999e-05 2024-09-14T05:29:30.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.565 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.157224+0000", 2024-09-14T05:29:30.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3377999999999997e-05 2024-09-14T05:29:30.566 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.566 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.566 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.566 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1154 3.9 3:919c95c5:::rbd_data.112dd25fb4c.0000000000000072:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.566 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.174308+0000", 2024-09-14T05:29:30.566 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.36831975, 2024-09-14T05:29:30.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010732152999999999, 2024-09-14T05:29:30.566 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.566 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.567 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:30.567 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.567 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.567 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.567 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1154 2024-09-14T05:29:30.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.567 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.174308+0000", 2024-09-14T05:29:30.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.174308+0000", 2024-09-14T05:29:30.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.174309+0000", 2024-09-14T05:29:30.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8599999999999997e-07 2024-09-14T05:29:30.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.174313+0000", 2024-09-14T05:29:30.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2250000000000002e-06 2024-09-14T05:29:30.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.174326+0000", 2024-09-14T05:29:30.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2758000000000001e-05 2024-09-14T05:29:30.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.174334+0000", 2024-09-14T05:29:30.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0760000000000003e-06 2024-09-14T05:29:30.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.174383+0000", 2024-09-14T05:29:30.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8918999999999997e-05 2024-09-14T05:29:30.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.570 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.570 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.174459+0000", 2024-09-14T05:29:30.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6352000000000004e-05 2024-09-14T05:29:30.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.570 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-14T05:29:30.570 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.174653+0000", 2024-09-14T05:29:30.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019318499999999999 2024-09-14T05:29:30.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:30.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.178124+0000", 2024-09-14T05:29:30.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034713080000000002 2024-09-14T05:29:30.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:30.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.184948+0000", 2024-09-14T05:29:30.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0068236659999999999 2024-09-14T05:29:30.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:30.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.184994+0000", 2024-09-14T05:29:30.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6001000000000002e-05 2024-09-14T05:29:30.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.185040+0000", 2024-09-14T05:29:30.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6777000000000003e-05 2024-09-14T05:29:30.572 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.572 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.572 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.572 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1164 3.14 3:29d40917:::rbd_data.112dd25fb4c.0000000000000071:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.572 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.177293+0000", 2024-09-14T05:29:30.572 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.36533531899999999, 2024-09-14T05:29:30.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.011789762000000001, 2024-09-14T05:29:30.573 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.573 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.573 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:30.573 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.573 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.573 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.573 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1164 2024-09-14T05:29:30.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.573 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177293+0000", 2024-09-14T05:29:30.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177293+0000", 2024-09-14T05:29:30.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177293+0000", 2024-09-14T05:29:30.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.09e-07 2024-09-14T05:29:30.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177297+0000", 2024-09-14T05:29:30.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7589999999999999e-06 2024-09-14T05:29:30.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.575 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.575 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177307+0000", 2024-09-14T05:29:30.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8539999999999992e-06 2024-09-14T05:29:30.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.575 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.575 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177312+0000", 2024-09-14T05:29:30.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9269999999999996e-06 2024-09-14T05:29:30.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.575 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.575 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177359+0000", 2024-09-14T05:29:30.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6796999999999999e-05 2024-09-14T05:29:30.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.576 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177437+0000", 2024-09-14T05:29:30.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7891000000000004e-05 2024-09-14T05:29:30.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:30.576 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177575+0000", 2024-09-14T05:29:30.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000138176 2024-09-14T05:29:30.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:30.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.185301+0000", 2024-09-14T05:29:30.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0077263139999999998 2024-09-14T05:29:30.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:30.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.188985+0000", 2024-09-14T05:29:30.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036843039999999998 2024-09-14T05:29:30.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:30.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.189039+0000", 2024-09-14T05:29:30.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3919000000000003e-05 2024-09-14T05:29:30.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.189082+0000", 2024-09-14T05:29:30.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3012000000000001e-05 2024-09-14T05:29:30.578 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.578 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.578 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.578 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1199 3.13 3:ce6c3324:::rbd_data.112dd25fb4c.0000000000000085:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.578 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.194776+0000", 2024-09-14T05:29:30.579 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.34785188299999997, 2024-09-14T05:29:30.579 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0076098700000000003, 2024-09-14T05:29:30.579 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.579 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.579 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:30.579 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.579 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.579 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.579 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1199 2024-09-14T05:29:30.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.579 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.580 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194776+0000", 2024-09-14T05:29:30.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.580 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.580 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194776+0000", 2024-09-14T05:29:30.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.580 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.580 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194777+0000", 2024-09-14T05:29:30.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0729999999999999e-06 2024-09-14T05:29:30.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.581 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194781+0000", 2024-09-14T05:29:30.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.675e-06 2024-09-14T05:29:30.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.581 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194792+0000", 2024-09-14T05:29:30.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1596e-05 2024-09-14T05:29:30.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.581 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194802+0000", 2024-09-14T05:29:30.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9769999999999995e-06 2024-09-14T05:29:30.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194857+0000", 2024-09-14T05:29:30.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4698999999999999e-05 2024-09-14T05:29:30.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194973+0000", 2024-09-14T05:29:30.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011576 2024-09-14T05:29:30.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202386+0000", 2024-09-14T05:29:30.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0074130899999999998 2024-09-14T05:29:30.582 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.583 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.583 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.583 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1201 3.9 3:93781545:::rbd_data.112dd25fb4c.000000000000007a:head [call rbd.sparse_copyup in=14115b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.583 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.197062+0000", 2024-09-14T05:29:30.583 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.34556624600000002, 2024-09-14T05:29:30.583 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.013609586, 2024-09-14T05:29:30.583 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.583 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.583 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:30.583 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.583 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.584 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.584 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1201 2024-09-14T05:29:30.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.584 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.584 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.584 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.584 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197062+0000", 2024-09-14T05:29:30.584 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.584 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.584 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.584 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197062+0000", 2024-09-14T05:29:30.584 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.585 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197062+0000", 2024-09-14T05:29:30.585 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9400000000000005e-07 2024-09-14T05:29:30.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.585 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197079+0000", 2024-09-14T05:29:30.585 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6118999999999999e-05 2024-09-14T05:29:30.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.585 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197087+0000", 2024-09-14T05:29:30.586 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3170000000000002e-06 2024-09-14T05:29:30.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.586 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.586 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197093+0000", 2024-09-14T05:29:30.586 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.246e-06 2024-09-14T05:29:30.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.586 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.586 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197136+0000", 2024-09-14T05:29:30.586 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3362000000000003e-05 2024-09-14T05:29:30.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.586 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.587 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197185+0000", 2024-09-14T05:29:30.587 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8374000000000001e-05 2024-09-14T05:29:30.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.587 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.587 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-14T05:29:30.587 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197305+0000", 2024-09-14T05:29:30.587 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119782 2024-09-14T05:29:30.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.587 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.587 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:30.587 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202953+0000", 2024-09-14T05:29:30.587 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056481090000000001 2024-09-14T05:29:30.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.587 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.588 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:30.588 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.210445+0000", 2024-09-14T05:29:30.588 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0074919499999999998 2024-09-14T05:29:30.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.588 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:30.588 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.210521+0000", 2024-09-14T05:29:30.588 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6520000000000006e-05 2024-09-14T05:29:30.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.588 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.588 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.210671+0000", 2024-09-14T05:29:30.588 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000150113 2024-09-14T05:29:30.589 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.589 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.589 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.589 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.589 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1202 3.9 3:93781545:::rbd_data.112dd25fb4c.000000000000007a:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.589 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.197108+0000", 2024-09-14T05:29:30.589 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.34551996200000001, 2024-09-14T05:29:30.589 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017664335999999999, 2024-09-14T05:29:30.589 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.589 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.589 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:30.589 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.589 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.590 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.590 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1202 2024-09-14T05:29:30.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.590 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.590 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.590 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.590 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197108+0000", 2024-09-14T05:29:30.590 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.590 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.590 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.590 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197108+0000", 2024-09-14T05:29:30.590 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.591 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.591 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.591 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197109+0000", 2024-09-14T05:29:30.591 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.13e-07 2024-09-14T05:29:30.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.591 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.591 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.591 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197111+0000", 2024-09-14T05:29:30.591 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.711e-06 2024-09-14T05:29:30.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.591 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.591 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.591 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197120+0000", 2024-09-14T05:29:30.592 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7409999999999998e-06 2024-09-14T05:29:30.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.592 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.592 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197125+0000", 2024-09-14T05:29:30.592 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4990000000000002e-06 2024-09-14T05:29:30.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.592 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.592 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197993+0000", 2024-09-14T05:29:30.592 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000867307 2024-09-14T05:29:30.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.592 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.593 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.198063+0000", 2024-09-14T05:29:30.593 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0652000000000001e-05 2024-09-14T05:29:30.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.593 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.593 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-14T05:29:30.593 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.198256+0000", 2024-09-14T05:29:30.593 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019209399999999999 2024-09-14T05:29:30.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.593 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.593 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:30.593 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203008+0000", 2024-09-14T05:29:30.593 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0047523670000000004 2024-09-14T05:29:30.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.593 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.594 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:30.594 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.214661+0000", 2024-09-14T05:29:30.594 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.011653035000000001 2024-09-14T05:29:30.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.594 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.594 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:30.594 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.214708+0000", 2024-09-14T05:29:30.594 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6737000000000003e-05 2024-09-14T05:29:30.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.594 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.594 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.594 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.214772+0000", 2024-09-14T05:29:30.594 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4679999999999997e-05 2024-09-14T05:29:30.594 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1208 3.d 3:b1b1b80b:::rbd_data.112dd25fb4c.000000000000008a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.200746+0000", 2024-09-14T05:29:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.341881985, 2024-09-14T05:29:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0098531460000000001, 2024-09-14T05:29:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.596 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.596 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1208 2024-09-14T05:29:30.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.596 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.596 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.596 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.596 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200746+0000", 2024-09-14T05:29:30.596 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.596 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.596 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.596 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200746+0000", 2024-09-14T05:29:30.596 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.597 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.597 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.597 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200747+0000", 2024-09-14T05:29:30.597 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7400000000000002e-07 2024-09-14T05:29:30.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.597 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.597 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.597 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200757+0000", 2024-09-14T05:29:30.597 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9299999999999998e-06 2024-09-14T05:29:30.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.597 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.597 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.597 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200768+0000", 2024-09-14T05:29:30.598 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-05 2024-09-14T05:29:30.598 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.598 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.598 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.598 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200772+0000", 2024-09-14T05:29:30.598 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9400000000000001e-06 2024-09-14T05:29:30.598 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.598 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.598 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.598 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200816+0000", 2024-09-14T05:29:30.598 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3730999999999999e-05 2024-09-14T05:29:30.598 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.598 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.598 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.599 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200902+0000", 2024-09-14T05:29:30.599 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5317999999999999e-05 2024-09-14T05:29:30.599 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.599 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.599 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.599 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.210599+0000", 2024-09-14T05:29:30.599 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0096975829999999992 2024-09-14T05:29:30.599 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.599 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.599 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.599 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.599 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.599 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1214 3.13 3:ce6c3324:::rbd_data.112dd25fb4c.0000000000000085:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.600 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.202983+0000", 2024-09-14T05:29:30.600 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.33964510199999998, 2024-09-14T05:29:30.600 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.011801188000000001, 2024-09-14T05:29:30.600 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.600 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.600 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:30.600 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.600 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.600 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.600 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1214 2024-09-14T05:29:30.600 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.600 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.600 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.600 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.601 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202983+0000", 2024-09-14T05:29:30.601 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.601 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.601 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.601 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.601 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202983+0000", 2024-09-14T05:29:30.601 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.601 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.601 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.601 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.601 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202984+0000", 2024-09-14T05:29:30.601 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0669999999999999e-06 2024-09-14T05:29:30.601 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.601 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.602 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.602 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202989+0000", 2024-09-14T05:29:30.602 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1200000000000001e-06 2024-09-14T05:29:30.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.602 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:30.602 INFO:teuthology.orchestra.run.smithi071.stdout:184 read 0x87e3dc2 thru 0x87e5318 (0x1557 bytes) 2024-09-14T05:29:30.603 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.603 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.603 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203001+0000", 2024-09-14T05:29:30.603 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2191999999999999e-05 2024-09-14T05:29:30.603 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.603 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.604 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.604 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203009+0000", 2024-09-14T05:29:30.604 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.259e-06 2024-09-14T05:29:30.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.604 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.604 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.604 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203068+0000", 2024-09-14T05:29:30.604 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8675000000000003e-05 2024-09-14T05:29:30.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.604 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.604 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.604 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203149+0000", 2024-09-14T05:29:30.604 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0358999999999994e-05 2024-09-14T05:29:30.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.605 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-14T05:29:30.605 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203306+0000", 2024-09-14T05:29:30.605 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157529 2024-09-14T05:29:30.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.605 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:30.605 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.210920+0000", 2024-09-14T05:29:30.605 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0076139249999999997 2024-09-14T05:29:30.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.605 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:30.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.214697+0000", 2024-09-14T05:29:30.606 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0037769430000000001 2024-09-14T05:29:30.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.606 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:30.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.214747+0000", 2024-09-14T05:29:30.606 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0136000000000002e-05 2024-09-14T05:29:30.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.606 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.214784+0000", 2024-09-14T05:29:30.606 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6983000000000001e-05 2024-09-14T05:29:30.606 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.606 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.607 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.607 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1224 3.d 3:b1b1b80b:::rbd_data.112dd25fb4c.000000000000008a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.607 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.211081+0000", 2024-09-14T05:29:30.607 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.33154662499999998, 2024-09-14T05:29:30.607 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0081961900000000008, 2024-09-14T05:29:30.607 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.607 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.607 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:30.607 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.607 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.607 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.607 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1224 2024-09-14T05:29:30.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.608 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.608 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.608 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.608 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211081+0000", 2024-09-14T05:29:30.608 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.608 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.608 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.608 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211081+0000", 2024-09-14T05:29:30.608 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.608 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211083+0000", 2024-09-14T05:29:30.609 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2130000000000001e-06 2024-09-14T05:29:30.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211090+0000", 2024-09-14T05:29:30.609 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2559999999999996e-06 2024-09-14T05:29:30.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211100+0000", 2024-09-14T05:29:30.610 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7329999999999998e-06 2024-09-14T05:29:30.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.610 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.610 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.610 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211106+0000", 2024-09-14T05:29:30.610 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.579e-06 2024-09-14T05:29:30.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.610 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.610 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.610 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211157+0000", 2024-09-14T05:29:30.610 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0426999999999997e-05 2024-09-14T05:29:30.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.610 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.610 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.611 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211227+0000", 2024-09-14T05:29:30.611 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0238000000000001e-05 2024-09-14T05:29:30.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.611 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-14T05:29:30.611 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211358+0000", 2024-09-14T05:29:30.611 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000131407 2024-09-14T05:29:30.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.611 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:30.611 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.214743+0000", 2024-09-14T05:29:30.611 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033846739999999998 2024-09-14T05:29:30.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.612 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:30.612 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.219184+0000", 2024-09-14T05:29:30.612 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0044408640000000001 2024-09-14T05:29:30.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.612 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:30.612 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.219237+0000", 2024-09-14T05:29:30.612 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2834000000000003e-05 2024-09-14T05:29:30.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.612 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.612 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.219278+0000", 2024-09-14T05:29:30.612 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0964999999999998e-05 2024-09-14T05:29:30.612 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.613 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.613 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.613 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.613 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.613 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1255 3.a 3:50e3dba7:::rbd_data.112dd25fb4c.0000000000000096:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.613 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.223799+0000", 2024-09-14T05:29:30.613 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.31882894899999997, 2024-09-14T05:29:30.613 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0075545279999999996, 2024-09-14T05:29:30.613 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.613 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.613 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:30.613 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.613 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.614 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.614 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1255 2024-09-14T05:29:30.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.614 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.614 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.614 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.614 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.223799+0000", 2024-09-14T05:29:30.614 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.614 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.614 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.614 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.223799+0000", 2024-09-14T05:29:30.614 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.615 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.223800+0000", 2024-09-14T05:29:30.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.001e-06 2024-09-14T05:29:30.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.615 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.223805+0000", 2024-09-14T05:29:30.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0100000000000003e-06 2024-09-14T05:29:30.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.615 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.223816+0000", 2024-09-14T05:29:30.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0879e-05 2024-09-14T05:29:30.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.616 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.616 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.223822+0000", 2024-09-14T05:29:30.616 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3319999999999997e-06 2024-09-14T05:29:30.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.616 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.616 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.223874+0000", 2024-09-14T05:29:30.616 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2031999999999999e-05 2024-09-14T05:29:30.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.616 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.616 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.223946+0000", 2024-09-14T05:29:30.617 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1279000000000007e-05 2024-09-14T05:29:30.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.617 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-14T05:29:30.617 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.224068+0000", 2024-09-14T05:29:30.617 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000122021 2024-09-14T05:29:30.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.617 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:30.617 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.228274+0000", 2024-09-14T05:29:30.617 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0042066330000000004 2024-09-14T05:29:30.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.617 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:30.618 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231272+0000", 2024-09-14T05:29:30.618 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0029979220000000001 2024-09-14T05:29:30.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.618 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:30.618 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231314+0000", 2024-09-14T05:29:30.618 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1882000000000003e-05 2024-09-14T05:29:30.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.618 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.618 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231354+0000", 2024-09-14T05:29:30.618 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9536999999999998e-05 2024-09-14T05:29:30.618 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.619 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.619 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.619 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "pg_update_log_missing(3.17 epoch 30/24 rep_tid 108 entries 30'12 (0'0) error 3:e9c60235:::rbd_data.112dd25fb4c.00000000000000a1:head by client.4397.0:1262 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'10)", 2024-09-14T05:29:30.619 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.227798+0000", 2024-09-14T05:29:30.619 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.31483039800000001, 2024-09-14T05:29:30.619 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0076164079999999999, 2024-09-14T05:29:30.619 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.619 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.619 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:29:30.619 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.619 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.620 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.227798+0000", 2024-09-14T05:29:30.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.620 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.620 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.227798+0000", 2024-09-14T05:29:30.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.620 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.620 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.227798+0000", 2024-09-14T05:29:30.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6700000000000003e-07 2024-09-14T05:29:30.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.227804+0000", 2024-09-14T05:29:30.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2570000000000004e-06 2024-09-14T05:29:30.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.227819+0000", 2024-09-14T05:29:30.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4738e-05 2024-09-14T05:29:30.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.227828+0000", 2024-09-14T05:29:30.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9129999999999992e-06 2024-09-14T05:29:30.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.622 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.622 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.227890+0000", 2024-09-14T05:29:30.622 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1939000000000004e-05 2024-09-14T05:29:30.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.622 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.622 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.235414+0000", 2024-09-14T05:29:30.622 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.007523994 2024-09-14T05:29:30.622 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.622 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.622 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.623 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "pg_update_log_missing(3.17 epoch 30/24 rep_tid 110 entries 30'14 (0'0) error 3:eebf0247:::rbd_data.112dd25fb4c.00000000000000a3:head by client.4397.0:1266 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'10)", 2024-09-14T05:29:30.623 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.231661+0000", 2024-09-14T05:29:30.623 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.310967096, 2024-09-14T05:29:30.623 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0078005110000000004, 2024-09-14T05:29:30.623 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.623 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.623 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:29:30.623 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.623 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.623 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.623 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231661+0000", 2024-09-14T05:29:30.623 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.623 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.624 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.624 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231661+0000", 2024-09-14T05:29:30.624 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.624 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.624 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231662+0000", 2024-09-14T05:29:30.624 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2900000000000003e-07 2024-09-14T05:29:30.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.624 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.624 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231669+0000", 2024-09-14T05:29:30.624 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7689999999999996e-06 2024-09-14T05:29:30.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.625 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.625 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231690+0000", 2024-09-14T05:29:30.625 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0509e-05 2024-09-14T05:29:30.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.625 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.625 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231704+0000", 2024-09-14T05:29:30.625 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4569000000000001e-05 2024-09-14T05:29:30.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.625 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.625 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231781+0000", 2024-09-14T05:29:30.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6983999999999993e-05 2024-09-14T05:29:30.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.239461+0000", 2024-09-14T05:29:30.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.007679951 2024-09-14T05:29:30.626 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.626 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.626 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.626 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:1270 3.16 e30/24 3:6e66fdcc:::rbd_data.112dd25fb4c.000000000000009f:head v 30'19, mlcod=30'17)", 2024-09-14T05:29:30.626 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.231949+0000", 2024-09-14T05:29:30.626 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.31067901799999997, 2024-09-14T05:29:30.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0076097400000000003, 2024-09-14T05:29:30.627 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.627 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.627 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:30.627 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231949+0000", 2024-09-14T05:29:30.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231949+0000", 2024-09-14T05:29:30.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.628 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.628 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231950+0000", 2024-09-14T05:29:30.628 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5899999999999996e-07 2024-09-14T05:29:30.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.628 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.628 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231956+0000", 2024-09-14T05:29:30.628 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3509999999999998e-06 2024-09-14T05:29:30.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.628 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.628 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231968+0000", 2024-09-14T05:29:30.629 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1688000000000001e-05 2024-09-14T05:29:30.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.629 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.629 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231974+0000", 2024-09-14T05:29:30.629 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7560000000000002e-06 2024-09-14T05:29:30.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.629 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.629 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.232028+0000", 2024-09-14T05:29:30.629 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3860999999999998e-05 2024-09-14T05:29:30.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.629 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.630 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.232045+0000", 2024-09-14T05:29:30.630 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7159e-05 2024-09-14T05:29:30.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.630 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:30.630 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.239458+0000", 2024-09-14T05:29:30.630 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0074129310000000002 2024-09-14T05:29:30.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.630 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.630 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.239559+0000", 2024-09-14T05:29:30.630 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010033499999999999 2024-09-14T05:29:30.630 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.630 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.631 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.631 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1598 3.14 3:2ae9c1cd:::rbd_data.112dd25fb4c.0000000000000143:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.631 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.364021+0000", 2024-09-14T05:29:30.631 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.17860706100000001, 2024-09-14T05:29:30.631 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0080707320000000006, 2024-09-14T05:29:30.631 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.631 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.631 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:30.632 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.632 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.632 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.632 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1598 2024-09-14T05:29:30.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.632 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.632 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.632 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364021+0000", 2024-09-14T05:29:30.632 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.632 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364021+0000", 2024-09-14T05:29:30.633 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364022+0000", 2024-09-14T05:29:30.633 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9000000000000005e-07 2024-09-14T05:29:30.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.634 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364035+0000", 2024-09-14T05:29:30.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3302000000000001e-05 2024-09-14T05:29:30.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.634 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.634 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364049+0000", 2024-09-14T05:29:30.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3329999999999999e-05 2024-09-14T05:29:30.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.634 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.634 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364056+0000", 2024-09-14T05:29:30.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5050000000000001e-06 2024-09-14T05:29:30.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.635 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.635 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364124+0000", 2024-09-14T05:29:30.635 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7739999999999999e-05 2024-09-14T05:29:30.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.635 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.635 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364247+0000", 2024-09-14T05:29:30.635 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012286900000000001 2024-09-14T05:29:30.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.635 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.636 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.372092+0000", 2024-09-14T05:29:30.636 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0078449959999999999 2024-09-14T05:29:30.636 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.636 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.636 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.636 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.636 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1599 3.9 3:977a0fb9:::rbd_data.112dd25fb4c.000000000000013f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.636 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.364092+0000", 2024-09-14T05:29:30.636 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.17853617099999999, 2024-09-14T05:29:30.636 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.008035805, 2024-09-14T05:29:30.636 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.636 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.637 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:30.637 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.637 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.637 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.637 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1599 2024-09-14T05:29:30.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.637 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.637 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.637 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364092+0000", 2024-09-14T05:29:30.637 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.637 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.638 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364092+0000", 2024-09-14T05:29:30.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.638 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.638 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364092+0000", 2024-09-14T05:29:30.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.68e-07 2024-09-14T05:29:30.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.638 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.638 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364096+0000", 2024-09-14T05:29:30.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8029999999999998e-06 2024-09-14T05:29:30.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364105+0000", 2024-09-14T05:29:30.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4069999999999993e-06 2024-09-14T05:29:30.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364110+0000", 2024-09-14T05:29:30.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4999999999999999e-06 2024-09-14T05:29:30.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364169+0000", 2024-09-14T05:29:30.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8517999999999998e-05 2024-09-14T05:29:30.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.640 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364240+0000", 2024-09-14T05:29:30.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0947999999999999e-05 2024-09-14T05:29:30.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.640 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-14T05:29:30.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364456+0000", 2024-09-14T05:29:30.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021666599999999999 2024-09-14T05:29:30.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.640 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:30.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.365279+0000", 2024-09-14T05:29:30.641 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000822287 2024-09-14T05:29:30.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.641 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:30.641 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.372030+0000", 2024-09-14T05:29:30.641 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0067517740000000003 2024-09-14T05:29:30.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.641 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:30.641 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.372089+0000", 2024-09-14T05:29:30.641 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8218999999999999e-05 2024-09-14T05:29:30.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.642 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.642 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.372128+0000", 2024-09-14T05:29:30.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9014999999999998e-05 2024-09-14T05:29:30.642 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.642 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.642 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.642 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.642 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1603 3.d 3:b4928c19:::rbd_data.112dd25fb4c.000000000000013d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.642 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.364654+0000", 2024-09-14T05:29:30.642 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.177973941, 2024-09-14T05:29:30.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.007721197, 2024-09-14T05:29:30.642 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1603 2024-09-14T05:29:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364654+0000", 2024-09-14T05:29:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364654+0000", 2024-09-14T05:29:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364655+0000", 2024-09-14T05:29:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4099999999999997e-07 2024-09-14T05:29:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364659+0000", 2024-09-14T05:29:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1640000000000003e-06 2024-09-14T05:29:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364668+0000", 2024-09-14T05:29:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1819999999999995e-06 2024-09-14T05:29:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364675+0000", 2024-09-14T05:29:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4180000000000002e-06 2024-09-14T05:29:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364721+0000", 2024-09-14T05:29:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6706000000000003e-05 2024-09-14T05:29:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364793+0000", 2024-09-14T05:29:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1619000000000007e-05 2024-09-14T05:29:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-14T05:29:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.365006+0000", 2024-09-14T05:29:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000212945 2024-09-14T05:29:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.372010+0000", 2024-09-14T05:29:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0070042739999999996 2024-09-14T05:29:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.372240+0000", 2024-09-14T05:29:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023001 2024-09-14T05:29:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.372299+0000", 2024-09-14T05:29:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8436000000000001e-05 2024-09-14T05:29:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.372375+0000", 2024-09-14T05:29:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6601999999999996e-05 2024-09-14T05:29:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1819 3.14 3:2f3733d0:::rbd_data.112dd25fb4c.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.456147+0000", 2024-09-14T05:29:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.086480884999999993, 2024-09-14T05:29:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0080915729999999995, 2024-09-14T05:29:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1819 2024-09-14T05:29:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.456147+0000", 2024-09-14T05:29:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.456147+0000", 2024-09-14T05:29:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.456149+0000", 2024-09-14T05:29:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.266e-06 2024-09-14T05:29:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.456157+0000", 2024-09-14T05:29:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8339999999999999e-06 2024-09-14T05:29:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.456177+0000", 2024-09-14T05:29:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9834999999999999e-05 2024-09-14T05:29:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.456186+0000", 2024-09-14T05:29:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.465e-06 2024-09-14T05:29:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.456246+0000", 2024-09-14T05:29:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0004999999999999e-05 2024-09-14T05:29:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.456384+0000", 2024-09-14T05:29:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000137236 2024-09-14T05:29:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.456581+0000", 2024-09-14T05:29:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019773600000000001 2024-09-14T05:29:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.457656+0000", 2024-09-14T05:29:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001074565 2024-09-14T05:29:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.464156+0000", 2024-09-14T05:29:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.006500273 2024-09-14T05:29:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.464202+0000", 2024-09-14T05:29:30.654 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5939999999999997e-05 2024-09-14T05:29:30.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:30.654 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:30.654 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:30.654 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.464239+0000", 2024-09-14T05:29:30.654 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6418000000000003e-05 2024-09-14T05:29:30.654 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.654 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.654 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.654 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:30.654 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:30.654 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:30.655 INFO:teuthology.orchestra.run.smithi071.stdout:187 read 0x81d93cc thru 0x81d974c (0x381 bytes) 2024-09-14T05:29:30.655 INFO:teuthology.orchestra.run.smithi071.stdout:188 write 0x3791dc0 thru 0x379f29f (0xd4e0 bytes) 2024-09-14T05:29:30.656 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:29:30.665 INFO:teuthology.orchestra.run.smithi071.stdout:190 write 0xa56ca1d thru 0xa57c247 (0xf82b bytes) 2024-09-14T05:29:30.665 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:30.664+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:30.738 INFO:teuthology.orchestra.run.smithi071.stdout:191 read 0x5a229f9 thru 0x5a2e3c1 (0xb9c9 bytes) 2024-09-14T05:29:30.738 INFO:teuthology.orchestra.run.smithi071.stdout:193 punch from 0x160aa to 0x1d604, (0x755a bytes) 2024-09-14T05:29:30.738 INFO:teuthology.orchestra.run.smithi071.stdout:195 punch from 0x3f99c9b to 0x3fa0287, (0x65ec bytes) 2024-09-14T05:29:30.745 INFO:teuthology.orchestra.run.smithi071.stdout:196 writesame 0xc5a54ba thru 0xc5a98ae data_size 0x1(0x43f5 bytes) 2024-09-14T05:29:30.745 INFO:teuthology.orchestra.run.smithi071.stdout:197 read 0xc25804b thru 0xc263f34 (0xbeea bytes) 2024-09-14T05:29:30.756 INFO:teuthology.orchestra.run.smithi071.stdout:198 trunc from 0xc5a98af to 0x3679500 2024-09-14T05:29:30.758 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:30.757+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:30.772 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:29:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2127 3.19 3:9dfcbed9:::rbd_object_map.112d1f36d097:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.768558+0000", 2024-09-14T05:29:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.003283008, 2024-09-14T05:29:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033026510000000002, 2024-09-14T05:29:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "waiting for sub ops", 2024-09-14T05:29:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2127 2024-09-14T05:29:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.768558+0000", 2024-09-14T05:29:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.768558+0000", 2024-09-14T05:29:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.768559+0000", 2024-09-14T05:29:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6000000000000003e-07 2024-09-14T05:29:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.768565+0000", 2024-09-14T05:29:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9290000000000003e-06 2024-09-14T05:29:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.768573+0000", 2024-09-14T05:29:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4730000000000003e-06 2024-09-14T05:29:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.768591+0000", 2024-09-14T05:29:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8385000000000001e-05 2024-09-14T05:29:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.769166+0000", 2024-09-14T05:29:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00057512399999999999 2024-09-14T05:29:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.769220+0000", 2024-09-14T05:29:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3497e-05 2024-09-14T05:29:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 1", 2024-09-14T05:29:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.769414+0000", 2024-09-14T05:29:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000194377 2024-09-14T05:29:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.770712+0000", 2024-09-14T05:29:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001298081 2024-09-14T05:29:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2128 3.19 3:9dfcbed9:::rbd_object_map.112d1f36d097:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.768986+0000", 2024-09-14T05:29:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0028558699999999999, 2024-09-14T05:29:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0029185980000000001, 2024-09-14T05:29:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "waiting for sub ops", 2024-09-14T05:29:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2128 2024-09-14T05:29:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.768986+0000", 2024-09-14T05:29:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.768986+0000", 2024-09-14T05:29:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.768986+0000", 2024-09-14T05:29:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5099999999999998e-07 2024-09-14T05:29:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.768998+0000", 2024-09-14T05:29:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1194e-05 2024-09-14T05:29:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.769008+0000", 2024-09-14T05:29:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0366e-05 2024-09-14T05:29:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.769015+0000", 2024-09-14T05:29:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.34e-06 2024-09-14T05:29:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.769667+0000", 2024-09-14T05:29:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00065135200000000001 2024-09-14T05:29:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.769713+0000", 2024-09-14T05:29:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6545999999999998e-05 2024-09-14T05:29:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 1", 2024-09-14T05:29:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.769932+0000", 2024-09-14T05:29:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000218428 2024-09-14T05:29:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.770715+0000", 2024-09-14T05:29:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00078374499999999997 2024-09-14T05:29:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2129 3.19 3:9dfcbed9:::rbd_object_map.112d1f36d097:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.769601+0000", 2024-09-14T05:29:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0022405860000000001, 2024-09-14T05:29:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002339378, 2024-09-14T05:29:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "waiting for sub ops", 2024-09-14T05:29:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2129 2024-09-14T05:29:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.769601+0000", 2024-09-14T05:29:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.769601+0000", 2024-09-14T05:29:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.769602+0000", 2024-09-14T05:29:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8400000000000004e-07 2024-09-14T05:29:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.769605+0000", 2024-09-14T05:29:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5269999999999999e-06 2024-09-14T05:29:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.769614+0000", 2024-09-14T05:29:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0750000000000004e-06 2024-09-14T05:29:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.769619+0000", 2024-09-14T05:29:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4680000000000003e-06 2024-09-14T05:29:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.770215+0000", 2024-09-14T05:29:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00059631400000000002 2024-09-14T05:29:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.770258+0000", 2024-09-14T05:29:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3235000000000002e-05 2024-09-14T05:29:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 1", 2024-09-14T05:29:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.770458+0000", 2024-09-14T05:29:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020012099999999999 2024-09-14T05:29:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.771078+0000", 2024-09-14T05:29:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00062012000000000005 2024-09-14T05:29:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-14T05:29:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 3 2024-09-14T05:29:30.787 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:30.788 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:29:30.835 INFO:teuthology.orchestra.run.smithi071.stdout:199 clone 4 order 22 su 32768 sc 15 2024-09-14T05:29:30.872 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:30.871+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:30.931 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:30.931 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:30.931 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:29:30.931 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:29:30.931 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:30.940 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:29:30.981 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:30.980+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:31.049 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T05:29:31.088 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:31.088 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:29:31.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:29:31.089 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:29:31.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.089 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1134 3.7 3:e1ef7f87:::rbd_data.112dd25fb4c.000000000000005f:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.089 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.165347+0000", 2024-09-14T05:29:31.089 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.92137711700000002, 2024-09-14T05:29:31.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0074546439999999999, 2024-09-14T05:29:31.089 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.089 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.089 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:31.089 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.089 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.090 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1134 2024-09-14T05:29:31.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.090 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.165347+0000", 2024-09-14T05:29:31.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.165347+0000", 2024-09-14T05:29:31.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.165348+0000", 2024-09-14T05:29:31.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3900000000000004e-07 2024-09-14T05:29:31.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.165351+0000", 2024-09-14T05:29:31.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7400000000000002e-06 2024-09-14T05:29:31.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.165357+0000", 2024-09-14T05:29:31.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3410000000000002e-06 2024-09-14T05:29:31.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.165363+0000", 2024-09-14T05:29:31.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8690000000000001e-06 2024-09-14T05:29:31.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166050+0000", 2024-09-14T05:29:31.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00068774500000000002 2024-09-14T05:29:31.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166130+0000", 2024-09-14T05:29:31.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9373000000000001e-05 2024-09-14T05:29:31.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-14T05:29:31.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166316+0000", 2024-09-14T05:29:31.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000185964 2024-09-14T05:29:31.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:31.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.169990+0000", 2024-09-14T05:29:31.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003674105 2024-09-14T05:29:31.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:31.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.172691+0000", 2024-09-14T05:29:31.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0027014700000000001 2024-09-14T05:29:31.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:31.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.172744+0000", 2024-09-14T05:29:31.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3127999999999997e-05 2024-09-14T05:29:31.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.172802+0000", 2024-09-14T05:29:31.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7170000000000003e-05 2024-09-14T05:29:31.095 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.095 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.095 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.095 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1168 3.6 3:67f1c97b:::rbd_data.112dd25fb4c.0000000000000077:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.095 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.180950+0000", 2024-09-14T05:29:31.095 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.90577398200000003, 2024-09-14T05:29:31.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0079994799999999998, 2024-09-14T05:29:31.095 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.096 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.096 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:31.096 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.096 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.096 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.096 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1168 2024-09-14T05:29:31.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.096 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.180950+0000", 2024-09-14T05:29:31.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.180950+0000", 2024-09-14T05:29:31.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.180951+0000", 2024-09-14T05:29:31.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2500000000000004e-07 2024-09-14T05:29:31.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.180956+0000", 2024-09-14T05:29:31.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8330000000000001e-06 2024-09-14T05:29:31.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.180967+0000", 2024-09-14T05:29:31.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1534e-05 2024-09-14T05:29:31.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.180973+0000", 2024-09-14T05:29:31.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5130000000000004e-06 2024-09-14T05:29:31.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.181016+0000", 2024-09-14T05:29:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2744000000000003e-05 2024-09-14T05:29:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.181069+0000", 2024-09-14T05:29:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3643999999999997e-05 2024-09-14T05:29:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 7", 2024-09-14T05:29:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.181192+0000", 2024-09-14T05:29:31.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012239400000000001 2024-09-14T05:29:31.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:31.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.182161+0000", 2024-09-14T05:29:31.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00096957399999999996 2024-09-14T05:29:31.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:31.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.188886+0000", 2024-09-14T05:29:31.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0067245550000000001 2024-09-14T05:29:31.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:31.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.188925+0000", 2024-09-14T05:29:31.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9094999999999998e-05 2024-09-14T05:29:31.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.188950+0000", 2024-09-14T05:29:31.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4669e-05 2024-09-14T05:29:31.101 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.101 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.101 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.101 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1170 3.2 3:42a545a9:::rbd_data.112dd25fb4c.0000000000000075:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.101 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.182701+0000", 2024-09-14T05:29:31.102 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.904023244, 2024-09-14T05:29:31.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0067644180000000003, 2024-09-14T05:29:31.102 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.102 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.102 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:31.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.102 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1170 2024-09-14T05:29:31.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.102 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.182701+0000", 2024-09-14T05:29:31.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.182701+0000", 2024-09-14T05:29:31.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.182702+0000", 2024-09-14T05:29:31.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2600000000000001e-07 2024-09-14T05:29:31.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.182707+0000", 2024-09-14T05:29:31.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1540000000000001e-06 2024-09-14T05:29:31.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.182717+0000", 2024-09-14T05:29:31.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0302000000000001e-05 2024-09-14T05:29:31.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.182739+0000", 2024-09-14T05:29:31.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1548000000000001e-05 2024-09-14T05:29:31.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.182794+0000", 2024-09-14T05:29:31.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4849999999999998e-05 2024-09-14T05:29:31.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.182881+0000", 2024-09-14T05:29:31.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7387999999999997e-05 2024-09-14T05:29:31.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:31.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.183101+0000", 2024-09-14T05:29:31.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022023 2024-09-14T05:29:31.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:31.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.188908+0000", 2024-09-14T05:29:31.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058068809999999998 2024-09-14T05:29:31.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:31.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.189370+0000", 2024-09-14T05:29:31.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046237300000000003 2024-09-14T05:29:31.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:31.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.189424+0000", 2024-09-14T05:29:31.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4015999999999999e-05 2024-09-14T05:29:31.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.189465+0000", 2024-09-14T05:29:31.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0750000000000001e-05 2024-09-14T05:29:31.108 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.108 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.108 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.108 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1194 3.2 3:42efeed1:::rbd_data.112dd25fb4c.0000000000000084:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.108 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.194365+0000", 2024-09-14T05:29:31.108 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.89235907000000003, 2024-09-14T05:29:31.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0083733379999999993, 2024-09-14T05:29:31.109 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.109 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.109 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:31.109 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.109 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.109 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.109 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1194 2024-09-14T05:29:31.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.109 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.194365+0000", 2024-09-14T05:29:31.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.194365+0000", 2024-09-14T05:29:31.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.194366+0000", 2024-09-14T05:29:31.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0699999999999996e-07 2024-09-14T05:29:31.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.194370+0000", 2024-09-14T05:29:31.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8600000000000003e-06 2024-09-14T05:29:31.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.194380+0000", 2024-09-14T05:29:31.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9629999999999993e-06 2024-09-14T05:29:31.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.194385+0000", 2024-09-14T05:29:31.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.04e-06 2024-09-14T05:29:31.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.194431+0000", 2024-09-14T05:29:31.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6857000000000002e-05 2024-09-14T05:29:31.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.194520+0000", 2024-09-14T05:29:31.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8029999999999996e-05 2024-09-14T05:29:31.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.202738+0000", 2024-09-14T05:29:31.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082187809999999997 2024-09-14T05:29:31.113 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.113 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.113 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.113 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1203 3.2 3:4286a8c7:::rbd_data.112dd25fb4c.0000000000000086:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.113 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.197222+0000", 2024-09-14T05:29:31.113 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.88950192500000003, 2024-09-14T05:29:31.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.009216765, 2024-09-14T05:29:31.113 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.113 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.113 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:31.113 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.114 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.114 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.114 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1203 2024-09-14T05:29:31.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.114 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197222+0000", 2024-09-14T05:29:31.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197222+0000", 2024-09-14T05:29:31.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197223+0000", 2024-09-14T05:29:31.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.009e-06 2024-09-14T05:29:31.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197230+0000", 2024-09-14T05:29:31.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7240000000000003e-06 2024-09-14T05:29:31.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197243+0000", 2024-09-14T05:29:31.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3389e-05 2024-09-14T05:29:31.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197251+0000", 2024-09-14T05:29:31.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8220000000000007e-06 2024-09-14T05:29:31.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197305+0000", 2024-09-14T05:29:31.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4208e-05 2024-09-14T05:29:31.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197449+0000", 2024-09-14T05:29:31.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014399599999999999 2024-09-14T05:29:31.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206439+0000", 2024-09-14T05:29:31.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0089896170000000001 2024-09-14T05:29:31.118 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.118 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.118 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.118 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1215 3.2 3:42efeed1:::rbd_data.112dd25fb4c.0000000000000084:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.118 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.203257+0000", 2024-09-14T05:29:31.118 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.88346707499999999, 2024-09-14T05:29:31.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011527635, 2024-09-14T05:29:31.119 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.119 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.119 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:31.119 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.119 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.119 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.119 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1215 2024-09-14T05:29:31.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.119 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.203257+0000", 2024-09-14T05:29:31.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.203257+0000", 2024-09-14T05:29:31.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.203258+0000", 2024-09-14T05:29:31.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5000000000000002e-07 2024-09-14T05:29:31.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.203269+0000", 2024-09-14T05:29:31.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0832999999999999e-05 2024-09-14T05:29:31.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.203278+0000", 2024-09-14T05:29:31.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6269999999999995e-06 2024-09-14T05:29:31.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.203282+0000", 2024-09-14T05:29:31.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.296e-06 2024-09-14T05:29:31.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.203348+0000", 2024-09-14T05:29:31.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5116000000000005e-05 2024-09-14T05:29:31.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.203419+0000", 2024-09-14T05:29:31.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1625000000000006e-05 2024-09-14T05:29:31.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:31.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.203616+0000", 2024-09-14T05:29:31.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019725799999999999 2024-09-14T05:29:31.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:31.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.204332+0000", 2024-09-14T05:29:31.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00071532799999999999 2024-09-14T05:29:31.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:31.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.214725+0000", 2024-09-14T05:29:31.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010393032 2024-09-14T05:29:31.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:31.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.214759+0000", 2024-09-14T05:29:31.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4660999999999999e-05 2024-09-14T05:29:31.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.214785+0000", 2024-09-14T05:29:31.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5109e-05 2024-09-14T05:29:31.124 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.124 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.124 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.124 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1218 3.2 3:4286a8c7:::rbd_data.112dd25fb4c.0000000000000086:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.124 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.206609+0000", 2024-09-14T05:29:31.124 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.88011474099999998, 2024-09-14T05:29:31.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012902059, 2024-09-14T05:29:31.125 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.125 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.125 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:31.125 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.125 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.125 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.125 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1218 2024-09-14T05:29:31.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.125 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206609+0000", 2024-09-14T05:29:31.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206609+0000", 2024-09-14T05:29:31.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206610+0000", 2024-09-14T05:29:31.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1800000000000005e-07 2024-09-14T05:29:31.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206614+0000", 2024-09-14T05:29:31.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3390000000000003e-06 2024-09-14T05:29:31.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206624+0000", 2024-09-14T05:29:31.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0034000000000001e-05 2024-09-14T05:29:31.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206630+0000", 2024-09-14T05:29:31.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1800000000000004e-06 2024-09-14T05:29:31.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206666+0000", 2024-09-14T05:29:31.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6439999999999997e-05 2024-09-14T05:29:31.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206756+0000", 2024-09-14T05:29:31.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9938000000000005e-05 2024-09-14T05:29:31.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:31.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206967+0000", 2024-09-14T05:29:31.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021120399999999999 2024-09-14T05:29:31.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:31.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.207438+0000", 2024-09-14T05:29:31.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00047123100000000002 2024-09-14T05:29:31.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:31.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219403+0000", 2024-09-14T05:29:31.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011964305 2024-09-14T05:29:31.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:31.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219460+0000", 2024-09-14T05:29:31.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7559000000000002e-05 2024-09-14T05:29:31.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219511+0000", 2024-09-14T05:29:31.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1010999999999997e-05 2024-09-14T05:29:31.130 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.130 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.130 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.131 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1265 3.2 3:4509e8b5:::rbd_data.112dd25fb4c.00000000000000a2:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.131 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.228532+0000", 2024-09-14T05:29:31.131 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.85819217599999997, 2024-09-14T05:29:31.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0071511630000000003, 2024-09-14T05:29:31.131 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.131 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.131 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:31.131 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.131 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.131 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.131 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1265 2024-09-14T05:29:31.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.132 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228532+0000", 2024-09-14T05:29:31.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228532+0000", 2024-09-14T05:29:31.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228533+0000", 2024-09-14T05:29:31.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8700000000000005e-07 2024-09-14T05:29:31.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228536+0000", 2024-09-14T05:29:31.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6909999999999999e-06 2024-09-14T05:29:31.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228547+0000", 2024-09-14T05:29:31.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0224000000000001e-05 2024-09-14T05:29:31.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228552+0000", 2024-09-14T05:29:31.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3990000000000003e-06 2024-09-14T05:29:31.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228602+0000", 2024-09-14T05:29:31.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9657999999999998e-05 2024-09-14T05:29:31.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228688+0000", 2024-09-14T05:29:31.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6507000000000004e-05 2024-09-14T05:29:31.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235683+0000", 2024-09-14T05:29:31.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069948969999999999 2024-09-14T05:29:31.136 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.136 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.137 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.137 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1272 3.1b 3:dd2c2221:::rbd_data.112dd25fb4c.00000000000000a0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.137 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.232088+0000", 2024-09-14T05:29:31.137 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.854635539, 2024-09-14T05:29:31.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.007683428, 2024-09-14T05:29:31.137 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.137 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.137 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:31.137 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.138 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.138 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.138 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1272 2024-09-14T05:29:31.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.138 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.232088+0000", 2024-09-14T05:29:31.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.232088+0000", 2024-09-14T05:29:31.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.232089+0000", 2024-09-14T05:29:31.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.47e-07 2024-09-14T05:29:31.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.232094+0000", 2024-09-14T05:29:31.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8230000000000002e-06 2024-09-14T05:29:31.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.232104+0000", 2024-09-14T05:29:31.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8859999999999999e-06 2024-09-14T05:29:31.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.232111+0000", 2024-09-14T05:29:31.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6490000000000002e-06 2024-09-14T05:29:31.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.232171+0000", 2024-09-14T05:29:31.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9851e-05 2024-09-14T05:29:31.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.232249+0000", 2024-09-14T05:29:31.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7917000000000006e-05 2024-09-14T05:29:31.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-14T05:29:31.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.232432+0000", 2024-09-14T05:29:31.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00018371800000000001 2024-09-14T05:29:31.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:31.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.232821+0000", 2024-09-14T05:29:31.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038869600000000001 2024-09-14T05:29:31.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:31.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.239684+0000", 2024-09-14T05:29:31.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0068626210000000002 2024-09-14T05:29:31.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:31.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.239736+0000", 2024-09-14T05:29:31.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2345000000000001e-05 2024-09-14T05:29:31.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.239772+0000", 2024-09-14T05:29:31.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5975e-05 2024-09-14T05:29:31.143 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.143 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.143 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.143 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1277 3.2 3:4509e8b5:::rbd_data.112dd25fb4c.00000000000000a2:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.143 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.235840+0000", 2024-09-14T05:29:31.143 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.85088370300000005, 2024-09-14T05:29:31.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006184213, 2024-09-14T05:29:31.144 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.144 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.144 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:31.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.144 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1277 2024-09-14T05:29:31.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.144 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235840+0000", 2024-09-14T05:29:31.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235840+0000", 2024-09-14T05:29:31.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235841+0000", 2024-09-14T05:29:31.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6400000000000003e-07 2024-09-14T05:29:31.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235843+0000", 2024-09-14T05:29:31.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.48e-06 2024-09-14T05:29:31.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235860+0000", 2024-09-14T05:29:31.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6837000000000002e-05 2024-09-14T05:29:31.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235866+0000", 2024-09-14T05:29:31.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3759999999999996e-06 2024-09-14T05:29:31.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235908+0000", 2024-09-14T05:29:31.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1774999999999999e-05 2024-09-14T05:29:31.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235985+0000", 2024-09-14T05:29:31.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6607999999999996e-05 2024-09-14T05:29:31.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:31.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.236166+0000", 2024-09-14T05:29:31.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00018158399999999999 2024-09-14T05:29:31.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:31.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.236727+0000", 2024-09-14T05:29:31.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00056008399999999997 2024-09-14T05:29:31.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:31.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.241926+0000", 2024-09-14T05:29:31.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051990650000000001 2024-09-14T05:29:31.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:31.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.241983+0000", 2024-09-14T05:29:31.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7513000000000003e-05 2024-09-14T05:29:31.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.242025+0000", 2024-09-14T05:29:31.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1427000000000002e-05 2024-09-14T05:29:31.150 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.150 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.150 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.150 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1278 3.2 3:40ee44a4:::rbd_data.112dd25fb4c.00000000000000a7:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.150 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.235882+0000", 2024-09-14T05:29:31.150 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.85084156300000002, 2024-09-14T05:29:31.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062804510000000003, 2024-09-14T05:29:31.151 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.151 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.151 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:31.151 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.151 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.152 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.152 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1278 2024-09-14T05:29:31.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.152 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235882+0000", 2024-09-14T05:29:31.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235882+0000", 2024-09-14T05:29:31.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235883+0000", 2024-09-14T05:29:31.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4499999999999997e-07 2024-09-14T05:29:31.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235886+0000", 2024-09-14T05:29:31.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.559e-06 2024-09-14T05:29:31.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235894+0000", 2024-09-14T05:29:31.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.636e-06 2024-09-14T05:29:31.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235900+0000", 2024-09-14T05:29:31.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6239999999999999e-06 2024-09-14T05:29:31.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.236404+0000", 2024-09-14T05:29:31.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00050383600000000002 2024-09-14T05:29:31.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.236540+0000", 2024-09-14T05:29:31.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013599799999999999 2024-09-14T05:29:31.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.242163+0000", 2024-09-14T05:29:31.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0056231529999999997 2024-09-14T05:29:31.156 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.156 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.156 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.156 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1455 3.1b 3:dcddadc1:::rbd_data.112dd25fb4c.00000000000000f6:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.156 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.293838+0000", 2024-09-14T05:29:31.156 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.79288606500000003, 2024-09-14T05:29:31.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0056705000000000002, 2024-09-14T05:29:31.157 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.157 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.157 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:31.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.157 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1455 2024-09-14T05:29:31.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.157 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.293838+0000", 2024-09-14T05:29:31.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.293838+0000", 2024-09-14T05:29:31.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.293839+0000", 2024-09-14T05:29:31.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0109999999999999e-06 2024-09-14T05:29:31.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.293844+0000", 2024-09-14T05:29:31.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9919999999999998e-06 2024-09-14T05:29:31.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.293856+0000", 2024-09-14T05:29:31.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2354000000000001e-05 2024-09-14T05:29:31.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.293867+0000", 2024-09-14T05:29:31.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0964999999999999e-05 2024-09-14T05:29:31.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.293922+0000", 2024-09-14T05:29:31.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4305999999999998e-05 2024-09-14T05:29:31.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.293985+0000", 2024-09-14T05:29:31.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3212000000000004e-05 2024-09-14T05:29:31.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-14T05:29:31.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.294157+0000", 2024-09-14T05:29:31.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017214300000000001 2024-09-14T05:29:31.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:31.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.296731+0000", 2024-09-14T05:29:31.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025739090000000001 2024-09-14T05:29:31.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:31.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.299393+0000", 2024-09-14T05:29:31.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002662471 2024-09-14T05:29:31.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:31.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.299461+0000", 2024-09-14T05:29:31.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7464000000000003e-05 2024-09-14T05:29:31.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.299508+0000", 2024-09-14T05:29:31.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7673000000000002e-05 2024-09-14T05:29:31.162 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.162 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.162 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.163 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1483 3.1b 3:d82a462f:::rbd_data.112dd25fb4c.0000000000000105:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.163 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.307842+0000", 2024-09-14T05:29:31.163 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.778882517, 2024-09-14T05:29:31.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057465939999999998, 2024-09-14T05:29:31.163 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.163 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.163 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:31.163 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.163 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.163 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.163 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1483 2024-09-14T05:29:31.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.164 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.307842+0000", 2024-09-14T05:29:31.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.307842+0000", 2024-09-14T05:29:31.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.307842+0000", 2024-09-14T05:29:31.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9899999999999999e-07 2024-09-14T05:29:31.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.307846+0000", 2024-09-14T05:29:31.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.861e-06 2024-09-14T05:29:31.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.307856+0000", 2024-09-14T05:29:31.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0152999999999999e-05 2024-09-14T05:29:31.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.307863+0000", 2024-09-14T05:29:31.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3860000000000004e-06 2024-09-14T05:29:31.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.307904+0000", 2024-09-14T05:29:31.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1053000000000001e-05 2024-09-14T05:29:31.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.307951+0000", 2024-09-14T05:29:31.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6758000000000002e-05 2024-09-14T05:29:31.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-14T05:29:31.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.308121+0000", 2024-09-14T05:29:31.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017007400000000001 2024-09-14T05:29:31.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:31.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.310302+0000", 2024-09-14T05:29:31.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0021811700000000001 2024-09-14T05:29:31.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:31.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.313524+0000", 2024-09-14T05:29:31.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0032217740000000002 2024-09-14T05:29:31.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:31.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.313555+0000", 2024-09-14T05:29:31.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1503e-05 2024-09-14T05:29:31.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.313588+0000", 2024-09-14T05:29:31.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3062999999999998e-05 2024-09-14T05:29:31.169 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.169 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.169 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.169 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1508 3.2 3:411acbb7:::rbd_data.112dd25fb4c.0000000000000115:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.169 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.319136+0000", 2024-09-14T05:29:31.169 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.76758770200000004, 2024-09-14T05:29:31.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058054600000000001, 2024-09-14T05:29:31.170 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.170 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.170 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:31.170 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.170 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.170 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.170 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1508 2024-09-14T05:29:31.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.170 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.319136+0000", 2024-09-14T05:29:31.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.319136+0000", 2024-09-14T05:29:31.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.319137+0000", 2024-09-14T05:29:31.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8299999999999996e-07 2024-09-14T05:29:31.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.319142+0000", 2024-09-14T05:29:31.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.062e-06 2024-09-14T05:29:31.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.319153+0000", 2024-09-14T05:29:31.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0538e-05 2024-09-14T05:29:31.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.319158+0000", 2024-09-14T05:29:31.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1410000000000004e-06 2024-09-14T05:29:31.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.319201+0000", 2024-09-14T05:29:31.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3106999999999999e-05 2024-09-14T05:29:31.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.319275+0000", 2024-09-14T05:29:31.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4289000000000007e-05 2024-09-14T05:29:31.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.324942+0000", 2024-09-14T05:29:31.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0056665400000000003 2024-09-14T05:29:31.174 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.174 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.174 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.174 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1570 3.19 3:9c7bbf12:::rbd_data.112dd25fb4c.0000000000000135:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.174 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.349217+0000", 2024-09-14T05:29:31.174 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.73750713300000004, 2024-09-14T05:29:31.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0068876880000000003, 2024-09-14T05:29:31.174 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.175 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.175 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:31.175 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.175 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.175 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.175 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1570 2024-09-14T05:29:31.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.175 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.349217+0000", 2024-09-14T05:29:31.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.349217+0000", 2024-09-14T05:29:31.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.349218+0000", 2024-09-14T05:29:31.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9199999999999999e-07 2024-09-14T05:29:31.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.349222+0000", 2024-09-14T05:29:31.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0289999999999999e-06 2024-09-14T05:29:31.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.349232+0000", 2024-09-14T05:29:31.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0243e-05 2024-09-14T05:29:31.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.349237+0000", 2024-09-14T05:29:31.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0289999999999996e-06 2024-09-14T05:29:31.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.349285+0000", 2024-09-14T05:29:31.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8078000000000003e-05 2024-09-14T05:29:31.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.349408+0000", 2024-09-14T05:29:31.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012235999999999999 2024-09-14T05:29:31.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.356105+0000", 2024-09-14T05:29:31.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066970570000000002 2024-09-14T05:29:31.180 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.180 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.180 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.180 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1643 3.7 3:e316663b:::rbd_data.112dd25fb4c.0000000000000153:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.181 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.379471+0000", 2024-09-14T05:29:31.181 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.70725262700000002, 2024-09-14T05:29:31.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058808899999999997, 2024-09-14T05:29:31.181 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.181 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.181 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:31.181 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.181 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.181 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.181 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1643 2024-09-14T05:29:31.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.182 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.379471+0000", 2024-09-14T05:29:31.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.379471+0000", 2024-09-14T05:29:31.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.379472+0000", 2024-09-14T05:29:31.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9099999999999991e-07 2024-09-14T05:29:31.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.379477+0000", 2024-09-14T05:29:31.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7879999999999997e-06 2024-09-14T05:29:31.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.379491+0000", 2024-09-14T05:29:31.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4003e-05 2024-09-14T05:29:31.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.379518+0000", 2024-09-14T05:29:31.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6974e-05 2024-09-14T05:29:31.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.379576+0000", 2024-09-14T05:29:31.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7949999999999999e-05 2024-09-14T05:29:31.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.379652+0000", 2024-09-14T05:29:31.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5492000000000002e-05 2024-09-14T05:29:31.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-14T05:29:31.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.380082+0000", 2024-09-14T05:29:31.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043000900000000002 2024-09-14T05:29:31.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:31.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.384958+0000", 2024-09-14T05:29:31.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0048766840000000001 2024-09-14T05:29:31.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:31.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.385256+0000", 2024-09-14T05:29:31.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029748600000000002 2024-09-14T05:29:31.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:31.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.385313+0000", 2024-09-14T05:29:31.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6990000000000002e-05 2024-09-14T05:29:31.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.385352+0000", 2024-09-14T05:29:31.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9523000000000001e-05 2024-09-14T05:29:31.188 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.188 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.188 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.188 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1757 3.8 3:13f000c1:::rbd_data.112dd25fb4c.000000000000018b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.188 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.422057+0000", 2024-09-14T05:29:31.188 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.66466719399999996, 2024-09-14T05:29:31.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0055805749999999999, 2024-09-14T05:29:31.189 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.189 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.189 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:31.189 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.189 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.189 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.189 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1757 2024-09-14T05:29:31.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.189 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.422057+0000", 2024-09-14T05:29:31.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.422057+0000", 2024-09-14T05:29:31.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.422058+0000", 2024-09-14T05:29:31.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0109999999999999e-06 2024-09-14T05:29:31.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.422063+0000", 2024-09-14T05:29:31.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2240000000000003e-06 2024-09-14T05:29:31.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.422072+0000", 2024-09-14T05:29:31.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1169999999999993e-06 2024-09-14T05:29:31.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.422079+0000", 2024-09-14T05:29:31.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6390000000000003e-06 2024-09-14T05:29:31.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.422288+0000", 2024-09-14T05:29:31.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020928599999999999 2024-09-14T05:29:31.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.422369+0000", 2024-09-14T05:29:31.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1116000000000001e-05 2024-09-14T05:29:31.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 7", 2024-09-14T05:29:31.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.422580+0000", 2024-09-14T05:29:31.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000210447 2024-09-14T05:29:31.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:31.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.424229+0000", 2024-09-14T05:29:31.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0016498190000000001 2024-09-14T05:29:31.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:31.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.427537+0000", 2024-09-14T05:29:31.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033075489999999999 2024-09-14T05:29:31.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:31.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.427592+0000", 2024-09-14T05:29:31.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5442000000000002e-05 2024-09-14T05:29:31.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.427637+0000", 2024-09-14T05:29:31.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4925000000000001e-05 2024-09-14T05:29:31.195 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.195 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.195 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.195 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1827 3.2 3:444e135e:::rbd_data.112dd25fb4c.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.195 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.457190+0000", 2024-09-14T05:29:31.195 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.62953397799999999, 2024-09-14T05:29:31.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.007070982, 2024-09-14T05:29:31.196 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.196 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.196 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:31.196 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.196 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.196 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.196 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1827 2024-09-14T05:29:31.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.196 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.457190+0000", 2024-09-14T05:29:31.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.457190+0000", 2024-09-14T05:29:31.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.457191+0000", 2024-09-14T05:29:31.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8299999999999995e-07 2024-09-14T05:29:31.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.457195+0000", 2024-09-14T05:29:31.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5650000000000003e-06 2024-09-14T05:29:31.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.457207+0000", 2024-09-14T05:29:31.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1368e-05 2024-09-14T05:29:31.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.457212+0000", 2024-09-14T05:29:31.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.955e-06 2024-09-14T05:29:31.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.457263+0000", 2024-09-14T05:29:31.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1214000000000001e-05 2024-09-14T05:29:31.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.457389+0000", 2024-09-14T05:29:31.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012592300000000001 2024-09-14T05:29:31.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:31.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.457585+0000", 2024-09-14T05:29:31.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019554699999999999 2024-09-14T05:29:31.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:31.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.458066+0000", 2024-09-14T05:29:31.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0004819 2024-09-14T05:29:31.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:31.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.464179+0000", 2024-09-14T05:29:31.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061123599999999998 2024-09-14T05:29:31.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:31.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.464234+0000", 2024-09-14T05:29:31.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.482e-05 2024-09-14T05:29:31.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.464261+0000", 2024-09-14T05:29:31.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7447e-05 2024-09-14T05:29:31.202 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.202 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.202 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.202 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2130 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.202 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.771622+0000", 2024-09-14T05:29:31.202 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.31510223199999998, 2024-09-14T05:29:31.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046214419, 2024-09-14T05:29:31.203 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.203 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.203 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:31.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.203 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2130 2024-09-14T05:29:31.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.203 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.771622+0000", 2024-09-14T05:29:31.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.771622+0000", 2024-09-14T05:29:31.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.771623+0000", 2024-09-14T05:29:31.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6000000000000003e-07 2024-09-14T05:29:31.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.781526+0000", 2024-09-14T05:29:31.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0099037610000000005 2024-09-14T05:29:31.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.781550+0000", 2024-09-14T05:29:31.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3816999999999999e-05 2024-09-14T05:29:31.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.781560+0000", 2024-09-14T05:29:31.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0179e-05 2024-09-14T05:29:31.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.781629+0000", 2024-09-14T05:29:31.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8644000000000002e-05 2024-09-14T05:29:31.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.781764+0000", 2024-09-14T05:29:31.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013474200000000001 2024-09-14T05:29:31.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-14T05:29:31.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.782079+0000", 2024-09-14T05:29:31.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031578900000000002 2024-09-14T05:29:31.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:31.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.805784+0000", 2024-09-14T05:29:31.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023704310999999999 2024-09-14T05:29:31.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:31.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.817750+0000", 2024-09-14T05:29:31.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0119658 2024-09-14T05:29:31.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:31.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.817794+0000", 2024-09-14T05:29:31.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4005999999999998e-05 2024-09-14T05:29:31.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.817836+0000", 2024-09-14T05:29:31.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.261e-05 2024-09-14T05:29:31.209 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.209 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.209 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.209 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2132 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:31.209 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.781591+0000", 2024-09-14T05:29:31.210 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.30513350700000003, 2024-09-14T05:29:31.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048319642000000003, 2024-09-14T05:29:31.210 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.210 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.210 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:31.210 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.210 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.210 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.210 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2132 2024-09-14T05:29:31.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.210 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.781591+0000", 2024-09-14T05:29:31.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.781591+0000", 2024-09-14T05:29:31.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.781592+0000", 2024-09-14T05:29:31.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2219999999999999e-06 2024-09-14T05:29:31.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.793183+0000", 2024-09-14T05:29:31.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011591704 2024-09-14T05:29:31.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.793225+0000", 2024-09-14T05:29:31.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1347999999999998e-05 2024-09-14T05:29:31.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.793243+0000", 2024-09-14T05:29:31.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-05 2024-09-14T05:29:31.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:31.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.793362+0000", 2024-09-14T05:29:31.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011859899999999999 2024-09-14T05:29:31.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:31.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.793488+0000", 2024-09-14T05:29:31.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012657400000000001 2024-09-14T05:29:31.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-14T05:29:31.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.794048+0000", 2024-09-14T05:29:31.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055983499999999996 2024-09-14T05:29:31.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:31.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824044+0000", 2024-09-14T05:29:31.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029996023 2024-09-14T05:29:31.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:31.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.829834+0000", 2024-09-14T05:29:31.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057904760000000001 2024-09-14T05:29:31.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:31.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.829871+0000", 2024-09-14T05:29:31.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6375999999999999e-05 2024-09-14T05:29:31.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:31.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.829910+0000", 2024-09-14T05:29:31.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9285000000000001e-05 2024-09-14T05:29:31.216 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.216 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.216 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.216 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.216 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.216 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:31.217 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:31.093+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:31.218 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone3 from 0x3679500 (overlap 0x3679500) to 0x0 2024-09-14T05:29:31.218 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-14T05:29:31.219 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:29:31.219 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:31.217+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:31.243 INFO:teuthology.orchestra.run.smithi071.stdout:201 writesame 0xdae35ec thru 0xdaee21a data_size 0x7(0xac2f bytes) 2024-09-14T05:29:31.243 INFO:teuthology.orchestra.run.smithi071.stdout:202 punch from 0xbcbedf3 to 0xbcbf1ad, (0x3ba bytes) 2024-09-14T05:29:31.243 INFO:teuthology.orchestra.run.smithi071.stdout:203 punch from 0x15531b4 to 0x1559738, (0x6584 bytes) 2024-09-14T05:29:31.243 INFO:teuthology.orchestra.run.smithi071.stdout:204 write 0xaf6ffb2 thru 0xaf74113 (0x4162 bytes) 2024-09-14T05:29:31.248 INFO:teuthology.orchestra.run.smithi071.stdout:205 read 0xc36f8d5 thru 0xc37c74b (0xce77 bytes) 2024-09-14T05:29:31.248 INFO:teuthology.orchestra.run.smithi071.stdout:206 write 0xabf198d thru 0xabf77ba (0x5e2e bytes) 2024-09-14T05:29:31.251 INFO:teuthology.orchestra.run.smithi071.stdout:207 punch from 0x7c1c52f to 0x7c1f667, (0x3138 bytes) 2024-09-14T05:29:31.251 INFO:teuthology.orchestra.run.smithi071.stdout:208 write 0x5821365 thru 0x582b2e6 (0x9f82 bytes) 2024-09-14T05:29:31.254 INFO:teuthology.orchestra.run.smithi071.stdout:209 writesame 0xd02c1cc thru 0xd02e618 data_size 0x1(0x244d bytes) 2024-09-14T05:29:31.275 INFO:teuthology.orchestra.run.smithi071.stdout:210 trunc from 0xdaee21b to 0x3fe9fe 2024-09-14T05:29:31.319 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:31.318+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:31.419 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:29:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3356 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.394945+0000", 2024-09-14T05:29:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0082471300000000001, 2024-09-14T05:29:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082687650000000008, 2024-09-14T05:29:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3356 2024-09-14T05:29:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.394945+0000", 2024-09-14T05:29:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.394945+0000", 2024-09-14T05:29:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.394946+0000", 2024-09-14T05:29:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1199999999999999e-07 2024-09-14T05:29:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.394947+0000", 2024-09-14T05:29:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.578e-06 2024-09-14T05:29:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.394953+0000", 2024-09-14T05:29:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2850000000000002e-06 2024-09-14T05:29:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.394955+0000", 2024-09-14T05:29:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6850000000000001e-06 2024-09-14T05:29:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3473 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395678+0000", 2024-09-14T05:29:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0075141510000000002, 2024-09-14T05:29:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075823849999999996, 2024-09-14T05:29:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3473 2024-09-14T05:29:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395678+0000", 2024-09-14T05:29:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395678+0000", 2024-09-14T05:29:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395679+0000", 2024-09-14T05:29:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6199999999999999e-07 2024-09-14T05:29:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395680+0000", 2024-09-14T05:29:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0139999999999999e-06 2024-09-14T05:29:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395683+0000", 2024-09-14T05:29:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.881e-06 2024-09-14T05:29:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395686+0000", 2024-09-14T05:29:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5549999999999998e-06 2024-09-14T05:29:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3568 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.396513+0000", 2024-09-14T05:29:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0066792350000000004, 2024-09-14T05:29:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0067723999999999996, 2024-09-14T05:29:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3568 2024-09-14T05:29:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396513+0000", 2024-09-14T05:29:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396513+0000", 2024-09-14T05:29:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396514+0000", 2024-09-14T05:29:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9500000000000002e-07 2024-09-14T05:29:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396516+0000", 2024-09-14T05:29:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0660000000000002e-06 2024-09-14T05:29:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396522+0000", 2024-09-14T05:29:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2389999999999999e-06 2024-09-14T05:29:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396527+0000", 2024-09-14T05:29:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9949999999999996e-06 2024-09-14T05:29:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3617 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397567+0000", 2024-09-14T05:29:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.005625133, 2024-09-14T05:29:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057457710000000002, 2024-09-14T05:29:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3617 2024-09-14T05:29:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397567+0000", 2024-09-14T05:29:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397567+0000", 2024-09-14T05:29:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397568+0000", 2024-09-14T05:29:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1400000000000001e-07 2024-09-14T05:29:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397569+0000", 2024-09-14T05:29:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0210000000000001e-06 2024-09-14T05:29:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397572+0000", 2024-09-14T05:29:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0029999999999999e-06 2024-09-14T05:29:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397576+0000", 2024-09-14T05:29:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0729999999999998e-06 2024-09-14T05:29:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3683 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.434 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398375+0000", 2024-09-14T05:29:31.434 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0048176499999999997, 2024-09-14T05:29:31.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049614239999999999, 2024-09-14T05:29:31.434 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.434 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.434 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.434 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3683 2024-09-14T05:29:31.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.434 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398375+0000", 2024-09-14T05:29:31.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398375+0000", 2024-09-14T05:29:31.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398375+0000", 2024-09-14T05:29:31.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9000000000000002e-07 2024-09-14T05:29:31.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398377+0000", 2024-09-14T05:29:31.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.677e-06 2024-09-14T05:29:31.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398383+0000", 2024-09-14T05:29:31.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9070000000000004e-06 2024-09-14T05:29:31.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398388+0000", 2024-09-14T05:29:31.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8199999999999996e-06 2024-09-14T05:29:31.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.437 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.437 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3733 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.437 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399094+0000", 2024-09-14T05:29:31.437 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0040988020000000003, 2024-09-14T05:29:31.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042665109999999997, 2024-09-14T05:29:31.438 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.438 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.438 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.438 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3733 2024-09-14T05:29:31.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.438 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399094+0000", 2024-09-14T05:29:31.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399094+0000", 2024-09-14T05:29:31.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399094+0000", 2024-09-14T05:29:31.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9799999999999999e-07 2024-09-14T05:29:31.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399096+0000", 2024-09-14T05:29:31.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.88e-06 2024-09-14T05:29:31.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399102+0000", 2024-09-14T05:29:31.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3119999999999999e-06 2024-09-14T05:29:31.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399107+0000", 2024-09-14T05:29:31.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8269999999999997e-06 2024-09-14T05:29:31.441 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.441 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.441 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.441 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3799 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.441 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399766+0000", 2024-09-14T05:29:31.441 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0034268810000000001, 2024-09-14T05:29:31.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003616953, 2024-09-14T05:29:31.441 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.441 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.441 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.442 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.442 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.442 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3799 2024-09-14T05:29:31.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.442 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399766+0000", 2024-09-14T05:29:31.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399766+0000", 2024-09-14T05:29:31.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399766+0000", 2024-09-14T05:29:31.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7399999999999998e-07 2024-09-14T05:29:31.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399768+0000", 2024-09-14T05:29:31.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9700000000000002e-06 2024-09-14T05:29:31.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399775+0000", 2024-09-14T05:29:31.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8290000000000002e-06 2024-09-14T05:29:31.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399780+0000", 2024-09-14T05:29:31.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8910000000000003e-06 2024-09-14T05:29:31.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.444 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.444 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3848 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.444 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400450+0000", 2024-09-14T05:29:31.444 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0027427200000000001, 2024-09-14T05:29:31.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0029616009999999999, 2024-09-14T05:29:31.444 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.445 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.445 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.445 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3848 2024-09-14T05:29:31.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.445 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400450+0000", 2024-09-14T05:29:31.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400450+0000", 2024-09-14T05:29:31.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400450+0000", 2024-09-14T05:29:31.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3000000000000001e-07 2024-09-14T05:29:31.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400452+0000", 2024-09-14T05:29:31.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8560000000000001e-06 2024-09-14T05:29:31.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400459+0000", 2024-09-14T05:29:31.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2790000000000001e-06 2024-09-14T05:29:31.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400464+0000", 2024-09-14T05:29:31.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.904e-06 2024-09-14T05:29:31.447 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.447 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.447 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.448 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3914 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.448 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401128+0000", 2024-09-14T05:29:31.448 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0020648569999999998, 2024-09-14T05:29:31.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0023157849999999999, 2024-09-14T05:29:31.448 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.448 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.448 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.448 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.448 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.448 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.448 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3914 2024-09-14T05:29:31.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.449 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401128+0000", 2024-09-14T05:29:31.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401128+0000", 2024-09-14T05:29:31.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401128+0000", 2024-09-14T05:29:31.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2500000000000001e-07 2024-09-14T05:29:31.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401130+0000", 2024-09-14T05:29:31.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9099999999999999e-06 2024-09-14T05:29:31.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401135+0000", 2024-09-14T05:29:31.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4720000000000003e-06 2024-09-14T05:29:31.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401140+0000", 2024-09-14T05:29:31.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3100000000000002e-06 2024-09-14T05:29:31.451 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.451 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.451 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.451 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3357 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.451 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.394971+0000", 2024-09-14T05:29:31.451 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0082211899999999997, 2024-09-14T05:29:31.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084970700000000007, 2024-09-14T05:29:31.451 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.452 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.452 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.452 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.452 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.452 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.452 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3357 2024-09-14T05:29:31.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.452 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.394971+0000", 2024-09-14T05:29:31.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.394971+0000", 2024-09-14T05:29:31.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.394972+0000", 2024-09-14T05:29:31.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0200000000000002e-07 2024-09-14T05:29:31.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.394974+0000", 2024-09-14T05:29:31.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9870000000000002e-06 2024-09-14T05:29:31.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.394980+0000", 2024-09-14T05:29:31.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2339999999999999e-06 2024-09-14T05:29:31.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.394985+0000", 2024-09-14T05:29:31.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8180000000000003e-06 2024-09-14T05:29:31.454 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.454 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.455 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.455 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3474 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.455 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395691+0000", 2024-09-14T05:29:31.455 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0075010299999999997, 2024-09-14T05:29:31.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0077993560000000003, 2024-09-14T05:29:31.455 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.455 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.455 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.455 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.455 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.455 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.455 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3474 2024-09-14T05:29:31.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.456 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395691+0000", 2024-09-14T05:29:31.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395691+0000", 2024-09-14T05:29:31.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395692+0000", 2024-09-14T05:29:31.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3200000000000001e-07 2024-09-14T05:29:31.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395694+0000", 2024-09-14T05:29:31.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7150000000000001e-06 2024-09-14T05:29:31.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395699+0000", 2024-09-14T05:29:31.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2079999999999999e-06 2024-09-14T05:29:31.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395702+0000", 2024-09-14T05:29:31.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9019999999999999e-06 2024-09-14T05:29:31.458 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.458 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.458 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.458 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3569 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.458 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.396533+0000", 2024-09-14T05:29:31.458 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0066591979999999999, 2024-09-14T05:29:31.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069802370000000002, 2024-09-14T05:29:31.459 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.459 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.459 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.459 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.459 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.459 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.459 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3569 2024-09-14T05:29:31.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.459 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396533+0000", 2024-09-14T05:29:31.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396533+0000", 2024-09-14T05:29:31.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396534+0000", 2024-09-14T05:29:31.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9100000000000004e-07 2024-09-14T05:29:31.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396536+0000", 2024-09-14T05:29:31.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.753e-06 2024-09-14T05:29:31.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396541+0000", 2024-09-14T05:29:31.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.801e-06 2024-09-14T05:29:31.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396546+0000", 2024-09-14T05:29:31.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6439999999999999e-06 2024-09-14T05:29:31.462 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.462 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.462 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.462 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3618 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.462 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397580+0000", 2024-09-14T05:29:31.462 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0056124360000000002, 2024-09-14T05:29:31.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059758570000000002, 2024-09-14T05:29:31.462 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.462 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.462 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.462 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.462 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.463 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.463 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3618 2024-09-14T05:29:31.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.463 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397580+0000", 2024-09-14T05:29:31.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397580+0000", 2024-09-14T05:29:31.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397580+0000", 2024-09-14T05:29:31.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5699999999999998e-07 2024-09-14T05:29:31.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397582+0000", 2024-09-14T05:29:31.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.105e-06 2024-09-14T05:29:31.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397589+0000", 2024-09-14T05:29:31.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0180000000000004e-06 2024-09-14T05:29:31.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397593+0000", 2024-09-14T05:29:31.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5639999999999998e-06 2024-09-14T05:29:31.465 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.465 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.465 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.465 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3684 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.465 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398394+0000", 2024-09-14T05:29:31.466 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0047987990000000003, 2024-09-14T05:29:31.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051890340000000004, 2024-09-14T05:29:31.466 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.466 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.466 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.466 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.466 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.466 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.466 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3684 2024-09-14T05:29:31.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.466 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398394+0000", 2024-09-14T05:29:31.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398394+0000", 2024-09-14T05:29:31.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398394+0000", 2024-09-14T05:29:31.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9800000000000003e-07 2024-09-14T05:29:31.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398396+0000", 2024-09-14T05:29:31.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1129999999999999e-06 2024-09-14T05:29:31.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398402+0000", 2024-09-14T05:29:31.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0440000000000001e-06 2024-09-14T05:29:31.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398408+0000", 2024-09-14T05:29:31.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0710000000000002e-06 2024-09-14T05:29:31.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.468 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.469 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.469 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3734 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.469 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399113+0000", 2024-09-14T05:29:31.469 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.004079371, 2024-09-14T05:29:31.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0045009200000000003, 2024-09-14T05:29:31.469 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.469 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.469 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.470 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3734 2024-09-14T05:29:31.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.470 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399113+0000", 2024-09-14T05:29:31.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399113+0000", 2024-09-14T05:29:31.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399114+0000", 2024-09-14T05:29:31.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.01e-07 2024-09-14T05:29:31.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399115+0000", 2024-09-14T05:29:31.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7039999999999999e-06 2024-09-14T05:29:31.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399121+0000", 2024-09-14T05:29:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6019999999999999e-06 2024-09-14T05:29:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399126+0000", 2024-09-14T05:29:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7199999999999997e-06 2024-09-14T05:29:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3800 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.472 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399786+0000", 2024-09-14T05:29:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0034069259999999998, 2024-09-14T05:29:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0038557520000000001, 2024-09-14T05:29:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3800 2024-09-14T05:29:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399786+0000", 2024-09-14T05:29:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399786+0000", 2024-09-14T05:29:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399786+0000", 2024-09-14T05:29:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0400000000000002e-07 2024-09-14T05:29:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399795+0000", 2024-09-14T05:29:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5380000000000002e-06 2024-09-14T05:29:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399802+0000", 2024-09-14T05:29:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0400000000000004e-06 2024-09-14T05:29:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399807+0000", 2024-09-14T05:29:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.045e-06 2024-09-14T05:29:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.475 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3849 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400470+0000", 2024-09-14T05:29:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0027225169999999998, 2024-09-14T05:29:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031941199999999999, 2024-09-14T05:29:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3849 2024-09-14T05:29:31.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400470+0000", 2024-09-14T05:29:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400470+0000", 2024-09-14T05:29:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400471+0000", 2024-09-14T05:29:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.51e-07 2024-09-14T05:29:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400472+0000", 2024-09-14T05:29:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889999999999999e-06 2024-09-14T05:29:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400478+0000", 2024-09-14T05:29:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5809999999999996e-06 2024-09-14T05:29:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400482+0000", 2024-09-14T05:29:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2050000000000004e-06 2024-09-14T05:29:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3915 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401146+0000", 2024-09-14T05:29:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0020466500000000001, 2024-09-14T05:29:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025443250000000001, 2024-09-14T05:29:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3915 2024-09-14T05:29:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.480 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401146+0000", 2024-09-14T05:29:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401146+0000", 2024-09-14T05:29:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401146+0000", 2024-09-14T05:29:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3099999999999998e-07 2024-09-14T05:29:31.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401148+0000", 2024-09-14T05:29:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8419999999999999e-06 2024-09-14T05:29:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401154+0000", 2024-09-14T05:29:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6779999999999997e-06 2024-09-14T05:29:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401158+0000", 2024-09-14T05:29:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3490000000000002e-06 2024-09-14T05:29:31.482 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3358 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.394992+0000", 2024-09-14T05:29:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0082005350000000001, 2024-09-14T05:29:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0087207169999999994, 2024-09-14T05:29:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3358 2024-09-14T05:29:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.394992+0000", 2024-09-14T05:29:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.394992+0000", 2024-09-14T05:29:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.394992+0000", 2024-09-14T05:29:31.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6699999999999999e-07 2024-09-14T05:29:31.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.394995+0000", 2024-09-14T05:29:31.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2330000000000001e-06 2024-09-14T05:29:31.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395000+0000", 2024-09-14T05:29:31.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.857e-06 2024-09-14T05:29:31.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395005+0000", 2024-09-14T05:29:31.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6199999999999998e-06 2024-09-14T05:29:31.486 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.486 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.486 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.486 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3475 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.486 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395725+0000", 2024-09-14T05:29:31.486 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0074671599999999996, 2024-09-14T05:29:31.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080151949999999993, 2024-09-14T05:29:31.487 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.487 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.487 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.487 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.487 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.487 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.487 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3475 2024-09-14T05:29:31.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.487 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395725+0000", 2024-09-14T05:29:31.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395725+0000", 2024-09-14T05:29:31.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395726+0000", 2024-09-14T05:29:31.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3499999999999996e-07 2024-09-14T05:29:31.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395728+0000", 2024-09-14T05:29:31.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1629999999999999e-06 2024-09-14T05:29:31.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395734+0000", 2024-09-14T05:29:31.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2820000000000002e-06 2024-09-14T05:29:31.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395739+0000", 2024-09-14T05:29:31.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6609999999999999e-06 2024-09-14T05:29:31.489 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.489 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.489 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.490 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3570 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.490 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.396552+0000", 2024-09-14T05:29:31.490 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0066405580000000004, 2024-09-14T05:29:31.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0072111220000000004, 2024-09-14T05:29:31.490 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.490 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.490 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.490 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.490 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.490 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.490 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3570 2024-09-14T05:29:31.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.491 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396552+0000", 2024-09-14T05:29:31.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396552+0000", 2024-09-14T05:29:31.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396552+0000", 2024-09-14T05:29:31.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.75e-07 2024-09-14T05:29:31.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396554+0000", 2024-09-14T05:29:31.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.916e-06 2024-09-14T05:29:31.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396560+0000", 2024-09-14T05:29:31.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7409999999999998e-06 2024-09-14T05:29:31.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396565+0000", 2024-09-14T05:29:31.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5979999999999999e-06 2024-09-14T05:29:31.492 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.493 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.493 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.493 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3619 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.493 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397599+0000", 2024-09-14T05:29:31.493 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0055932229999999996, 2024-09-14T05:29:31.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061860429999999996, 2024-09-14T05:29:31.493 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.493 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.493 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.494 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.494 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3619 2024-09-14T05:29:31.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.494 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397599+0000", 2024-09-14T05:29:31.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397599+0000", 2024-09-14T05:29:31.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397600+0000", 2024-09-14T05:29:31.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3700000000000001e-07 2024-09-14T05:29:31.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397610+0000", 2024-09-14T05:29:31.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0247e-05 2024-09-14T05:29:31.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397617+0000", 2024-09-14T05:29:31.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3200000000000002e-06 2024-09-14T05:29:31.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397622+0000", 2024-09-14T05:29:31.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.454e-06 2024-09-14T05:29:31.496 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.496 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.496 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.497 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3686 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.497 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398415+0000", 2024-09-14T05:29:31.497 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.004777407, 2024-09-14T05:29:31.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053926900000000003, 2024-09-14T05:29:31.497 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.497 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.497 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.497 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.497 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.497 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.497 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3686 2024-09-14T05:29:31.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.498 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398415+0000", 2024-09-14T05:29:31.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398415+0000", 2024-09-14T05:29:31.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398416+0000", 2024-09-14T05:29:31.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-07 2024-09-14T05:29:31.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398418+0000", 2024-09-14T05:29:31.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.902e-06 2024-09-14T05:29:31.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398424+0000", 2024-09-14T05:29:31.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2199999999999997e-06 2024-09-14T05:29:31.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398429+0000", 2024-09-14T05:29:31.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8500000000000002e-06 2024-09-14T05:29:31.500 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.500 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.500 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.500 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3735 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.500 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399132+0000", 2024-09-14T05:29:31.500 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0040607969999999997, 2024-09-14T05:29:31.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0047071659999999996, 2024-09-14T05:29:31.501 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.501 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.501 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.501 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.501 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.501 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.501 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3735 2024-09-14T05:29:31.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.501 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399132+0000", 2024-09-14T05:29:31.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399132+0000", 2024-09-14T05:29:31.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399132+0000", 2024-09-14T05:29:31.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8099999999999998e-07 2024-09-14T05:29:31.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399134+0000", 2024-09-14T05:29:31.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7719999999999999e-06 2024-09-14T05:29:31.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399139+0000", 2024-09-14T05:29:31.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4229999999999996e-06 2024-09-14T05:29:31.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399144+0000", 2024-09-14T05:29:31.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4870000000000004e-06 2024-09-14T05:29:31.503 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.503 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.503 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.504 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3801 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.504 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399813+0000", 2024-09-14T05:29:31.504 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0033792649999999998, 2024-09-14T05:29:31.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040516600000000003, 2024-09-14T05:29:31.504 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.504 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.504 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.504 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.504 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.504 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.504 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3801 2024-09-14T05:29:31.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.505 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399813+0000", 2024-09-14T05:29:31.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399813+0000", 2024-09-14T05:29:31.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399814+0000", 2024-09-14T05:29:31.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4999999999999998e-07 2024-09-14T05:29:31.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399816+0000", 2024-09-14T05:29:31.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.95e-06 2024-09-14T05:29:31.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399821+0000", 2024-09-14T05:29:31.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.66e-06 2024-09-14T05:29:31.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399830+0000", 2024-09-14T05:29:31.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5329999999999994e-06 2024-09-14T05:29:31.507 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.507 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.507 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.507 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3850 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.507 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400488+0000", 2024-09-14T05:29:31.507 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0027043890000000002, 2024-09-14T05:29:31.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003400999, 2024-09-14T05:29:31.508 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.508 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.508 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.508 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3850 2024-09-14T05:29:31.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.508 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400488+0000", 2024-09-14T05:29:31.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400488+0000", 2024-09-14T05:29:31.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400489+0000", 2024-09-14T05:29:31.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3500000000000002e-07 2024-09-14T05:29:31.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400490+0000", 2024-09-14T05:29:31.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9039999999999999e-06 2024-09-14T05:29:31.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400497+0000", 2024-09-14T05:29:31.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.049e-06 2024-09-14T05:29:31.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400501+0000", 2024-09-14T05:29:31.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-14T05:29:31.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.511 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.511 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3916 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.511 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401164+0000", 2024-09-14T05:29:31.511 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0020281890000000001, 2024-09-14T05:29:31.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0027473369999999999, 2024-09-14T05:29:31.511 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.511 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.511 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.512 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3916 2024-09-14T05:29:31.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.512 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401164+0000", 2024-09-14T05:29:31.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401164+0000", 2024-09-14T05:29:31.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401165+0000", 2024-09-14T05:29:31.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1699999999999999e-07 2024-09-14T05:29:31.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401167+0000", 2024-09-14T05:29:31.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.883e-06 2024-09-14T05:29:31.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401173+0000", 2024-09-14T05:29:31.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.472e-06 2024-09-14T05:29:31.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401178+0000", 2024-09-14T05:29:31.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7099999999999998e-06 2024-09-14T05:29:31.514 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.514 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.514 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.514 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3359 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.514 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395020+0000", 2024-09-14T05:29:31.515 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0081723260000000006, 2024-09-14T05:29:31.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0089271960000000001, 2024-09-14T05:29:31.515 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.515 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.515 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.515 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3359 2024-09-14T05:29:31.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.515 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395020+0000", 2024-09-14T05:29:31.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395020+0000", 2024-09-14T05:29:31.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395021+0000", 2024-09-14T05:29:31.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2900000000000003e-07 2024-09-14T05:29:31.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395023+0000", 2024-09-14T05:29:31.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.148e-06 2024-09-14T05:29:31.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395029+0000", 2024-09-14T05:29:31.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8529999999999997e-06 2024-09-14T05:29:31.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395033+0000", 2024-09-14T05:29:31.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4150000000000001e-06 2024-09-14T05:29:31.518 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.518 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.518 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.518 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3476 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.518 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395764+0000", 2024-09-14T05:29:31.518 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0074282929999999999, 2024-09-14T05:29:31.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082074020000000008, 2024-09-14T05:29:31.518 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.518 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.518 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.519 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3476 2024-09-14T05:29:31.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.519 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395764+0000", 2024-09-14T05:29:31.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395764+0000", 2024-09-14T05:29:31.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395765+0000", 2024-09-14T05:29:31.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3700000000000001e-07 2024-09-14T05:29:31.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395767+0000", 2024-09-14T05:29:31.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8759999999999999e-06 2024-09-14T05:29:31.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395771+0000", 2024-09-14T05:29:31.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.916e-06 2024-09-14T05:29:31.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395774+0000", 2024-09-14T05:29:31.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.994e-06 2024-09-14T05:29:31.521 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.521 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.521 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.522 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3571 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.522 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.396570+0000", 2024-09-14T05:29:31.522 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.006622111, 2024-09-14T05:29:31.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.007442812, 2024-09-14T05:29:31.522 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.522 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.522 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.522 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.522 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.522 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.522 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3571 2024-09-14T05:29:31.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.523 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396570+0000", 2024-09-14T05:29:31.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396570+0000", 2024-09-14T05:29:31.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396571+0000", 2024-09-14T05:29:31.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6000000000000004e-07 2024-09-14T05:29:31.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396573+0000", 2024-09-14T05:29:31.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7600000000000001e-06 2024-09-14T05:29:31.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396578+0000", 2024-09-14T05:29:31.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6960000000000002e-06 2024-09-14T05:29:31.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396584+0000", 2024-09-14T05:29:31.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0769999999999998e-06 2024-09-14T05:29:31.525 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.525 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.525 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.525 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3620 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.525 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397627+0000", 2024-09-14T05:29:31.525 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.005565385, 2024-09-14T05:29:31.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0064142230000000001, 2024-09-14T05:29:31.526 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.526 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.526 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.526 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.526 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.526 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.526 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3620 2024-09-14T05:29:31.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.526 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397627+0000", 2024-09-14T05:29:31.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397627+0000", 2024-09-14T05:29:31.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397627+0000", 2024-09-14T05:29:31.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.96e-07 2024-09-14T05:29:31.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397629+0000", 2024-09-14T05:29:31.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.265e-06 2024-09-14T05:29:31.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397633+0000", 2024-09-14T05:29:31.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3490000000000002e-06 2024-09-14T05:29:31.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397636+0000", 2024-09-14T05:29:31.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7870000000000002e-06 2024-09-14T05:29:31.528 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.528 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.528 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.529 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3687 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.529 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398435+0000", 2024-09-14T05:29:31.529 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0047578309999999997, 2024-09-14T05:29:31.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0056402199999999996, 2024-09-14T05:29:31.529 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.529 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.529 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.529 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3687 2024-09-14T05:29:31.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.529 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398435+0000", 2024-09-14T05:29:31.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398435+0000", 2024-09-14T05:29:31.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398435+0000", 2024-09-14T05:29:31.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0100000000000004e-07 2024-09-14T05:29:31.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398437+0000", 2024-09-14T05:29:31.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7409999999999999e-06 2024-09-14T05:29:31.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398443+0000", 2024-09-14T05:29:31.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7780000000000004e-06 2024-09-14T05:29:31.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398448+0000", 2024-09-14T05:29:31.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9429999999999999e-06 2024-09-14T05:29:31.531 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.532 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.532 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.532 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3736 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.532 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399150+0000", 2024-09-14T05:29:31.532 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.004043011, 2024-09-14T05:29:31.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.004950299, 2024-09-14T05:29:31.532 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.532 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.532 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.532 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.532 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.532 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.532 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3736 2024-09-14T05:29:31.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.533 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399150+0000", 2024-09-14T05:29:31.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399150+0000", 2024-09-14T05:29:31.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399150+0000", 2024-09-14T05:29:31.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-14T05:29:31.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399152+0000", 2024-09-14T05:29:31.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-06 2024-09-14T05:29:31.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399157+0000", 2024-09-14T05:29:31.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6149999999999996e-06 2024-09-14T05:29:31.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399162+0000", 2024-09-14T05:29:31.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3610000000000003e-06 2024-09-14T05:29:31.535 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.535 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.535 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.535 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3802 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.535 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399838+0000", 2024-09-14T05:29:31.535 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0033548889999999998, 2024-09-14T05:29:31.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.004284548, 2024-09-14T05:29:31.535 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.535 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.535 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.536 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.536 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.536 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.536 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3802 2024-09-14T05:29:31.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.536 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399838+0000", 2024-09-14T05:29:31.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399838+0000", 2024-09-14T05:29:31.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399838+0000", 2024-09-14T05:29:31.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5600000000000001e-07 2024-09-14T05:29:31.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399840+0000", 2024-09-14T05:29:31.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2850000000000001e-06 2024-09-14T05:29:31.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399847+0000", 2024-09-14T05:29:31.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1709999999999999e-06 2024-09-14T05:29:31.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399852+0000", 2024-09-14T05:29:31.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9590000000000003e-06 2024-09-14T05:29:31.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.538 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.539 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3851 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.539 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400507+0000", 2024-09-14T05:29:31.539 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0026856269999999999, 2024-09-14T05:29:31.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0036493749999999998, 2024-09-14T05:29:31.539 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.539 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.539 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.539 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3851 2024-09-14T05:29:31.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.540 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400507+0000", 2024-09-14T05:29:31.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400507+0000", 2024-09-14T05:29:31.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400507+0000", 2024-09-14T05:29:31.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3700000000000001e-07 2024-09-14T05:29:31.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400509+0000", 2024-09-14T05:29:31.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6339999999999999e-06 2024-09-14T05:29:31.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400515+0000", 2024-09-14T05:29:31.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5310000000000001e-06 2024-09-14T05:29:31.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400519+0000", 2024-09-14T05:29:31.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9400000000000001e-06 2024-09-14T05:29:31.542 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.542 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.542 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.542 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3917 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.542 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401184+0000", 2024-09-14T05:29:31.542 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.002008138, 2024-09-14T05:29:31.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002994834, 2024-09-14T05:29:31.543 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.543 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.543 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.543 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3917 2024-09-14T05:29:31.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.543 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401184+0000", 2024-09-14T05:29:31.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401184+0000", 2024-09-14T05:29:31.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401185+0000", 2024-09-14T05:29:31.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9700000000000002e-07 2024-09-14T05:29:31.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401187+0000", 2024-09-14T05:29:31.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8700000000000001e-06 2024-09-14T05:29:31.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401192+0000", 2024-09-14T05:29:31.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6810000000000003e-06 2024-09-14T05:29:31.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401197+0000", 2024-09-14T05:29:31.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6990000000000002e-06 2024-09-14T05:29:31.546 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.546 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.546 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.546 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3360 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.546 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395040+0000", 2024-09-14T05:29:31.546 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0081522910000000007, 2024-09-14T05:29:31.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091644729999999994, 2024-09-14T05:29:31.546 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.546 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.546 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.546 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.547 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3360 2024-09-14T05:29:31.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.547 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395040+0000", 2024-09-14T05:29:31.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395040+0000", 2024-09-14T05:29:31.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395041+0000", 2024-09-14T05:29:31.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2800000000000002e-07 2024-09-14T05:29:31.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395042+0000", 2024-09-14T05:29:31.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2049999999999999e-06 2024-09-14T05:29:31.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395048+0000", 2024-09-14T05:29:31.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9349999999999999e-06 2024-09-14T05:29:31.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395053+0000", 2024-09-14T05:29:31.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6300000000000003e-06 2024-09-14T05:29:31.549 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.549 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.549 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.550 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3572 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.550 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.396591+0000", 2024-09-14T05:29:31.550 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0066010590000000003, 2024-09-14T05:29:31.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076359740000000002, 2024-09-14T05:29:31.550 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.550 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.550 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.550 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3572 2024-09-14T05:29:31.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.550 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396591+0000", 2024-09-14T05:29:31.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396591+0000", 2024-09-14T05:29:31.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396592+0000", 2024-09-14T05:29:31.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6100000000000001e-07 2024-09-14T05:29:31.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396594+0000", 2024-09-14T05:29:31.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9300000000000002e-06 2024-09-14T05:29:31.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396600+0000", 2024-09-14T05:29:31.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7069999999999997e-06 2024-09-14T05:29:31.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396605+0000", 2024-09-14T05:29:31.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0370000000000002e-06 2024-09-14T05:29:31.553 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.553 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.553 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.553 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3621 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.553 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397640+0000", 2024-09-14T05:29:31.553 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0055529050000000003, 2024-09-14T05:29:31.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066100500000000001, 2024-09-14T05:29:31.554 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.554 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.554 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.554 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.554 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.554 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.554 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3621 2024-09-14T05:29:31.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.554 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397640+0000", 2024-09-14T05:29:31.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397640+0000", 2024-09-14T05:29:31.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397640+0000", 2024-09-14T05:29:31.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2499999999999999e-07 2024-09-14T05:29:31.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397641+0000", 2024-09-14T05:29:31.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0759999999999999e-06 2024-09-14T05:29:31.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397644+0000", 2024-09-14T05:29:31.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0960000000000001e-06 2024-09-14T05:29:31.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397647+0000", 2024-09-14T05:29:31.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6790000000000001e-06 2024-09-14T05:29:31.557 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.557 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.557 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.557 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3688 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.557 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398454+0000", 2024-09-14T05:29:31.557 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0047384640000000004, 2024-09-14T05:29:31.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058199059999999997, 2024-09-14T05:29:31.557 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.557 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.557 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.557 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.557 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.558 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.558 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3688 2024-09-14T05:29:31.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.558 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398454+0000", 2024-09-14T05:29:31.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398454+0000", 2024-09-14T05:29:31.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398454+0000", 2024-09-14T05:29:31.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9000000000000002e-07 2024-09-14T05:29:31.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398456+0000", 2024-09-14T05:29:31.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8360000000000001e-06 2024-09-14T05:29:31.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398462+0000", 2024-09-14T05:29:31.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6969999999999998e-06 2024-09-14T05:29:31.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398466+0000", 2024-09-14T05:29:31.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3209999999999998e-06 2024-09-14T05:29:31.560 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.560 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.560 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.561 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3737 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.561 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399168+0000", 2024-09-14T05:29:31.561 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0040248139999999998, 2024-09-14T05:29:31.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005144371, 2024-09-14T05:29:31.561 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.561 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.561 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.561 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.561 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.561 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.561 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3737 2024-09-14T05:29:31.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.562 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399168+0000", 2024-09-14T05:29:31.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399168+0000", 2024-09-14T05:29:31.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399168+0000", 2024-09-14T05:29:31.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7700000000000004e-07 2024-09-14T05:29:31.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399170+0000", 2024-09-14T05:29:31.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-14T05:29:31.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399176+0000", 2024-09-14T05:29:31.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7910000000000001e-06 2024-09-14T05:29:31.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399181+0000", 2024-09-14T05:29:31.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6480000000000002e-06 2024-09-14T05:29:31.564 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.564 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.564 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.564 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3803 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.564 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399871+0000", 2024-09-14T05:29:31.564 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.003321615, 2024-09-14T05:29:31.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0044706850000000003, 2024-09-14T05:29:31.564 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.565 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.565 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.565 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.565 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.565 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.565 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3803 2024-09-14T05:29:31.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.565 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399871+0000", 2024-09-14T05:29:31.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399871+0000", 2024-09-14T05:29:31.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399871+0000", 2024-09-14T05:29:31.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7000000000000005e-07 2024-09-14T05:29:31.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399874+0000", 2024-09-14T05:29:31.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2199999999999999e-06 2024-09-14T05:29:31.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399881+0000", 2024-09-14T05:29:31.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0169999999999999e-06 2024-09-14T05:29:31.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399886+0000", 2024-09-14T05:29:31.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7320000000000003e-06 2024-09-14T05:29:31.567 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.567 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.567 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.568 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3852 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.568 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400529+0000", 2024-09-14T05:29:31.568 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0026639849999999998, 2024-09-14T05:29:31.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0038358139999999999, 2024-09-14T05:29:31.568 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.568 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.568 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.568 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.568 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.568 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.568 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3852 2024-09-14T05:29:31.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.568 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400529+0000", 2024-09-14T05:29:31.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400529+0000", 2024-09-14T05:29:31.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400529+0000", 2024-09-14T05:29:31.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1600000000000002e-07 2024-09-14T05:29:31.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400531+0000", 2024-09-14T05:29:31.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-14T05:29:31.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400537+0000", 2024-09-14T05:29:31.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.02e-06 2024-09-14T05:29:31.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400544+0000", 2024-09-14T05:29:31.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3590000000000002e-06 2024-09-14T05:29:31.571 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.571 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.571 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.571 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3918 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.571 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401203+0000", 2024-09-14T05:29:31.571 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0019898310000000001, 2024-09-14T05:29:31.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003183824, 2024-09-14T05:29:31.572 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.572 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.572 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.572 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.572 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.572 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.572 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3918 2024-09-14T05:29:31.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.572 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401203+0000", 2024-09-14T05:29:31.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401203+0000", 2024-09-14T05:29:31.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401203+0000", 2024-09-14T05:29:31.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.72e-07 2024-09-14T05:29:31.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401206+0000", 2024-09-14T05:29:31.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8210000000000002e-06 2024-09-14T05:29:31.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401212+0000", 2024-09-14T05:29:31.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7790000000000001e-06 2024-09-14T05:29:31.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401217+0000", 2024-09-14T05:29:31.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-14T05:29:31.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.574 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3446 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395080+0000", 2024-09-14T05:29:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0081123589999999995, 2024-09-14T05:29:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0093288699999999995, 2024-09-14T05:29:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3446 2024-09-14T05:29:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395080+0000", 2024-09-14T05:29:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395080+0000", 2024-09-14T05:29:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395081+0000", 2024-09-14T05:29:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3999999999999997e-07 2024-09-14T05:29:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395082+0000", 2024-09-14T05:29:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.739e-06 2024-09-14T05:29:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395087+0000", 2024-09-14T05:29:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6500000000000004e-06 2024-09-14T05:29:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395091+0000", 2024-09-14T05:29:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7859999999999998e-06 2024-09-14T05:29:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3573 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.396612+0000", 2024-09-14T05:29:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0065802150000000004, 2024-09-14T05:29:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078199350000000001, 2024-09-14T05:29:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3573 2024-09-14T05:29:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396612+0000", 2024-09-14T05:29:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396612+0000", 2024-09-14T05:29:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396613+0000", 2024-09-14T05:29:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4000000000000002e-07 2024-09-14T05:29:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396615+0000", 2024-09-14T05:29:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9640000000000002e-06 2024-09-14T05:29:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396620+0000", 2024-09-14T05:29:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6409999999999999e-06 2024-09-14T05:29:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396625+0000", 2024-09-14T05:29:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.69e-06 2024-09-14T05:29:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3622 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397652+0000", 2024-09-14T05:29:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0055408180000000003, 2024-09-14T05:29:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0068041100000000004, 2024-09-14T05:29:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3622 2024-09-14T05:29:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397652+0000", 2024-09-14T05:29:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397652+0000", 2024-09-14T05:29:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397652+0000", 2024-09-14T05:29:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0800000000000004e-07 2024-09-14T05:29:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397653+0000", 2024-09-14T05:29:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1260000000000001e-06 2024-09-14T05:29:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397657+0000", 2024-09-14T05:29:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.461e-06 2024-09-14T05:29:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397661+0000", 2024-09-14T05:29:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4050000000000002e-06 2024-09-14T05:29:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3689 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398505+0000", 2024-09-14T05:29:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0046877220000000001, 2024-09-14T05:29:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059844060000000003, 2024-09-14T05:29:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3689 2024-09-14T05:29:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398505+0000", 2024-09-14T05:29:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398505+0000", 2024-09-14T05:29:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398505+0000", 2024-09-14T05:29:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6100000000000001e-07 2024-09-14T05:29:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398508+0000", 2024-09-14T05:29:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.34e-06 2024-09-14T05:29:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398514+0000", 2024-09-14T05:29:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.691e-06 2024-09-14T05:29:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398520+0000", 2024-09-14T05:29:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2560000000000002e-06 2024-09-14T05:29:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3738 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399187+0000", 2024-09-14T05:29:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0040060260000000002, 2024-09-14T05:29:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053264779999999999, 2024-09-14T05:29:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3738 2024-09-14T05:29:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399187+0000", 2024-09-14T05:29:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399187+0000", 2024-09-14T05:29:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399187+0000", 2024-09-14T05:29:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-14T05:29:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399189+0000", 2024-09-14T05:29:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.765e-06 2024-09-14T05:29:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399195+0000", 2024-09-14T05:29:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8549999999999998e-06 2024-09-14T05:29:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399199+0000", 2024-09-14T05:29:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8110000000000002e-06 2024-09-14T05:29:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3804 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399893+0000", 2024-09-14T05:29:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0032993440000000001, 2024-09-14T05:29:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0046438829999999997, 2024-09-14T05:29:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3804 2024-09-14T05:29:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399893+0000", 2024-09-14T05:29:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399893+0000", 2024-09-14T05:29:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399894+0000", 2024-09-14T05:29:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7000000000000005e-07 2024-09-14T05:29:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399896+0000", 2024-09-14T05:29:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9039999999999999e-06 2024-09-14T05:29:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399902+0000", 2024-09-14T05:29:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3600000000000001e-06 2024-09-14T05:29:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399908+0000", 2024-09-14T05:29:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9510000000000003e-06 2024-09-14T05:29:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3853 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400550+0000", 2024-09-14T05:29:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0026420979999999998, 2024-09-14T05:29:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040099790000000003, 2024-09-14T05:29:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3853 2024-09-14T05:29:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400550+0000", 2024-09-14T05:29:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400550+0000", 2024-09-14T05:29:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400551+0000", 2024-09-14T05:29:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6199999999999998e-07 2024-09-14T05:29:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400557+0000", 2024-09-14T05:29:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3239999999999999e-06 2024-09-14T05:29:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400564+0000", 2024-09-14T05:29:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.404e-06 2024-09-14T05:29:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400569+0000", 2024-09-14T05:29:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9790000000000001e-06 2024-09-14T05:29:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3919 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401226+0000", 2024-09-14T05:29:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0019661240000000001, 2024-09-14T05:29:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003357053, 2024-09-14T05:29:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3919 2024-09-14T05:29:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401226+0000", 2024-09-14T05:29:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401226+0000", 2024-09-14T05:29:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401227+0000", 2024-09-14T05:29:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0399999999999996e-07 2024-09-14T05:29:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401229+0000", 2024-09-14T05:29:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.046e-06 2024-09-14T05:29:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401236+0000", 2024-09-14T05:29:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1420000000000004e-06 2024-09-14T05:29:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401241+0000", 2024-09-14T05:29:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.113e-06 2024-09-14T05:29:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3447 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395098+0000", 2024-09-14T05:29:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0080947910000000005, 2024-09-14T05:29:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0095211170000000008, 2024-09-14T05:29:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3447 2024-09-14T05:29:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395098+0000", 2024-09-14T05:29:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395098+0000", 2024-09-14T05:29:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395098+0000", 2024-09-14T05:29:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7800000000000001e-07 2024-09-14T05:29:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395100+0000", 2024-09-14T05:29:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8330000000000001e-06 2024-09-14T05:29:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395106+0000", 2024-09-14T05:29:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.981e-06 2024-09-14T05:29:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395111+0000", 2024-09-14T05:29:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1919999999999996e-06 2024-09-14T05:29:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3574 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.396655+0000", 2024-09-14T05:29:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0065376690000000003, 2024-09-14T05:29:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0079891470000000003, 2024-09-14T05:29:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3574 2024-09-14T05:29:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396655+0000", 2024-09-14T05:29:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396655+0000", 2024-09-14T05:29:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396655+0000", 2024-09-14T05:29:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7899999999999998e-07 2024-09-14T05:29:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396659+0000", 2024-09-14T05:29:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.371e-06 2024-09-14T05:29:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396667+0000", 2024-09-14T05:29:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4889999999999995e-06 2024-09-14T05:29:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396674+0000", 2024-09-14T05:29:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5239999999999997e-06 2024-09-14T05:29:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3623 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397667+0000", 2024-09-14T05:29:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0055257350000000004, 2024-09-14T05:29:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0070114089999999997, 2024-09-14T05:29:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3623 2024-09-14T05:29:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397667+0000", 2024-09-14T05:29:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397667+0000", 2024-09-14T05:29:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397667+0000", 2024-09-14T05:29:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0899999999999997e-07 2024-09-14T05:29:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397669+0000", 2024-09-14T05:29:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.497e-06 2024-09-14T05:29:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397674+0000", 2024-09-14T05:29:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6489999999999996e-06 2024-09-14T05:29:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397679+0000", 2024-09-14T05:29:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4560000000000002e-06 2024-09-14T05:29:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3690 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398535+0000", 2024-09-14T05:29:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.004658003, 2024-09-14T05:29:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061747679999999997, 2024-09-14T05:29:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3690 2024-09-14T05:29:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398535+0000", 2024-09-14T05:29:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398535+0000", 2024-09-14T05:29:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398535+0000", 2024-09-14T05:29:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-14T05:29:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398538+0000", 2024-09-14T05:29:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.39e-06 2024-09-14T05:29:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398544+0000", 2024-09-14T05:29:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6290000000000004e-06 2024-09-14T05:29:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398550+0000", 2024-09-14T05:29:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2389999999999999e-06 2024-09-14T05:29:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3739 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399205+0000", 2024-09-14T05:29:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0039872960000000004, 2024-09-14T05:29:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0055313419999999999, 2024-09-14T05:29:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3739 2024-09-14T05:29:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399205+0000", 2024-09-14T05:29:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399205+0000", 2024-09-14T05:29:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399206+0000", 2024-09-14T05:29:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0900000000000002e-07 2024-09-14T05:29:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399208+0000", 2024-09-14T05:29:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8610000000000001e-06 2024-09-14T05:29:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399213+0000", 2024-09-14T05:29:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8619999999999999e-06 2024-09-14T05:29:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399219+0000", 2024-09-14T05:29:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3129999999999998e-06 2024-09-14T05:29:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3805 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399914+0000", 2024-09-14T05:29:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.003278394, 2024-09-14T05:29:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0048460689999999997, 2024-09-14T05:29:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3805 2024-09-14T05:29:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399914+0000", 2024-09-14T05:29:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399914+0000", 2024-09-14T05:29:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399915+0000", 2024-09-14T05:29:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1799999999999995e-07 2024-09-14T05:29:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399917+0000", 2024-09-14T05:29:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.869e-06 2024-09-14T05:29:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399923+0000", 2024-09-14T05:29:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4479999999999999e-06 2024-09-14T05:29:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399929+0000", 2024-09-14T05:29:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0909999999999998e-06 2024-09-14T05:29:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3854 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400575+0000", 2024-09-14T05:29:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0026177330000000001, 2024-09-14T05:29:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042079179999999997, 2024-09-14T05:29:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3854 2024-09-14T05:29:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400575+0000", 2024-09-14T05:29:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400575+0000", 2024-09-14T05:29:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400575+0000", 2024-09-14T05:29:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4999999999999998e-07 2024-09-14T05:29:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400577+0000", 2024-09-14T05:29:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0049999999999999e-06 2024-09-14T05:29:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400583+0000", 2024-09-14T05:29:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8479999999999997e-06 2024-09-14T05:29:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400588+0000", 2024-09-14T05:29:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.561e-06 2024-09-14T05:29:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3920 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401248+0000", 2024-09-14T05:29:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.001944791, 2024-09-14T05:29:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035571969999999998, 2024-09-14T05:29:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3920 2024-09-14T05:29:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401248+0000", 2024-09-14T05:29:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401248+0000", 2024-09-14T05:29:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401248+0000", 2024-09-14T05:29:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.37e-07 2024-09-14T05:29:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401250+0000", 2024-09-14T05:29:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9319999999999999e-06 2024-09-14T05:29:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401256+0000", 2024-09-14T05:29:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8880000000000002e-06 2024-09-14T05:29:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401261+0000", 2024-09-14T05:29:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8060000000000002e-06 2024-09-14T05:29:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3448 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395117+0000", 2024-09-14T05:29:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0080750779999999994, 2024-09-14T05:29:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0097307109999999995, 2024-09-14T05:29:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3448 2024-09-14T05:29:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395117+0000", 2024-09-14T05:29:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395117+0000", 2024-09-14T05:29:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395118+0000", 2024-09-14T05:29:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.65e-07 2024-09-14T05:29:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395120+0000", 2024-09-14T05:29:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8309999999999999e-06 2024-09-14T05:29:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395126+0000", 2024-09-14T05:29:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4799999999999998e-06 2024-09-14T05:29:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395130+0000", 2024-09-14T05:29:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8199999999999998e-06 2024-09-14T05:29:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3575 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.396700+0000", 2024-09-14T05:29:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0064927120000000003, 2024-09-14T05:29:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081941270000000007, 2024-09-14T05:29:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3575 2024-09-14T05:29:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396700+0000", 2024-09-14T05:29:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396700+0000", 2024-09-14T05:29:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396700+0000", 2024-09-14T05:29:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9299999999999998e-07 2024-09-14T05:29:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396703+0000", 2024-09-14T05:29:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5289999999999998e-06 2024-09-14T05:29:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396709+0000", 2024-09-14T05:29:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.229e-06 2024-09-14T05:29:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396714+0000", 2024-09-14T05:29:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2719999999999997e-06 2024-09-14T05:29:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3624 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397684+0000", 2024-09-14T05:29:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0055085250000000002, 2024-09-14T05:29:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0072561680000000003, 2024-09-14T05:29:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3624 2024-09-14T05:29:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397684+0000", 2024-09-14T05:29:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397684+0000", 2024-09-14T05:29:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397685+0000", 2024-09-14T05:29:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5000000000000003e-07 2024-09-14T05:29:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397686+0000", 2024-09-14T05:29:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.497e-06 2024-09-14T05:29:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397691+0000", 2024-09-14T05:29:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4140000000000002e-06 2024-09-14T05:29:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397696+0000", 2024-09-14T05:29:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3749999999999996e-06 2024-09-14T05:29:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3691 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398571+0000", 2024-09-14T05:29:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0046216399999999998, 2024-09-14T05:29:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0064210200000000004, 2024-09-14T05:29:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3691 2024-09-14T05:29:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398571+0000", 2024-09-14T05:29:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398571+0000", 2024-09-14T05:29:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398572+0000", 2024-09-14T05:29:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4000000000000001e-07 2024-09-14T05:29:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398574+0000", 2024-09-14T05:29:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2699999999999999e-06 2024-09-14T05:29:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398581+0000", 2024-09-14T05:29:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.793e-06 2024-09-14T05:29:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398586+0000", 2024-09-14T05:29:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8839999999999999e-06 2024-09-14T05:29:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3740 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399225+0000", 2024-09-14T05:29:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0039679270000000004, 2024-09-14T05:29:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00581323, 2024-09-14T05:29:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3740 2024-09-14T05:29:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399225+0000", 2024-09-14T05:29:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399225+0000", 2024-09-14T05:29:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399225+0000", 2024-09-14T05:29:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9900000000000001e-07 2024-09-14T05:29:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399227+0000", 2024-09-14T05:29:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8729999999999999e-06 2024-09-14T05:29:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399234+0000", 2024-09-14T05:29:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7920000000000004e-06 2024-09-14T05:29:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399239+0000", 2024-09-14T05:29:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9089999999999999e-06 2024-09-14T05:29:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3806 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399935+0000", 2024-09-14T05:29:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0032571980000000002, 2024-09-14T05:29:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051366800000000002, 2024-09-14T05:29:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3806 2024-09-14T05:29:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399935+0000", 2024-09-14T05:29:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399935+0000", 2024-09-14T05:29:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399936+0000", 2024-09-14T05:29:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1899999999999998e-07 2024-09-14T05:29:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399938+0000", 2024-09-14T05:29:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9439999999999999e-06 2024-09-14T05:29:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399944+0000", 2024-09-14T05:29:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9869999999999996e-06 2024-09-14T05:29:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399948+0000", 2024-09-14T05:29:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.741e-06 2024-09-14T05:29:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3855 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400593+0000", 2024-09-14T05:29:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0025991679999999998, 2024-09-14T05:29:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0045040389999999996, 2024-09-14T05:29:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3855 2024-09-14T05:29:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400593+0000", 2024-09-14T05:29:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400593+0000", 2024-09-14T05:29:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400594+0000", 2024-09-14T05:29:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8799999999999998e-07 2024-09-14T05:29:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400595+0000", 2024-09-14T05:29:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.75e-06 2024-09-14T05:29:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400602+0000", 2024-09-14T05:29:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5570000000000001e-06 2024-09-14T05:29:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400607+0000", 2024-09-14T05:29:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:29:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3921 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:29:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.001925143, 2024-09-14T05:29:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0038542509999999999, 2024-09-14T05:29:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3921 2024-09-14T05:29:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:29:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:29:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401705+0000", 2024-09-14T05:29:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768499999999999 2024-09-14T05:29:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401709+0000", 2024-09-14T05:29:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0319999999999997e-06 2024-09-14T05:29:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401720+0000", 2024-09-14T05:29:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1005000000000001e-05 2024-09-14T05:29:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401727+0000", 2024-09-14T05:29:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1879999999999996e-06 2024-09-14T05:29:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3449 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395147+0000", 2024-09-14T05:29:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0080453020000000007, 2024-09-14T05:29:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010070887000000001, 2024-09-14T05:29:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3449 2024-09-14T05:29:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395147+0000", 2024-09-14T05:29:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395147+0000", 2024-09-14T05:29:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395148+0000", 2024-09-14T05:29:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000002e-07 2024-09-14T05:29:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395150+0000", 2024-09-14T05:29:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.041e-06 2024-09-14T05:29:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395156+0000", 2024-09-14T05:29:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.387e-06 2024-09-14T05:29:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395161+0000", 2024-09-14T05:29:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3019999999999996e-06 2024-09-14T05:29:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3576 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.396936+0000", 2024-09-14T05:29:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0062568110000000001, 2024-09-14T05:29:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0083065840000000005, 2024-09-14T05:29:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3576 2024-09-14T05:29:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396936+0000", 2024-09-14T05:29:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396936+0000", 2024-09-14T05:29:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396936+0000", 2024-09-14T05:29:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5899999999999996e-07 2024-09-14T05:29:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396939+0000", 2024-09-14T05:29:31.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8289999999999999e-06 2024-09-14T05:29:31.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396947+0000", 2024-09-14T05:29:31.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9470000000000003e-06 2024-09-14T05:29:31.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396952+0000", 2024-09-14T05:29:31.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.51e-06 2024-09-14T05:29:31.665 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.665 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.665 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.665 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3625 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.665 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397711+0000", 2024-09-14T05:29:31.665 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0054812070000000001, 2024-09-14T05:29:31.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075619499999999996, 2024-09-14T05:29:31.665 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.666 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.666 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.666 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.666 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.666 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.666 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3625 2024-09-14T05:29:31.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.666 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397711+0000", 2024-09-14T05:29:31.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397711+0000", 2024-09-14T05:29:31.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397712+0000", 2024-09-14T05:29:31.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0699999999999998e-07 2024-09-14T05:29:31.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397713+0000", 2024-09-14T05:29:31.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.604e-06 2024-09-14T05:29:31.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397718+0000", 2024-09-14T05:29:31.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5070000000000002e-06 2024-09-14T05:29:31.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397720+0000", 2024-09-14T05:29:31.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7350000000000001e-06 2024-09-14T05:29:31.668 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.668 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.669 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.669 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3692 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.669 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398593+0000", 2024-09-14T05:29:31.669 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.004599161, 2024-09-14T05:29:31.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0067093179999999997, 2024-09-14T05:29:31.669 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.669 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.669 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.669 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.669 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.669 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.670 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3692 2024-09-14T05:29:31.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.670 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398593+0000", 2024-09-14T05:29:31.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398593+0000", 2024-09-14T05:29:31.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398594+0000", 2024-09-14T05:29:31.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4700000000000001e-07 2024-09-14T05:29:31.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398596+0000", 2024-09-14T05:29:31.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1689999999999999e-06 2024-09-14T05:29:31.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398602+0000", 2024-09-14T05:29:31.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1650000000000003e-06 2024-09-14T05:29:31.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398607+0000", 2024-09-14T05:29:31.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9790000000000001e-06 2024-09-14T05:29:31.672 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.672 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.672 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.672 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3741 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.672 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399245+0000", 2024-09-14T05:29:31.672 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0039471869999999996, 2024-09-14T05:29:31.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060992909999999997, 2024-09-14T05:29:31.673 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.673 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.673 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.673 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.673 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.673 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.673 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3741 2024-09-14T05:29:31.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.673 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399245+0000", 2024-09-14T05:29:31.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399245+0000", 2024-09-14T05:29:31.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399246+0000", 2024-09-14T05:29:31.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2899999999999999e-07 2024-09-14T05:29:31.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399248+0000", 2024-09-14T05:29:31.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.891e-06 2024-09-14T05:29:31.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399253+0000", 2024-09-14T05:29:31.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7089999999999999e-06 2024-09-14T05:29:31.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399258+0000", 2024-09-14T05:29:31.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-14T05:29:31.675 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.676 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.676 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.676 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3807 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.676 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399954+0000", 2024-09-14T05:29:31.676 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0032381670000000001, 2024-09-14T05:29:31.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054166209999999999, 2024-09-14T05:29:31.676 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.676 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.676 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.676 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.676 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.677 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.677 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3807 2024-09-14T05:29:31.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.677 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399954+0000", 2024-09-14T05:29:31.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399954+0000", 2024-09-14T05:29:31.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399955+0000", 2024-09-14T05:29:31.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3000000000000001e-07 2024-09-14T05:29:31.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399957+0000", 2024-09-14T05:29:31.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9099999999999999e-06 2024-09-14T05:29:31.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399963+0000", 2024-09-14T05:29:31.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9399999999999999e-06 2024-09-14T05:29:31.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399967+0000", 2024-09-14T05:29:31.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8069999999999999e-06 2024-09-14T05:29:31.679 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.679 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.679 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.679 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3856 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.679 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400613+0000", 2024-09-14T05:29:31.680 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0025791009999999999, 2024-09-14T05:29:31.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0047796269999999998, 2024-09-14T05:29:31.680 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.680 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.680 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.680 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.680 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.680 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.680 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3856 2024-09-14T05:29:31.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.680 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400613+0000", 2024-09-14T05:29:31.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400613+0000", 2024-09-14T05:29:31.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400614+0000", 2024-09-14T05:29:31.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3300000000000003e-07 2024-09-14T05:29:31.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400616+0000", 2024-09-14T05:29:31.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.075e-06 2024-09-14T05:29:31.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400622+0000", 2024-09-14T05:29:31.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7119999999999997e-06 2024-09-14T05:29:31.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400626+0000", 2024-09-14T05:29:31.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5340000000000001e-06 2024-09-14T05:29:31.683 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.683 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.683 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.683 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3922 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.683 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401739+0000", 2024-09-14T05:29:31.683 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.001453974, 2024-09-14T05:29:31.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0036965050000000001, 2024-09-14T05:29:31.683 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.683 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.683 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.683 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.684 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3922 2024-09-14T05:29:31.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.684 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401739+0000", 2024-09-14T05:29:31.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401739+0000", 2024-09-14T05:29:31.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401882+0000", 2024-09-14T05:29:31.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014307 2024-09-14T05:29:31.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401895+0000", 2024-09-14T05:29:31.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3149000000000001e-05 2024-09-14T05:29:31.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401901+0000", 2024-09-14T05:29:31.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6529999999999997e-06 2024-09-14T05:29:31.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401906+0000", 2024-09-14T05:29:31.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.352e-06 2024-09-14T05:29:31.686 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.686 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.686 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.686 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3450 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.687 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395176+0000", 2024-09-14T05:29:31.687 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0080161339999999994, 2024-09-14T05:29:31.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010300995, 2024-09-14T05:29:31.687 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.687 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.687 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.687 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.687 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.687 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.687 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3450 2024-09-14T05:29:31.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.687 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395176+0000", 2024-09-14T05:29:31.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395176+0000", 2024-09-14T05:29:31.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395177+0000", 2024-09-14T05:29:31.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2900000000000003e-07 2024-09-14T05:29:31.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395179+0000", 2024-09-14T05:29:31.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0329999999999998e-06 2024-09-14T05:29:31.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395185+0000", 2024-09-14T05:29:31.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2299999999999996e-06 2024-09-14T05:29:31.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395190+0000", 2024-09-14T05:29:31.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0259999999999998e-06 2024-09-14T05:29:31.690 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.690 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.690 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.690 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3577 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.690 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.396972+0000", 2024-09-14T05:29:31.690 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0062208209999999996, 2024-09-14T05:29:31.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085402299999999993, 2024-09-14T05:29:31.690 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.690 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.691 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.691 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.691 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.691 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.691 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3577 2024-09-14T05:29:31.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.691 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396972+0000", 2024-09-14T05:29:31.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396972+0000", 2024-09-14T05:29:31.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396972+0000", 2024-09-14T05:29:31.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8699999999999995e-07 2024-09-14T05:29:31.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396974+0000", 2024-09-14T05:29:31.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.852e-06 2024-09-14T05:29:31.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396979+0000", 2024-09-14T05:29:31.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9409999999999998e-06 2024-09-14T05:29:31.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396982+0000", 2024-09-14T05:29:31.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0350000000000002e-06 2024-09-14T05:29:31.693 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.693 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.694 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.694 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3626 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.694 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397724+0000", 2024-09-14T05:29:31.694 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0054682899999999998, 2024-09-14T05:29:31.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078112190000000003, 2024-09-14T05:29:31.694 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.694 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.694 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.694 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.694 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.694 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.695 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3626 2024-09-14T05:29:31.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.695 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397724+0000", 2024-09-14T05:29:31.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397724+0000", 2024-09-14T05:29:31.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397725+0000", 2024-09-14T05:29:31.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6899999999999998e-07 2024-09-14T05:29:31.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397726+0000", 2024-09-14T05:29:31.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.593e-06 2024-09-14T05:29:31.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397731+0000", 2024-09-14T05:29:31.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5929999999999999e-06 2024-09-14T05:29:31.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397734+0000", 2024-09-14T05:29:31.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6230000000000001e-06 2024-09-14T05:29:31.697 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.697 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.697 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.698 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3693 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.698 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398614+0000", 2024-09-14T05:29:31.698 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0045783459999999996, 2024-09-14T05:29:31.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069497029999999998, 2024-09-14T05:29:31.698 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.698 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.698 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.698 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.698 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.698 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.698 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3693 2024-09-14T05:29:31.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.698 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398614+0000", 2024-09-14T05:29:31.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398614+0000", 2024-09-14T05:29:31.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398615+0000", 2024-09-14T05:29:31.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.32e-07 2024-09-14T05:29:31.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398617+0000", 2024-09-14T05:29:31.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0530000000000001e-06 2024-09-14T05:29:31.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398623+0000", 2024-09-14T05:29:31.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0780000000000001e-06 2024-09-14T05:29:31.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398628+0000", 2024-09-14T05:29:31.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8210000000000001e-06 2024-09-14T05:29:31.701 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.701 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.701 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.701 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3742 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.701 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399264+0000", 2024-09-14T05:29:31.701 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0039283319999999997, 2024-09-14T05:29:31.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063226710000000002, 2024-09-14T05:29:31.701 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.701 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.701 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.702 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.702 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.702 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.702 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3742 2024-09-14T05:29:31.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.702 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399264+0000", 2024-09-14T05:29:31.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399264+0000", 2024-09-14T05:29:31.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399265+0000", 2024-09-14T05:29:31.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1600000000000002e-07 2024-09-14T05:29:31.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399270+0000", 2024-09-14T05:29:31.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9620000000000001e-06 2024-09-14T05:29:31.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399276+0000", 2024-09-14T05:29:31.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0549999999999996e-06 2024-09-14T05:29:31.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399280+0000", 2024-09-14T05:29:31.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6299999999999997e-06 2024-09-14T05:29:31.704 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.704 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.704 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.705 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3808 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.705 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399973+0000", 2024-09-14T05:29:31.705 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0032192449999999999, 2024-09-14T05:29:31.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0056375130000000002, 2024-09-14T05:29:31.705 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.705 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.705 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.705 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.705 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.705 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.706 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3808 2024-09-14T05:29:31.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.706 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399973+0000", 2024-09-14T05:29:31.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399973+0000", 2024-09-14T05:29:31.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399974+0000", 2024-09-14T05:29:31.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-14T05:29:31.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399976+0000", 2024-09-14T05:29:31.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8449999999999999e-06 2024-09-14T05:29:31.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399981+0000", 2024-09-14T05:29:31.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8769999999999998e-06 2024-09-14T05:29:31.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399986+0000", 2024-09-14T05:29:31.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6929999999999998e-06 2024-09-14T05:29:31.708 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.708 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.708 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.708 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3857 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.708 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400632+0000", 2024-09-14T05:29:31.708 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00256026, 2024-09-14T05:29:31.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050055560000000004, 2024-09-14T05:29:31.709 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.709 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.709 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.709 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.709 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.709 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.709 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3857 2024-09-14T05:29:31.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.709 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400632+0000", 2024-09-14T05:29:31.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400632+0000", 2024-09-14T05:29:31.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400633+0000", 2024-09-14T05:29:31.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2300000000000002e-07 2024-09-14T05:29:31.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400635+0000", 2024-09-14T05:29:31.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8980000000000001e-06 2024-09-14T05:29:31.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400641+0000", 2024-09-14T05:29:31.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4080000000000003e-06 2024-09-14T05:29:31.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400646+0000", 2024-09-14T05:29:31.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8269999999999997e-06 2024-09-14T05:29:31.711 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.711 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.712 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.712 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3451 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.712 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395217+0000", 2024-09-14T05:29:31.712 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0079757140000000001, 2024-09-14T05:29:31.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010445793, 2024-09-14T05:29:31.712 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.712 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.712 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.712 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.712 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.712 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.713 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3451 2024-09-14T05:29:31.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.713 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395217+0000", 2024-09-14T05:29:31.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395217+0000", 2024-09-14T05:29:31.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395217+0000", 2024-09-14T05:29:31.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9999999999999997e-07 2024-09-14T05:29:31.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395221+0000", 2024-09-14T05:29:31.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.552e-06 2024-09-14T05:29:31.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395228+0000", 2024-09-14T05:29:31.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8569999999999997e-06 2024-09-14T05:29:31.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395232+0000", 2024-09-14T05:29:31.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4739999999999999e-06 2024-09-14T05:29:31.715 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.715 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.715 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.716 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3578 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.716 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.396987+0000", 2024-09-14T05:29:31.716 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0062058850000000004, 2024-09-14T05:29:31.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0087009229999999993, 2024-09-14T05:29:31.716 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.716 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.716 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.716 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.716 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.716 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.716 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3578 2024-09-14T05:29:31.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.717 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396987+0000", 2024-09-14T05:29:31.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396987+0000", 2024-09-14T05:29:31.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396987+0000", 2024-09-14T05:29:31.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4200000000000002e-07 2024-09-14T05:29:31.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396998+0000", 2024-09-14T05:29:31.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0987e-05 2024-09-14T05:29:31.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397003+0000", 2024-09-14T05:29:31.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3680000000000001e-06 2024-09-14T05:29:31.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397007+0000", 2024-09-14T05:29:31.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0219999999999998e-06 2024-09-14T05:29:31.719 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.719 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.719 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.719 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3661 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.719 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397739+0000", 2024-09-14T05:29:31.720 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0054538260000000002, 2024-09-14T05:29:31.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0079747159999999997, 2024-09-14T05:29:31.720 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.720 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.720 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.720 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.720 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.720 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.720 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3661 2024-09-14T05:29:31.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.720 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397739+0000", 2024-09-14T05:29:31.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397739+0000", 2024-09-14T05:29:31.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397739+0000", 2024-09-14T05:29:31.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5899999999999996e-07 2024-09-14T05:29:31.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397742+0000", 2024-09-14T05:29:31.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1499999999999999e-06 2024-09-14T05:29:31.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397748+0000", 2024-09-14T05:29:31.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000003e-06 2024-09-14T05:29:31.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397754+0000", 2024-09-14T05:29:31.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.575e-06 2024-09-14T05:29:31.723 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.723 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.723 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.723 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3694 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.723 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398634+0000", 2024-09-14T05:29:31.723 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0045586719999999997, 2024-09-14T05:29:31.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0071166069999999996, 2024-09-14T05:29:31.723 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.723 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.723 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.724 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.724 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.724 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3694 2024-09-14T05:29:31.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.724 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398634+0000", 2024-09-14T05:29:31.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398634+0000", 2024-09-14T05:29:31.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398634+0000", 2024-09-14T05:29:31.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0499999999999999e-07 2024-09-14T05:29:31.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398638+0000", 2024-09-14T05:29:31.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.676e-06 2024-09-14T05:29:31.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398644+0000", 2024-09-14T05:29:31.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9279999999999998e-06 2024-09-14T05:29:31.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398649+0000", 2024-09-14T05:29:31.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.977e-06 2024-09-14T05:29:31.726 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.726 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.726 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.727 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3743 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.727 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399310+0000", 2024-09-14T05:29:31.727 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0038827670000000001, 2024-09-14T05:29:31.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00647279, 2024-09-14T05:29:31.727 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.727 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.727 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.727 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.727 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.727 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.727 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3743 2024-09-14T05:29:31.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.727 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399310+0000", 2024-09-14T05:29:31.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399310+0000", 2024-09-14T05:29:31.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399310+0000", 2024-09-14T05:29:31.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7100000000000002e-07 2024-09-14T05:29:31.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399314+0000", 2024-09-14T05:29:31.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8160000000000004e-06 2024-09-14T05:29:31.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399321+0000", 2024-09-14T05:29:31.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7619999999999998e-06 2024-09-14T05:29:31.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399327+0000", 2024-09-14T05:29:31.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7400000000000001e-06 2024-09-14T05:29:31.730 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.730 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.730 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.730 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3809 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.730 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399992+0000", 2024-09-14T05:29:31.730 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0032008660000000001, 2024-09-14T05:29:31.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058210859999999996, 2024-09-14T05:29:31.730 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.730 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.730 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.731 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.731 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.731 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.731 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3809 2024-09-14T05:29:31.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.731 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399992+0000", 2024-09-14T05:29:31.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399992+0000", 2024-09-14T05:29:31.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399992+0000", 2024-09-14T05:29:31.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7899999999999998e-07 2024-09-14T05:29:31.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399996+0000", 2024-09-14T05:29:31.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2789999999999999e-06 2024-09-14T05:29:31.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400002+0000", 2024-09-14T05:29:31.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1070000000000001e-06 2024-09-14T05:29:31.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400007+0000", 2024-09-14T05:29:31.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9450000000000001e-06 2024-09-14T05:29:31.733 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.733 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.733 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.733 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3858 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.734 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400652+0000", 2024-09-14T05:29:31.734 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0025405129999999999, 2024-09-14T05:29:31.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052012409999999997, 2024-09-14T05:29:31.734 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.734 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.734 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.734 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.734 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.734 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.734 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3858 2024-09-14T05:29:31.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.734 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400652+0000", 2024-09-14T05:29:31.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400652+0000", 2024-09-14T05:29:31.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400652+0000", 2024-09-14T05:29:31.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4299999999999998e-07 2024-09-14T05:29:31.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400655+0000", 2024-09-14T05:29:31.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5600000000000001e-06 2024-09-14T05:29:31.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400661+0000", 2024-09-14T05:29:31.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5809999999999996e-06 2024-09-14T05:29:31.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400665+0000", 2024-09-14T05:29:31.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6290000000000001e-06 2024-09-14T05:29:31.737 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.737 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.737 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.737 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3452 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.737 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395259+0000", 2024-09-14T05:29:31.737 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0079338949999999998, 2024-09-14T05:29:31.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010621223000000001, 2024-09-14T05:29:31.737 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.738 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.738 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.738 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.738 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.738 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.738 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3452 2024-09-14T05:29:31.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.738 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395259+0000", 2024-09-14T05:29:31.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395259+0000", 2024-09-14T05:29:31.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395259+0000", 2024-09-14T05:29:31.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0200000000000002e-07 2024-09-14T05:29:31.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395261+0000", 2024-09-14T05:29:31.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1059999999999998e-06 2024-09-14T05:29:31.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395267+0000", 2024-09-14T05:29:31.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6359999999999999e-06 2024-09-14T05:29:31.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395273+0000", 2024-09-14T05:29:31.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6799999999999998e-06 2024-09-14T05:29:31.740 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.740 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.740 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.741 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3579 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.741 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397012+0000", 2024-09-14T05:29:31.741 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0061806980000000001, 2024-09-14T05:29:31.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0088907250000000004, 2024-09-14T05:29:31.741 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.741 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.741 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.741 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3579 2024-09-14T05:29:31.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.741 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397012+0000", 2024-09-14T05:29:31.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397012+0000", 2024-09-14T05:29:31.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397012+0000", 2024-09-14T05:29:31.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5600000000000001e-07 2024-09-14T05:29:31.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397013+0000", 2024-09-14T05:29:31.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1939999999999999e-06 2024-09-14T05:29:31.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397017+0000", 2024-09-14T05:29:31.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9299999999999996e-06 2024-09-14T05:29:31.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397020+0000", 2024-09-14T05:29:31.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7439999999999999e-06 2024-09-14T05:29:31.744 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.744 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.744 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.744 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3662 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.744 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397760+0000", 2024-09-14T05:29:31.744 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0054329299999999999, 2024-09-14T05:29:31.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081766440000000003, 2024-09-14T05:29:31.744 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.744 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.745 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.745 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.745 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.745 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.745 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3662 2024-09-14T05:29:31.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.745 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397760+0000", 2024-09-14T05:29:31.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397760+0000", 2024-09-14T05:29:31.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397760+0000", 2024-09-14T05:29:31.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-14T05:29:31.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397762+0000", 2024-09-14T05:29:31.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.759e-06 2024-09-14T05:29:31.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397768+0000", 2024-09-14T05:29:31.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.314e-06 2024-09-14T05:29:31.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397772+0000", 2024-09-14T05:29:31.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2300000000000002e-06 2024-09-14T05:29:31.747 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.747 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.747 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.748 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3695 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.748 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398655+0000", 2024-09-14T05:29:31.748 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0045375140000000003, 2024-09-14T05:29:31.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0073201509999999996, 2024-09-14T05:29:31.748 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.748 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.748 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.748 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.748 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.748 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.748 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3695 2024-09-14T05:29:31.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.749 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398655+0000", 2024-09-14T05:29:31.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398655+0000", 2024-09-14T05:29:31.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398656+0000", 2024-09-14T05:29:31.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8699999999999995e-07 2024-09-14T05:29:31.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398657+0000", 2024-09-14T05:29:31.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.891e-06 2024-09-14T05:29:31.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398664+0000", 2024-09-14T05:29:31.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.088e-06 2024-09-14T05:29:31.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398668+0000", 2024-09-14T05:29:31.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999998e-06 2024-09-14T05:29:31.751 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.751 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.751 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.751 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3778 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.751 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399333+0000", 2024-09-14T05:29:31.752 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.003859127, 2024-09-14T05:29:31.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066743660000000002, 2024-09-14T05:29:31.752 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.752 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.752 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.752 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.752 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.752 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.752 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3778 2024-09-14T05:29:31.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.752 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399333+0000", 2024-09-14T05:29:31.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399333+0000", 2024-09-14T05:29:31.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399334+0000", 2024-09-14T05:29:31.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7300000000000001e-07 2024-09-14T05:29:31.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399336+0000", 2024-09-14T05:29:31.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9290000000000001e-06 2024-09-14T05:29:31.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399342+0000", 2024-09-14T05:29:31.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1179999999999997e-06 2024-09-14T05:29:31.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399348+0000", 2024-09-14T05:29:31.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.716e-06 2024-09-14T05:29:31.755 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.755 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.755 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.755 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3810 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.755 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400013+0000", 2024-09-14T05:29:31.755 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0031794369999999998, 2024-09-14T05:29:31.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060175389999999997, 2024-09-14T05:29:31.755 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.756 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.756 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.756 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.756 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.756 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.756 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3810 2024-09-14T05:29:31.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.756 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400013+0000", 2024-09-14T05:29:31.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400013+0000", 2024-09-14T05:29:31.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400014+0000", 2024-09-14T05:29:31.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6899999999999998e-07 2024-09-14T05:29:31.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400015+0000", 2024-09-14T05:29:31.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7409999999999999e-06 2024-09-14T05:29:31.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400021+0000", 2024-09-14T05:29:31.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9179999999999999e-06 2024-09-14T05:29:31.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400026+0000", 2024-09-14T05:29:31.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7910000000000004e-06 2024-09-14T05:29:31.758 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.758 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.758 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.759 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3859 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.759 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400671+0000", 2024-09-14T05:29:31.759 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0025217, 2024-09-14T05:29:31.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053840210000000001, 2024-09-14T05:29:31.759 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.759 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.759 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.759 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.759 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.759 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.760 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3859 2024-09-14T05:29:31.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.760 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400671+0000", 2024-09-14T05:29:31.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400671+0000", 2024-09-14T05:29:31.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400671+0000", 2024-09-14T05:29:31.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7199999999999999e-07 2024-09-14T05:29:31.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400673+0000", 2024-09-14T05:29:31.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.824e-06 2024-09-14T05:29:31.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400681+0000", 2024-09-14T05:29:31.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3499999999999999e-06 2024-09-14T05:29:31.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400686+0000", 2024-09-14T05:29:31.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1610000000000002e-06 2024-09-14T05:29:31.762 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.762 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.762 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.762 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3453 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.762 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395299+0000", 2024-09-14T05:29:31.762 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0078938629999999992, 2024-09-14T05:29:31.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010783054, 2024-09-14T05:29:31.762 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.763 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.763 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.763 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.763 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.763 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.763 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3453 2024-09-14T05:29:31.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.763 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395299+0000", 2024-09-14T05:29:31.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395299+0000", 2024-09-14T05:29:31.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395299+0000", 2024-09-14T05:29:31.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1800000000000002e-07 2024-09-14T05:29:31.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395301+0000", 2024-09-14T05:29:31.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.522e-06 2024-09-14T05:29:31.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395305+0000", 2024-09-14T05:29:31.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2250000000000002e-06 2024-09-14T05:29:31.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395308+0000", 2024-09-14T05:29:31.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9210000000000001e-06 2024-09-14T05:29:31.765 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.766 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.766 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.766 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3580 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.766 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397024+0000", 2024-09-14T05:29:31.766 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0061689739999999998, 2024-09-14T05:29:31.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091052430000000007, 2024-09-14T05:29:31.766 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.766 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.766 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.766 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.766 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.766 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.767 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3580 2024-09-14T05:29:31.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.767 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397024+0000", 2024-09-14T05:29:31.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397024+0000", 2024-09-14T05:29:31.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397024+0000", 2024-09-14T05:29:31.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4799999999999999e-07 2024-09-14T05:29:31.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397025+0000", 2024-09-14T05:29:31.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.048e-06 2024-09-14T05:29:31.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397028+0000", 2024-09-14T05:29:31.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1240000000000001e-06 2024-09-14T05:29:31.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.769 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397031+0000", 2024-09-14T05:29:31.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5000000000000002e-06 2024-09-14T05:29:31.769 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.769 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.769 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.769 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3663 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.769 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397777+0000", 2024-09-14T05:29:31.769 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0054151609999999999, 2024-09-14T05:29:31.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084096210000000008, 2024-09-14T05:29:31.770 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.770 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.770 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.770 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3663 2024-09-14T05:29:31.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.770 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397777+0000", 2024-09-14T05:29:31.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397777+0000", 2024-09-14T05:29:31.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397778+0000", 2024-09-14T05:29:31.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4400000000000001e-07 2024-09-14T05:29:31.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397779+0000", 2024-09-14T05:29:31.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1990000000000001e-06 2024-09-14T05:29:31.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397782+0000", 2024-09-14T05:29:31.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.427e-06 2024-09-14T05:29:31.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397786+0000", 2024-09-14T05:29:31.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1899999999999997e-06 2024-09-14T05:29:31.772 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.772 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.773 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.773 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3696 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.773 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398674+0000", 2024-09-14T05:29:31.773 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0045184279999999997, 2024-09-14T05:29:31.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.007561379, 2024-09-14T05:29:31.773 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.773 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.773 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.773 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.773 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.773 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.773 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3696 2024-09-14T05:29:31.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.774 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398674+0000", 2024-09-14T05:29:31.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398674+0000", 2024-09-14T05:29:31.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398675+0000", 2024-09-14T05:29:31.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5299999999999999e-07 2024-09-14T05:29:31.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398676+0000", 2024-09-14T05:29:31.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.739e-06 2024-09-14T05:29:31.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398682+0000", 2024-09-14T05:29:31.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8619999999999999e-06 2024-09-14T05:29:31.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398687+0000", 2024-09-14T05:29:31.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8779999999999997e-06 2024-09-14T05:29:31.776 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.776 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.776 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.776 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3779 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.776 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399354+0000", 2024-09-14T05:29:31.776 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0038387450000000002, 2024-09-14T05:29:31.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069091100000000004, 2024-09-14T05:29:31.776 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.776 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.776 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.776 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.776 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.777 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.777 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3779 2024-09-14T05:29:31.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.777 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399354+0000", 2024-09-14T05:29:31.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399354+0000", 2024-09-14T05:29:31.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399354+0000", 2024-09-14T05:29:31.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7e-07 2024-09-14T05:29:31.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399356+0000", 2024-09-14T05:29:31.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9360000000000002e-06 2024-09-14T05:29:31.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399362+0000", 2024-09-14T05:29:31.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.302e-06 2024-09-14T05:29:31.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399367+0000", 2024-09-14T05:29:31.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7960000000000003e-06 2024-09-14T05:29:31.779 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.779 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.779 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.779 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3811 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.779 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400032+0000", 2024-09-14T05:29:31.779 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0031603120000000002, 2024-09-14T05:29:31.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062645560000000001, 2024-09-14T05:29:31.779 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.780 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.780 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.780 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.780 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.780 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.780 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3811 2024-09-14T05:29:31.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.780 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400032+0000", 2024-09-14T05:29:31.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400032+0000", 2024-09-14T05:29:31.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400033+0000", 2024-09-14T05:29:31.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:29:31.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400034+0000", 2024-09-14T05:29:31.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8899999999999999e-06 2024-09-14T05:29:31.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400040+0000", 2024-09-14T05:29:31.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7690000000000002e-06 2024-09-14T05:29:31.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400045+0000", 2024-09-14T05:29:31.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9609999999999996e-06 2024-09-14T05:29:31.782 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.782 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.782 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.783 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3894 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.783 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400692+0000", 2024-09-14T05:29:31.783 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.002500525, 2024-09-14T05:29:31.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0056344710000000003, 2024-09-14T05:29:31.783 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.783 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.783 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.783 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.783 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.783 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.783 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3894 2024-09-14T05:29:31.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.784 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400692+0000", 2024-09-14T05:29:31.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400692+0000", 2024-09-14T05:29:31.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400692+0000", 2024-09-14T05:29:31.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.15e-07 2024-09-14T05:29:31.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400694+0000", 2024-09-14T05:29:31.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9920000000000002e-06 2024-09-14T05:29:31.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400700+0000", 2024-09-14T05:29:31.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7069999999999997e-06 2024-09-14T05:29:31.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400705+0000", 2024-09-14T05:29:31.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6589999999999998e-06 2024-09-14T05:29:31.786 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.786 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.786 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.786 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3454 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.786 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395313+0000", 2024-09-14T05:29:31.786 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0078796609999999996, 2024-09-14T05:29:31.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011038806999999999, 2024-09-14T05:29:31.786 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.787 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.787 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.787 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.787 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.787 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.787 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3454 2024-09-14T05:29:31.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.787 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395313+0000", 2024-09-14T05:29:31.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395313+0000", 2024-09-14T05:29:31.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395313+0000", 2024-09-14T05:29:31.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7599999999999997e-07 2024-09-14T05:29:31.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395315+0000", 2024-09-14T05:29:31.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7239999999999999e-06 2024-09-14T05:29:31.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395320+0000", 2024-09-14T05:29:31.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.899e-06 2024-09-14T05:29:31.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395325+0000", 2024-09-14T05:29:31.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3550000000000004e-06 2024-09-14T05:29:31.789 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.790 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.790 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.790 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3581 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.790 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397034+0000", 2024-09-14T05:29:31.790 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.006158167, 2024-09-14T05:29:31.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0093399799999999995, 2024-09-14T05:29:31.790 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.790 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.790 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.791 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.791 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.791 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3581 2024-09-14T05:29:31.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.791 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397034+0000", 2024-09-14T05:29:31.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397034+0000", 2024-09-14T05:29:31.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397035+0000", 2024-09-14T05:29:31.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0800000000000001e-07 2024-09-14T05:29:31.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397036+0000", 2024-09-14T05:29:31.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.133e-06 2024-09-14T05:29:31.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397039+0000", 2024-09-14T05:29:31.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1590000000000002e-06 2024-09-14T05:29:31.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397041+0000", 2024-09-14T05:29:31.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5890000000000001e-06 2024-09-14T05:29:31.793 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.793 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.793 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.794 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3664 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.794 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397792+0000", 2024-09-14T05:29:31.794 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0054007170000000002, 2024-09-14T05:29:31.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0086049509999999996, 2024-09-14T05:29:31.794 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.794 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.794 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.794 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.794 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.794 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.794 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3664 2024-09-14T05:29:31.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.795 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397792+0000", 2024-09-14T05:29:31.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397792+0000", 2024-09-14T05:29:31.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397792+0000", 2024-09-14T05:29:31.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3000000000000002e-07 2024-09-14T05:29:31.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397794+0000", 2024-09-14T05:29:31.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6300000000000001e-06 2024-09-14T05:29:31.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397807+0000", 2024-09-14T05:29:31.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3285000000000001e-05 2024-09-14T05:29:31.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397832+0000", 2024-09-14T05:29:31.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.527e-05 2024-09-14T05:29:31.797 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.797 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.797 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.797 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3697 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.797 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398693+0000", 2024-09-14T05:29:31.798 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0044994859999999996, 2024-09-14T05:29:31.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0077450840000000002, 2024-09-14T05:29:31.798 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.798 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.798 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.798 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.798 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.798 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.798 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3697 2024-09-14T05:29:31.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.798 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398693+0000", 2024-09-14T05:29:31.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398693+0000", 2024-09-14T05:29:31.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398693+0000", 2024-09-14T05:29:31.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.84e-07 2024-09-14T05:29:31.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398695+0000", 2024-09-14T05:29:31.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.708e-06 2024-09-14T05:29:31.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398701+0000", 2024-09-14T05:29:31.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5300000000000004e-06 2024-09-14T05:29:31.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398705+0000", 2024-09-14T05:29:31.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5920000000000002e-06 2024-09-14T05:29:31.801 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.801 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.801 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.801 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3780 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.801 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399373+0000", 2024-09-14T05:29:31.801 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0038194980000000002, 2024-09-14T05:29:31.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0070887800000000003, 2024-09-14T05:29:31.801 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.801 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.802 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.802 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.802 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.802 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.802 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3780 2024-09-14T05:29:31.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.802 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399373+0000", 2024-09-14T05:29:31.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399373+0000", 2024-09-14T05:29:31.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399374+0000", 2024-09-14T05:29:31.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8800000000000003e-07 2024-09-14T05:29:31.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399375+0000", 2024-09-14T05:29:31.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9589999999999998e-06 2024-09-14T05:29:31.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399381+0000", 2024-09-14T05:29:31.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9259999999999997e-06 2024-09-14T05:29:31.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399386+0000", 2024-09-14T05:29:31.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5070000000000002e-06 2024-09-14T05:29:31.804 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.804 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.805 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.805 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3812 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.805 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400060+0000", 2024-09-14T05:29:31.805 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0031323710000000001, 2024-09-14T05:29:31.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0064308569999999999, 2024-09-14T05:29:31.805 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.805 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.805 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.805 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.805 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.805 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.805 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3812 2024-09-14T05:29:31.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.806 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400060+0000", 2024-09-14T05:29:31.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400060+0000", 2024-09-14T05:29:31.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400061+0000", 2024-09-14T05:29:31.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7800000000000002e-07 2024-09-14T05:29:31.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400067+0000", 2024-09-14T05:29:31.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0639999999999999e-06 2024-09-14T05:29:31.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400073+0000", 2024-09-14T05:29:31.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4710000000000004e-06 2024-09-14T05:29:31.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400078+0000", 2024-09-14T05:29:31.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1200000000000001e-06 2024-09-14T05:29:31.808 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.808 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.808 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.808 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3895 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.808 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400711+0000", 2024-09-14T05:29:31.809 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0024818819999999999, 2024-09-14T05:29:31.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058072419999999998, 2024-09-14T05:29:31.809 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.809 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.809 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.809 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3895 2024-09-14T05:29:31.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.809 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400711+0000", 2024-09-14T05:29:31.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400711+0000", 2024-09-14T05:29:31.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400711+0000", 2024-09-14T05:29:31.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8699999999999995e-07 2024-09-14T05:29:31.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400719+0000", 2024-09-14T05:29:31.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6930000000000007e-06 2024-09-14T05:29:31.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400725+0000", 2024-09-14T05:29:31.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0530000000000003e-06 2024-09-14T05:29:31.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400730+0000", 2024-09-14T05:29:31.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7299999999999996e-06 2024-09-14T05:29:31.812 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.812 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.812 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.812 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3455 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.812 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395340+0000", 2024-09-14T05:29:31.812 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0078527219999999995, 2024-09-14T05:29:31.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011232595999999999, 2024-09-14T05:29:31.812 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.812 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.813 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.813 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3455 2024-09-14T05:29:31.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.813 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395340+0000", 2024-09-14T05:29:31.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395340+0000", 2024-09-14T05:29:31.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395340+0000", 2024-09-14T05:29:31.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0500000000000004e-07 2024-09-14T05:29:31.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395346+0000", 2024-09-14T05:29:31.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3319999999999999e-06 2024-09-14T05:29:31.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395352+0000", 2024-09-14T05:29:31.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2899999999999999e-06 2024-09-14T05:29:31.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395357+0000", 2024-09-14T05:29:31.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2920000000000003e-06 2024-09-14T05:29:31.815 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.815 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.815 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.816 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3582 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.816 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397045+0000", 2024-09-14T05:29:31.816 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0061480019999999996, 2024-09-14T05:29:31.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0095742420000000002, 2024-09-14T05:29:31.816 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.816 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.816 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.816 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.816 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.816 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.816 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3582 2024-09-14T05:29:31.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.817 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397045+0000", 2024-09-14T05:29:31.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397045+0000", 2024-09-14T05:29:31.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397045+0000", 2024-09-14T05:29:31.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0599999999999999e-07 2024-09-14T05:29:31.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397046+0000", 2024-09-14T05:29:31.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9000000000000005e-07 2024-09-14T05:29:31.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397049+0000", 2024-09-14T05:29:31.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.999e-06 2024-09-14T05:29:31.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397051+0000", 2024-09-14T05:29:31.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6189999999999998e-06 2024-09-14T05:29:31.819 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.819 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.819 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.819 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3665 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.819 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397844+0000", 2024-09-14T05:29:31.819 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0053483300000000001, 2024-09-14T05:29:31.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0088305880000000003, 2024-09-14T05:29:31.820 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.820 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.820 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.820 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.820 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.820 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.820 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3665 2024-09-14T05:29:31.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.820 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397844+0000", 2024-09-14T05:29:31.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397844+0000", 2024-09-14T05:29:31.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397845+0000", 2024-09-14T05:29:31.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7199999999999998e-07 2024-09-14T05:29:31.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397857+0000", 2024-09-14T05:29:31.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2339e-05 2024-09-14T05:29:31.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397866+0000", 2024-09-14T05:29:31.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4910000000000005e-06 2024-09-14T05:29:31.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397873+0000", 2024-09-14T05:29:31.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8979999999999999e-06 2024-09-14T05:29:31.822 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.823 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.823 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.823 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3698 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.823 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398711+0000", 2024-09-14T05:29:31.823 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0044815130000000003, 2024-09-14T05:29:31.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080149030000000003, 2024-09-14T05:29:31.823 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.823 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.823 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.823 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.823 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.823 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.824 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3698 2024-09-14T05:29:31.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.824 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398711+0000", 2024-09-14T05:29:31.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398711+0000", 2024-09-14T05:29:31.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398711+0000", 2024-09-14T05:29:31.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:29:31.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398713+0000", 2024-09-14T05:29:31.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.888e-06 2024-09-14T05:29:31.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398719+0000", 2024-09-14T05:29:31.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-14T05:29:31.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398724+0000", 2024-09-14T05:29:31.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8180000000000003e-06 2024-09-14T05:29:31.826 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.826 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.826 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.826 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3781 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.826 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399392+0000", 2024-09-14T05:29:31.826 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.003800795, 2024-09-14T05:29:31.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0073658459999999997, 2024-09-14T05:29:31.827 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.827 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.827 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.827 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.827 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.827 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.827 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3781 2024-09-14T05:29:31.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.827 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399392+0000", 2024-09-14T05:29:31.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399392+0000", 2024-09-14T05:29:31.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399392+0000", 2024-09-14T05:29:31.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4499999999999998e-07 2024-09-14T05:29:31.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399394+0000", 2024-09-14T05:29:31.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.73e-06 2024-09-14T05:29:31.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399400+0000", 2024-09-14T05:29:31.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8479999999999997e-06 2024-09-14T05:29:31.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399404+0000", 2024-09-14T05:29:31.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7339999999999999e-06 2024-09-14T05:29:31.830 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.830 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.830 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.830 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3813 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.830 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400085+0000", 2024-09-14T05:29:31.830 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0031073680000000001, 2024-09-14T05:29:31.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0067063560000000001, 2024-09-14T05:29:31.830 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.830 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.830 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.831 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.831 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.831 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3813 2024-09-14T05:29:31.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.831 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400085+0000", 2024-09-14T05:29:31.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400085+0000", 2024-09-14T05:29:31.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400086+0000", 2024-09-14T05:29:31.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4799999999999999e-07 2024-09-14T05:29:31.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400088+0000", 2024-09-14T05:29:31.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.917e-06 2024-09-14T05:29:31.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400093+0000", 2024-09-14T05:29:31.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7420000000000003e-06 2024-09-14T05:29:31.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400098+0000", 2024-09-14T05:29:31.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.758e-06 2024-09-14T05:29:31.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.833 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.833 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3896 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.833 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400736+0000", 2024-09-14T05:29:31.833 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0024567909999999998, 2024-09-14T05:29:31.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060917519999999998, 2024-09-14T05:29:31.834 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.834 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.834 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.834 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3896 2024-09-14T05:29:31.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.834 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400736+0000", 2024-09-14T05:29:31.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400736+0000", 2024-09-14T05:29:31.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400736+0000", 2024-09-14T05:29:31.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1500000000000005e-07 2024-09-14T05:29:31.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400738+0000", 2024-09-14T05:29:31.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9640000000000002e-06 2024-09-14T05:29:31.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400744+0000", 2024-09-14T05:29:31.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0499999999999997e-06 2024-09-14T05:29:31.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400749+0000", 2024-09-14T05:29:31.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7330000000000003e-06 2024-09-14T05:29:31.836 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.836 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.837 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.837 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3456 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.837 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395365+0000", 2024-09-14T05:29:31.837 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0078271580000000007, 2024-09-14T05:29:31.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011488272000000001, 2024-09-14T05:29:31.837 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.837 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.837 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.837 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.837 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.837 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.837 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3456 2024-09-14T05:29:31.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.838 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395365+0000", 2024-09-14T05:29:31.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395365+0000", 2024-09-14T05:29:31.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395366+0000", 2024-09-14T05:29:31.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8999999999999996e-07 2024-09-14T05:29:31.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395368+0000", 2024-09-14T05:29:31.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1890000000000001e-06 2024-09-14T05:29:31.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395374+0000", 2024-09-14T05:29:31.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8449999999999999e-06 2024-09-14T05:29:31.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395378+0000", 2024-09-14T05:29:31.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4390000000000002e-06 2024-09-14T05:29:31.840 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.840 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.840 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.840 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3583 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.840 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397055+0000", 2024-09-14T05:29:31.840 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.006137956, 2024-09-14T05:29:31.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0098351229999999994, 2024-09-14T05:29:31.840 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.841 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.841 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.841 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.841 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.841 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.841 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3583 2024-09-14T05:29:31.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.841 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397055+0000", 2024-09-14T05:29:31.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397055+0000", 2024-09-14T05:29:31.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397055+0000", 2024-09-14T05:29:31.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3200000000000001e-07 2024-09-14T05:29:31.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397056+0000", 2024-09-14T05:29:31.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.536e-06 2024-09-14T05:29:31.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397062+0000", 2024-09-14T05:29:31.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8050000000000003e-06 2024-09-14T05:29:31.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397066+0000", 2024-09-14T05:29:31.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2479999999999998e-06 2024-09-14T05:29:31.843 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.843 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.843 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.844 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3666 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.844 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397882+0000", 2024-09-14T05:29:31.844 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0053108410000000002, 2024-09-14T05:29:31.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0090419980000000007, 2024-09-14T05:29:31.844 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.844 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.844 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.844 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.844 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.844 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.844 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3666 2024-09-14T05:29:31.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.845 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397882+0000", 2024-09-14T05:29:31.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397882+0000", 2024-09-14T05:29:31.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397882+0000", 2024-09-14T05:29:31.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4700000000000001e-07 2024-09-14T05:29:31.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397885+0000", 2024-09-14T05:29:31.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.294e-06 2024-09-14T05:29:31.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397892+0000", 2024-09-14T05:29:31.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9659999999999999e-06 2024-09-14T05:29:31.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397897+0000", 2024-09-14T05:29:31.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6540000000000004e-06 2024-09-14T05:29:31.847 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.847 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.847 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.847 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3699 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.847 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398730+0000", 2024-09-14T05:29:31.847 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0044626179999999998, 2024-09-14T05:29:31.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082332540000000006, 2024-09-14T05:29:31.847 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.847 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.848 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.848 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.848 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.848 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.848 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3699 2024-09-14T05:29:31.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.848 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398730+0000", 2024-09-14T05:29:31.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398730+0000", 2024-09-14T05:29:31.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398730+0000", 2024-09-14T05:29:31.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6499999999999999e-07 2024-09-14T05:29:31.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398732+0000", 2024-09-14T05:29:31.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9700000000000002e-06 2024-09-14T05:29:31.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398739+0000", 2024-09-14T05:29:31.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9859999999999997e-06 2024-09-14T05:29:31.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398744+0000", 2024-09-14T05:29:31.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.707e-06 2024-09-14T05:29:31.850 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.850 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.850 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.851 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3782 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.851 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399411+0000", 2024-09-14T05:29:31.851 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0037818449999999998, 2024-09-14T05:29:31.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075972289999999996, 2024-09-14T05:29:31.851 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.851 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.851 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.851 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.851 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.851 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.851 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3782 2024-09-14T05:29:31.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.851 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399411+0000", 2024-09-14T05:29:31.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399411+0000", 2024-09-14T05:29:31.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399411+0000", 2024-09-14T05:29:31.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2499999999999995e-07 2024-09-14T05:29:31.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399413+0000", 2024-09-14T05:29:31.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9149999999999999e-06 2024-09-14T05:29:31.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399419+0000", 2024-09-14T05:29:31.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6030000000000004e-06 2024-09-14T05:29:31.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399424+0000", 2024-09-14T05:29:31.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7809999999999996e-06 2024-09-14T05:29:31.854 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.854 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.854 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.854 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3814 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.854 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400104+0000", 2024-09-14T05:29:31.854 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0030888589999999998, 2024-09-14T05:29:31.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069521649999999997, 2024-09-14T05:29:31.854 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.854 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.855 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.855 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.855 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.855 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.855 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3814 2024-09-14T05:29:31.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.855 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400104+0000", 2024-09-14T05:29:31.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400104+0000", 2024-09-14T05:29:31.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400104+0000", 2024-09-14T05:29:31.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:29:31.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400106+0000", 2024-09-14T05:29:31.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9709999999999998e-06 2024-09-14T05:29:31.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400112+0000", 2024-09-14T05:29:31.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8730000000000004e-06 2024-09-14T05:29:31.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400117+0000", 2024-09-14T05:29:31.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9429999999999999e-06 2024-09-14T05:29:31.857 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.857 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.857 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.858 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3897 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.858 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400755+0000", 2024-09-14T05:29:31.858 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0024376229999999999, 2024-09-14T05:29:31.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063399989999999998, 2024-09-14T05:29:31.858 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.858 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.858 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.858 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.858 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.858 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.858 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3897 2024-09-14T05:29:31.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.859 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400755+0000", 2024-09-14T05:29:31.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400755+0000", 2024-09-14T05:29:31.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400755+0000", 2024-09-14T05:29:31.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.01e-07 2024-09-14T05:29:31.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400757+0000", 2024-09-14T05:29:31.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.809e-06 2024-09-14T05:29:31.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400763+0000", 2024-09-14T05:29:31.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9179999999999999e-06 2024-09-14T05:29:31.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400768+0000", 2024-09-14T05:29:31.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6140000000000002e-06 2024-09-14T05:29:31.861 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.861 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.861 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.861 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3457 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.861 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395384+0000", 2024-09-14T05:29:31.861 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0078084640000000002, 2024-09-14T05:29:31.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011735020000000001, 2024-09-14T05:29:31.862 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.862 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.862 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.862 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.862 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.862 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.862 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3457 2024-09-14T05:29:31.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.862 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395384+0000", 2024-09-14T05:29:31.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395384+0000", 2024-09-14T05:29:31.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395384+0000", 2024-09-14T05:29:31.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7799999999999997e-07 2024-09-14T05:29:31.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395386+0000", 2024-09-14T05:29:31.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5340000000000001e-06 2024-09-14T05:29:31.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395391+0000", 2024-09-14T05:29:31.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4689999999999996e-06 2024-09-14T05:29:31.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395395+0000", 2024-09-14T05:29:31.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.326e-06 2024-09-14T05:29:31.864 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.864 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.864 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.865 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3584 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.865 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397117+0000", 2024-09-14T05:29:31.865 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0060755890000000002, 2024-09-14T05:29:31.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010027761, 2024-09-14T05:29:31.865 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.865 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.865 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.865 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.865 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.865 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.865 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3584 2024-09-14T05:29:31.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.866 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397117+0000", 2024-09-14T05:29:31.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397117+0000", 2024-09-14T05:29:31.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397118+0000", 2024-09-14T05:29:31.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2500000000000004e-07 2024-09-14T05:29:31.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397121+0000", 2024-09-14T05:29:31.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4750000000000002e-06 2024-09-14T05:29:31.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397130+0000", 2024-09-14T05:29:31.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2429999999999994e-06 2024-09-14T05:29:31.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397137+0000", 2024-09-14T05:29:31.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.455e-06 2024-09-14T05:29:31.868 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.868 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.868 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.868 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3667 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.868 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397904+0000", 2024-09-14T05:29:31.868 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0052885429999999997, 2024-09-14T05:29:31.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0092641419999999995, 2024-09-14T05:29:31.868 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.868 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.869 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.869 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.869 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.869 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.869 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3667 2024-09-14T05:29:31.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.869 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397904+0000", 2024-09-14T05:29:31.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397904+0000", 2024-09-14T05:29:31.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397905+0000", 2024-09-14T05:29:31.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4099999999999999e-07 2024-09-14T05:29:31.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397906+0000", 2024-09-14T05:29:31.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.857e-06 2024-09-14T05:29:31.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397913+0000", 2024-09-14T05:29:31.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.754e-06 2024-09-14T05:29:31.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397919+0000", 2024-09-14T05:29:31.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3859999999999998e-06 2024-09-14T05:29:31.871 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.871 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.871 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.872 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3700 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.872 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398752+0000", 2024-09-14T05:29:31.872 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00444079, 2024-09-14T05:29:31.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084480700000000002, 2024-09-14T05:29:31.872 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.872 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.872 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.872 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.872 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.872 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.872 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3700 2024-09-14T05:29:31.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.873 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398752+0000", 2024-09-14T05:29:31.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398752+0000", 2024-09-14T05:29:31.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398752+0000", 2024-09-14T05:29:31.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7100000000000002e-07 2024-09-14T05:29:31.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398754+0000", 2024-09-14T05:29:31.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0530000000000001e-06 2024-09-14T05:29:31.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398760+0000", 2024-09-14T05:29:31.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.947e-06 2024-09-14T05:29:31.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398765+0000", 2024-09-14T05:29:31.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7589999999999997e-06 2024-09-14T05:29:31.875 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.875 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.875 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.875 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3783 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.875 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399432+0000", 2024-09-14T05:29:31.875 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.003760579, 2024-09-14T05:29:31.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.007791928, 2024-09-14T05:29:31.876 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.876 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.876 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.876 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.876 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.876 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.876 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3783 2024-09-14T05:29:31.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.876 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399432+0000", 2024-09-14T05:29:31.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399432+0000", 2024-09-14T05:29:31.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399432+0000", 2024-09-14T05:29:31.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1199999999999998e-07 2024-09-14T05:29:31.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399435+0000", 2024-09-14T05:29:31.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2060000000000001e-06 2024-09-14T05:29:31.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399441+0000", 2024-09-14T05:29:31.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0050000000000001e-06 2024-09-14T05:29:31.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399446+0000", 2024-09-14T05:29:31.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9849999999999997e-06 2024-09-14T05:29:31.878 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.878 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.878 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.879 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3815 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.879 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400123+0000", 2024-09-14T05:29:31.879 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0030695179999999998, 2024-09-14T05:29:31.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0071252779999999996, 2024-09-14T05:29:31.879 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.879 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.879 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.879 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3815 2024-09-14T05:29:31.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.880 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400123+0000", 2024-09-14T05:29:31.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400123+0000", 2024-09-14T05:29:31.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400123+0000", 2024-09-14T05:29:31.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3700000000000001e-07 2024-09-14T05:29:31.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400125+0000", 2024-09-14T05:29:31.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9130000000000001e-06 2024-09-14T05:29:31.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400132+0000", 2024-09-14T05:29:31.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6880000000000002e-06 2024-09-14T05:29:31.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400136+0000", 2024-09-14T05:29:31.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4100000000000001e-06 2024-09-14T05:29:31.882 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.882 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.882 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.882 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3898 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.882 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400773+0000", 2024-09-14T05:29:31.882 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0024190349999999999, 2024-09-14T05:29:31.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0064972720000000001, 2024-09-14T05:29:31.882 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.883 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.883 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.883 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.883 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.883 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.883 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3898 2024-09-14T05:29:31.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.883 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400773+0000", 2024-09-14T05:29:31.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400773+0000", 2024-09-14T05:29:31.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400774+0000", 2024-09-14T05:29:31.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3099999999999998e-07 2024-09-14T05:29:31.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400776+0000", 2024-09-14T05:29:31.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.739e-06 2024-09-14T05:29:31.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400782+0000", 2024-09-14T05:29:31.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2519999999999996e-06 2024-09-14T05:29:31.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400790+0000", 2024-09-14T05:29:31.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8359999999999992e-06 2024-09-14T05:29:31.885 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.885 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.885 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.886 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3458 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.886 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395398+0000", 2024-09-14T05:29:31.886 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.007794105, 2024-09-14T05:29:31.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011915663, 2024-09-14T05:29:31.886 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.886 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.886 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.886 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.886 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.886 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.886 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3458 2024-09-14T05:29:31.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.887 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395398+0000", 2024-09-14T05:29:31.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395398+0000", 2024-09-14T05:29:31.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395399+0000", 2024-09-14T05:29:31.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.39e-07 2024-09-14T05:29:31.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395400+0000", 2024-09-14T05:29:31.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.156e-06 2024-09-14T05:29:31.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395403+0000", 2024-09-14T05:29:31.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1099999999999999e-06 2024-09-14T05:29:31.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395407+0000", 2024-09-14T05:29:31.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6109999999999998e-06 2024-09-14T05:29:31.889 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.889 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.889 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.889 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3585 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.889 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397148+0000", 2024-09-14T05:29:31.889 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.006045005, 2024-09-14T05:29:31.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01019114, 2024-09-14T05:29:31.890 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.890 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.890 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.890 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3585 2024-09-14T05:29:31.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.890 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397148+0000", 2024-09-14T05:29:31.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397148+0000", 2024-09-14T05:29:31.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397148+0000", 2024-09-14T05:29:31.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5799999999999999e-07 2024-09-14T05:29:31.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397151+0000", 2024-09-14T05:29:31.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.357e-06 2024-09-14T05:29:31.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397157+0000", 2024-09-14T05:29:31.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7349999999999999e-06 2024-09-14T05:29:31.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397163+0000", 2024-09-14T05:29:31.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2850000000000002e-06 2024-09-14T05:29:31.892 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.892 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.893 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.893 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3668 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.893 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397945+0000", 2024-09-14T05:29:31.893 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0052472439999999999, 2024-09-14T05:29:31.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094180540000000004, 2024-09-14T05:29:31.893 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.893 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.893 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.893 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.893 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.893 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.894 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3668 2024-09-14T05:29:31.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.894 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397945+0000", 2024-09-14T05:29:31.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397945+0000", 2024-09-14T05:29:31.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397946+0000", 2024-09-14T05:29:31.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0500000000000003e-07 2024-09-14T05:29:31.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397948+0000", 2024-09-14T05:29:31.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.43e-06 2024-09-14T05:29:31.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397956+0000", 2024-09-14T05:29:31.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7589999999999997e-06 2024-09-14T05:29:31.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397962+0000", 2024-09-14T05:29:31.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5500000000000002e-06 2024-09-14T05:29:31.896 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.896 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.896 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.896 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3701 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.896 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398791+0000", 2024-09-14T05:29:31.896 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0044010999999999998, 2024-09-14T05:29:31.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085947669999999997, 2024-09-14T05:29:31.896 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.896 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.897 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.897 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.897 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.897 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.897 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3701 2024-09-14T05:29:31.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.897 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398791+0000", 2024-09-14T05:29:31.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398791+0000", 2024-09-14T05:29:31.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398792+0000", 2024-09-14T05:29:31.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3200000000000005e-07 2024-09-14T05:29:31.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398798+0000", 2024-09-14T05:29:31.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8799999999999996e-06 2024-09-14T05:29:31.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398805+0000", 2024-09-14T05:29:31.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.827e-06 2024-09-14T05:29:31.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398810+0000", 2024-09-14T05:29:31.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1440000000000002e-06 2024-09-14T05:29:31.899 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.899 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.899 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.900 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3784 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.900 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399452+0000", 2024-09-14T05:29:31.900 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0037409600000000002, 2024-09-14T05:29:31.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080618719999999994, 2024-09-14T05:29:31.900 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.900 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.900 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.900 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.900 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.900 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.900 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3784 2024-09-14T05:29:31.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.900 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399452+0000", 2024-09-14T05:29:31.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399452+0000", 2024-09-14T05:29:31.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399452+0000", 2024-09-14T05:29:31.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6399999999999998e-07 2024-09-14T05:29:31.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399454+0000", 2024-09-14T05:29:31.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-14T05:29:31.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399459+0000", 2024-09-14T05:29:31.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5110000000000003e-06 2024-09-14T05:29:31.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399464+0000", 2024-09-14T05:29:31.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9389999999999996e-06 2024-09-14T05:29:31.903 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.903 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.903 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.903 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3816 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.903 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400143+0000", 2024-09-14T05:29:31.903 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0030492620000000001, 2024-09-14T05:29:31.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0074164829999999998, 2024-09-14T05:29:31.903 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.903 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.903 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.904 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.904 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.904 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.904 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3816 2024-09-14T05:29:31.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.904 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400143+0000", 2024-09-14T05:29:31.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400143+0000", 2024-09-14T05:29:31.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400144+0000", 2024-09-14T05:29:31.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.27e-07 2024-09-14T05:29:31.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400146+0000", 2024-09-14T05:29:31.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.956e-06 2024-09-14T05:29:31.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400151+0000", 2024-09-14T05:29:31.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7470000000000002e-06 2024-09-14T05:29:31.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400156+0000", 2024-09-14T05:29:31.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4020000000000004e-06 2024-09-14T05:29:31.906 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.906 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.906 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.906 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3899 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.906 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400796+0000", 2024-09-14T05:29:31.906 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.002396104, 2024-09-14T05:29:31.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0068122759999999999, 2024-09-14T05:29:31.907 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.907 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.907 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.907 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.907 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.907 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.907 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3899 2024-09-14T05:29:31.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.907 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400796+0000", 2024-09-14T05:29:31.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400796+0000", 2024-09-14T05:29:31.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400797+0000", 2024-09-14T05:29:31.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1199999999999998e-07 2024-09-14T05:29:31.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400803+0000", 2024-09-14T05:29:31.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5699999999999998e-06 2024-09-14T05:29:31.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400814+0000", 2024-09-14T05:29:31.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0778000000000001e-05 2024-09-14T05:29:31.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400820+0000", 2024-09-14T05:29:31.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4539999999999998e-06 2024-09-14T05:29:31.910 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.910 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.910 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.910 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3459 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.910 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395411+0000", 2024-09-14T05:29:31.910 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0077819259999999998, 2024-09-14T05:29:31.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012245041, 2024-09-14T05:29:31.910 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.910 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.910 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.911 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.911 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.911 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.911 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3459 2024-09-14T05:29:31.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.911 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395411+0000", 2024-09-14T05:29:31.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395411+0000", 2024-09-14T05:29:31.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395411+0000", 2024-09-14T05:29:31.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1199999999999999e-07 2024-09-14T05:29:31.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395412+0000", 2024-09-14T05:29:31.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.11e-06 2024-09-14T05:29:31.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395415+0000", 2024-09-14T05:29:31.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9840000000000001e-06 2024-09-14T05:29:31.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395417+0000", 2024-09-14T05:29:31.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4339999999999999e-06 2024-09-14T05:29:31.913 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.913 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.913 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.914 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3586 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.914 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397178+0000", 2024-09-14T05:29:31.914 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0060143009999999997, 2024-09-14T05:29:31.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010517782999999999, 2024-09-14T05:29:31.914 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.914 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.914 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.914 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3586 2024-09-14T05:29:31.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.914 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397178+0000", 2024-09-14T05:29:31.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397178+0000", 2024-09-14T05:29:31.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397179+0000", 2024-09-14T05:29:31.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2500000000000005e-07 2024-09-14T05:29:31.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397181+0000", 2024-09-14T05:29:31.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4899999999999999e-06 2024-09-14T05:29:31.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397189+0000", 2024-09-14T05:29:31.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9270000000000005e-06 2024-09-14T05:29:31.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397195+0000", 2024-09-14T05:29:31.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3179999999999997e-06 2024-09-14T05:29:31.917 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.917 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.917 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.917 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3669 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.917 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397970+0000", 2024-09-14T05:29:31.917 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.005222956, 2024-09-14T05:29:31.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0097506469999999994, 2024-09-14T05:29:31.917 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.917 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.917 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.917 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.918 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3669 2024-09-14T05:29:31.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.918 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397970+0000", 2024-09-14T05:29:31.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397970+0000", 2024-09-14T05:29:31.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397970+0000", 2024-09-14T05:29:31.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9500000000000003e-07 2024-09-14T05:29:31.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397972+0000", 2024-09-14T05:29:31.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.956e-06 2024-09-14T05:29:31.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397979+0000", 2024-09-14T05:29:31.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8700000000000003e-06 2024-09-14T05:29:31.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397984+0000", 2024-09-14T05:29:31.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7040000000000002e-06 2024-09-14T05:29:31.920 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.920 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.920 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.920 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3702 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.921 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398816+0000", 2024-09-14T05:29:31.921 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0043761570000000003, 2024-09-14T05:29:31.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0089265449999999993, 2024-09-14T05:29:31.921 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.921 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.921 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.921 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3702 2024-09-14T05:29:31.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.921 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398816+0000", 2024-09-14T05:29:31.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398816+0000", 2024-09-14T05:29:31.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398817+0000", 2024-09-14T05:29:31.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4799999999999998e-07 2024-09-14T05:29:31.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398819+0000", 2024-09-14T05:29:31.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7579999999999999e-06 2024-09-14T05:29:31.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398829+0000", 2024-09-14T05:29:31.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0137e-05 2024-09-14T05:29:31.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398835+0000", 2024-09-14T05:29:31.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.998e-06 2024-09-14T05:29:31.923 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.924 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.924 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.924 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3785 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.924 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399470+0000", 2024-09-14T05:29:31.924 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0037224419999999999, 2024-09-14T05:29:31.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082991160000000005, 2024-09-14T05:29:31.924 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.924 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.924 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.924 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.924 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.925 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.925 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3785 2024-09-14T05:29:31.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.925 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399470+0000", 2024-09-14T05:29:31.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399470+0000", 2024-09-14T05:29:31.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399470+0000", 2024-09-14T05:29:31.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9000000000000002e-07 2024-09-14T05:29:31.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399472+0000", 2024-09-14T05:29:31.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8959999999999999e-06 2024-09-14T05:29:31.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399478+0000", 2024-09-14T05:29:31.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6049999999999997e-06 2024-09-14T05:29:31.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399483+0000", 2024-09-14T05:29:31.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7010000000000004e-06 2024-09-14T05:29:31.927 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.927 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.927 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.927 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3817 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.927 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400162+0000", 2024-09-14T05:29:31.927 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.003030704, 2024-09-14T05:29:31.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076453340000000002, 2024-09-14T05:29:31.928 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.928 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.928 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.928 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.928 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.928 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.928 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3817 2024-09-14T05:29:31.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.928 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400162+0000", 2024-09-14T05:29:31.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400162+0000", 2024-09-14T05:29:31.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400162+0000", 2024-09-14T05:29:31.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5000000000000003e-07 2024-09-14T05:29:31.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400164+0000", 2024-09-14T05:29:31.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8300000000000001e-06 2024-09-14T05:29:31.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400170+0000", 2024-09-14T05:29:31.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9039999999999997e-06 2024-09-14T05:29:31.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400175+0000", 2024-09-14T05:29:31.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7559999999999999e-06 2024-09-14T05:29:31.930 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.931 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.931 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.931 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3900 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.931 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400833+0000", 2024-09-14T05:29:31.931 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0023592449999999998, 2024-09-14T05:29:31.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0070074860000000003, 2024-09-14T05:29:31.931 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.931 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.931 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.932 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3900 2024-09-14T05:29:31.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.932 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400833+0000", 2024-09-14T05:29:31.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400833+0000", 2024-09-14T05:29:31.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400834+0000", 2024-09-14T05:29:31.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.13e-07 2024-09-14T05:29:31.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400836+0000", 2024-09-14T05:29:31.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3209999999999999e-06 2024-09-14T05:29:31.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400843+0000", 2024-09-14T05:29:31.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6499999999999999e-06 2024-09-14T05:29:31.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400848+0000", 2024-09-14T05:29:31.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3589999999999999e-06 2024-09-14T05:29:31.934 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.934 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.934 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.934 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3460 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.934 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395420+0000", 2024-09-14T05:29:31.934 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.007772134, 2024-09-14T05:29:31.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012462055999999999, 2024-09-14T05:29:31.935 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.935 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.935 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.935 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.935 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.935 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.935 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3460 2024-09-14T05:29:31.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.935 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395420+0000", 2024-09-14T05:29:31.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395420+0000", 2024-09-14T05:29:31.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395421+0000", 2024-09-14T05:29:31.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0800000000000001e-07 2024-09-14T05:29:31.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395422+0000", 2024-09-14T05:29:31.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0109999999999999e-06 2024-09-14T05:29:31.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395425+0000", 2024-09-14T05:29:31.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9569999999999998e-06 2024-09-14T05:29:31.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395427+0000", 2024-09-14T05:29:31.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5000000000000002e-06 2024-09-14T05:29:31.938 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.938 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.938 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.938 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3587 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.938 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397210+0000", 2024-09-14T05:29:31.938 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00598279, 2024-09-14T05:29:31.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010720489, 2024-09-14T05:29:31.938 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.939 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.939 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.939 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.939 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.939 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.939 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3587 2024-09-14T05:29:31.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.939 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397210+0000", 2024-09-14T05:29:31.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397210+0000", 2024-09-14T05:29:31.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397210+0000", 2024-09-14T05:29:31.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4299999999999998e-07 2024-09-14T05:29:31.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397213+0000", 2024-09-14T05:29:31.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3590000000000002e-06 2024-09-14T05:29:31.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397219+0000", 2024-09-14T05:29:31.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6390000000000003e-06 2024-09-14T05:29:31.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397224+0000", 2024-09-14T05:29:31.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1390000000000003e-06 2024-09-14T05:29:31.941 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.941 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.942 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.942 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3670 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.942 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397990+0000", 2024-09-14T05:29:31.942 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0052020900000000004, 2024-09-14T05:29:31.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0099867959999999992, 2024-09-14T05:29:31.942 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.942 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.942 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.942 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3670 2024-09-14T05:29:31.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.943 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397990+0000", 2024-09-14T05:29:31.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397990+0000", 2024-09-14T05:29:31.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397991+0000", 2024-09-14T05:29:31.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4799999999999999e-07 2024-09-14T05:29:31.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397993+0000", 2024-09-14T05:29:31.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2680000000000001e-06 2024-09-14T05:29:31.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398000+0000", 2024-09-14T05:29:31.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0439999999999998e-06 2024-09-14T05:29:31.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398005+0000", 2024-09-14T05:29:31.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8600000000000001e-06 2024-09-14T05:29:31.945 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.945 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.945 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.945 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3703 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.945 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398842+0000", 2024-09-14T05:29:31.945 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0043503190000000001, 2024-09-14T05:29:31.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091823430000000008, 2024-09-14T05:29:31.945 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.945 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.946 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.946 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3703 2024-09-14T05:29:31.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.946 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398842+0000", 2024-09-14T05:29:31.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398842+0000", 2024-09-14T05:29:31.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398843+0000", 2024-09-14T05:29:31.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2099999999999996e-07 2024-09-14T05:29:31.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398845+0000", 2024-09-14T05:29:31.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1289999999999999e-06 2024-09-14T05:29:31.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398851+0000", 2024-09-14T05:29:31.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.263e-06 2024-09-14T05:29:31.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398856+0000", 2024-09-14T05:29:31.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9359999999999998e-06 2024-09-14T05:29:31.948 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.948 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.948 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.949 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3786 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.949 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399488+0000", 2024-09-14T05:29:31.949 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0037042809999999998, 2024-09-14T05:29:31.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085830339999999998, 2024-09-14T05:29:31.949 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.949 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.949 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.949 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3786 2024-09-14T05:29:31.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.950 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399488+0000", 2024-09-14T05:29:31.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399488+0000", 2024-09-14T05:29:31.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399489+0000", 2024-09-14T05:29:31.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6100000000000001e-07 2024-09-14T05:29:31.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399490+0000", 2024-09-14T05:29:31.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6479999999999999e-06 2024-09-14T05:29:31.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399496+0000", 2024-09-14T05:29:31.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5249999999999996e-06 2024-09-14T05:29:31.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399501+0000", 2024-09-14T05:29:31.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8250000000000004e-06 2024-09-14T05:29:31.952 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.952 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.952 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.952 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3818 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.952 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400181+0000", 2024-09-14T05:29:31.952 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.003012013, 2024-09-14T05:29:31.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.007926799, 2024-09-14T05:29:31.953 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.953 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.953 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.953 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.953 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.953 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.953 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3818 2024-09-14T05:29:31.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.953 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400181+0000", 2024-09-14T05:29:31.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400181+0000", 2024-09-14T05:29:31.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400181+0000", 2024-09-14T05:29:31.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.03e-07 2024-09-14T05:29:31.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400183+0000", 2024-09-14T05:29:31.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.778e-06 2024-09-14T05:29:31.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400188+0000", 2024-09-14T05:29:31.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.587e-06 2024-09-14T05:29:31.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400193+0000", 2024-09-14T05:29:31.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7210000000000002e-06 2024-09-14T05:29:31.955 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.956 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.956 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.956 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3901 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.956 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400854+0000", 2024-09-14T05:29:31.956 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0023382070000000001, 2024-09-14T05:29:31.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0072767530000000004, 2024-09-14T05:29:31.956 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.956 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.956 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.957 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3901 2024-09-14T05:29:31.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.957 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400854+0000", 2024-09-14T05:29:31.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400854+0000", 2024-09-14T05:29:31.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400855+0000", 2024-09-14T05:29:31.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-07 2024-09-14T05:29:31.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400857+0000", 2024-09-14T05:29:31.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7269999999999999e-06 2024-09-14T05:29:31.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400868+0000", 2024-09-14T05:29:31.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1659e-05 2024-09-14T05:29:31.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400874+0000", 2024-09-14T05:29:31.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6400000000000002e-06 2024-09-14T05:29:31.959 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.959 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.959 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.959 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3461 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.959 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395430+0000", 2024-09-14T05:29:31.959 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0077625400000000001, 2024-09-14T05:29:31.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012727351, 2024-09-14T05:29:31.960 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.960 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.960 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.960 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3461 2024-09-14T05:29:31.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.960 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395430+0000", 2024-09-14T05:29:31.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395430+0000", 2024-09-14T05:29:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395430+0000", 2024-09-14T05:29:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1400000000000001e-07 2024-09-14T05:29:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395431+0000", 2024-09-14T05:29:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0920000000000001e-06 2024-09-14T05:29:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395434+0000", 2024-09-14T05:29:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.926e-06 2024-09-14T05:29:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395437+0000", 2024-09-14T05:29:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3800000000000001e-06 2024-09-14T05:29:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3588 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397234+0000", 2024-09-14T05:29:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0059583989999999996, 2024-09-14T05:29:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010946416, 2024-09-14T05:29:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3588 2024-09-14T05:29:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397234+0000", 2024-09-14T05:29:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397234+0000", 2024-09-14T05:29:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397235+0000", 2024-09-14T05:29:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5000000000000003e-07 2024-09-14T05:29:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397237+0000", 2024-09-14T05:29:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1179999999999999e-06 2024-09-14T05:29:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397243+0000", 2024-09-14T05:29:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2929999999999997e-06 2024-09-14T05:29:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397248+0000", 2024-09-14T05:29:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9699999999999998e-06 2024-09-14T05:29:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3671 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398011+0000", 2024-09-14T05:29:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0051815050000000003, 2024-09-14T05:29:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010194978, 2024-09-14T05:29:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3671 2024-09-14T05:29:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398011+0000", 2024-09-14T05:29:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398011+0000", 2024-09-14T05:29:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398011+0000", 2024-09-14T05:29:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.84e-07 2024-09-14T05:29:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398013+0000", 2024-09-14T05:29:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4899999999999999e-06 2024-09-14T05:29:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398016+0000", 2024-09-14T05:29:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5650000000000002e-06 2024-09-14T05:29:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398019+0000", 2024-09-14T05:29:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6809999999999998e-06 2024-09-14T05:29:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3704 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398862+0000", 2024-09-14T05:29:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0043304019999999997, 2024-09-14T05:29:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.009369891, 2024-09-14T05:29:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3704 2024-09-14T05:29:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398862+0000", 2024-09-14T05:29:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398862+0000", 2024-09-14T05:29:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398863+0000", 2024-09-14T05:29:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0699999999999998e-07 2024-09-14T05:29:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398864+0000", 2024-09-14T05:29:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9350000000000001e-06 2024-09-14T05:29:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398871+0000", 2024-09-14T05:29:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0809999999999999e-06 2024-09-14T05:29:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398875+0000", 2024-09-14T05:29:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6630000000000001e-06 2024-09-14T05:29:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3787 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399507+0000", 2024-09-14T05:29:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0036858189999999999, 2024-09-14T05:29:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0087650990000000002, 2024-09-14T05:29:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3787 2024-09-14T05:29:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399507+0000", 2024-09-14T05:29:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399507+0000", 2024-09-14T05:29:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399507+0000", 2024-09-14T05:29:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6899999999999998e-07 2024-09-14T05:29:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399509+0000", 2024-09-14T05:29:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.871e-06 2024-09-14T05:29:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399515+0000", 2024-09-14T05:29:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9970000000000004e-06 2024-09-14T05:29:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399519+0000", 2024-09-14T05:29:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5519999999999998e-06 2024-09-14T05:29:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3819 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400199+0000", 2024-09-14T05:29:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.002993959, 2024-09-14T05:29:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081034939999999993, 2024-09-14T05:29:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3819 2024-09-14T05:29:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400199+0000", 2024-09-14T05:29:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400199+0000", 2024-09-14T05:29:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400199+0000", 2024-09-14T05:29:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7800000000000002e-07 2024-09-14T05:29:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400201+0000", 2024-09-14T05:29:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8109999999999999e-06 2024-09-14T05:29:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400207+0000", 2024-09-14T05:29:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8470000000000001e-06 2024-09-14T05:29:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400211+0000", 2024-09-14T05:29:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8029999999999996e-06 2024-09-14T05:29:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3902 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400881+0000", 2024-09-14T05:29:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0023113560000000001, 2024-09-14T05:29:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0074652429999999999, 2024-09-14T05:29:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3902 2024-09-14T05:29:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400881+0000", 2024-09-14T05:29:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400881+0000", 2024-09-14T05:29:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400882+0000", 2024-09-14T05:29:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7399999999999998e-07 2024-09-14T05:29:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400884+0000", 2024-09-14T05:29:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1849999999999998e-06 2024-09-14T05:29:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400890+0000", 2024-09-14T05:29:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0000000000000002e-06 2024-09-14T05:29:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400894+0000", 2024-09-14T05:29:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6589999999999998e-06 2024-09-14T05:29:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3462 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395462+0000", 2024-09-14T05:29:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0077307799999999996, 2024-09-14T05:29:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012935993, 2024-09-14T05:29:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3462 2024-09-14T05:29:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395462+0000", 2024-09-14T05:29:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395462+0000", 2024-09-14T05:29:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395462+0000", 2024-09-14T05:29:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.53e-07 2024-09-14T05:29:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395464+0000", 2024-09-14T05:29:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5510000000000001e-06 2024-09-14T05:29:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395471+0000", 2024-09-14T05:29:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9110000000000004e-06 2024-09-14T05:29:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395474+0000", 2024-09-14T05:29:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.236e-06 2024-09-14T05:29:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3589 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397261+0000", 2024-09-14T05:29:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0059317220000000004, 2024-09-14T05:29:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011184171999999999, 2024-09-14T05:29:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3589 2024-09-14T05:29:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397261+0000", 2024-09-14T05:29:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397261+0000", 2024-09-14T05:29:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397261+0000", 2024-09-14T05:29:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5499999999999998e-07 2024-09-14T05:29:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397263+0000", 2024-09-14T05:29:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1610000000000001e-06 2024-09-14T05:29:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397270+0000", 2024-09-14T05:29:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6649999999999997e-06 2024-09-14T05:29:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397275+0000", 2024-09-14T05:29:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1699999999999996e-06 2024-09-14T05:29:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3672 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398023+0000", 2024-09-14T05:29:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0051700239999999996, 2024-09-14T05:29:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010471569, 2024-09-14T05:29:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3672 2024-09-14T05:29:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398023+0000", 2024-09-14T05:29:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398023+0000", 2024-09-14T05:29:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398023+0000", 2024-09-14T05:29:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.03e-07 2024-09-14T05:29:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398024+0000", 2024-09-14T05:29:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0869999999999999e-06 2024-09-14T05:29:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398048+0000", 2024-09-14T05:29:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3734e-05 2024-09-14T05:29:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398055+0000", 2024-09-14T05:29:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7700000000000001e-06 2024-09-14T05:29:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3705 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398881+0000", 2024-09-14T05:29:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0043112970000000004, 2024-09-14T05:29:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0096515820000000006, 2024-09-14T05:29:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3705 2024-09-14T05:29:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398881+0000", 2024-09-14T05:29:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398881+0000", 2024-09-14T05:29:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398882+0000", 2024-09-14T05:29:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0100000000000005e-07 2024-09-14T05:29:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398884+0000", 2024-09-14T05:29:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.835e-06 2024-09-14T05:29:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398889+0000", 2024-09-14T05:29:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7350000000000001e-06 2024-09-14T05:29:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398894+0000", 2024-09-14T05:29:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7859999999999996e-06 2024-09-14T05:29:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3788 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399526+0000", 2024-09-14T05:29:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.003666992, 2024-09-14T05:29:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0090335459999999999, 2024-09-14T05:29:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3788 2024-09-14T05:29:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399526+0000", 2024-09-14T05:29:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399526+0000", 2024-09-14T05:29:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399526+0000", 2024-09-14T05:29:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.75e-07 2024-09-14T05:29:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399537+0000", 2024-09-14T05:29:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0698e-05 2024-09-14T05:29:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399543+0000", 2024-09-14T05:29:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7299999999999999e-06 2024-09-14T05:29:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399549+0000", 2024-09-14T05:29:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0669999999999999e-06 2024-09-14T05:29:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3820 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400223+0000", 2024-09-14T05:29:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0029698540000000001, 2024-09-14T05:29:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0083595500000000003, 2024-09-14T05:29:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3820 2024-09-14T05:29:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400223+0000", 2024-09-14T05:29:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400223+0000", 2024-09-14T05:29:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400223+0000", 2024-09-14T05:29:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9200000000000001e-07 2024-09-14T05:29:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400225+0000", 2024-09-14T05:29:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0760000000000001e-06 2024-09-14T05:29:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400234+0000", 2024-09-14T05:29:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5250000000000005e-06 2024-09-14T05:29:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400239+0000", 2024-09-14T05:29:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2739999999999998e-06 2024-09-14T05:29:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3903 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400900+0000", 2024-09-14T05:29:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.002292522, 2024-09-14T05:29:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0077061270000000001, 2024-09-14T05:29:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3903 2024-09-14T05:29:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400900+0000", 2024-09-14T05:29:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400900+0000", 2024-09-14T05:29:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400901+0000", 2024-09-14T05:29:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2799999999999996e-07 2024-09-14T05:29:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400902+0000", 2024-09-14T05:29:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7090000000000001e-06 2024-09-14T05:29:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400908+0000", 2024-09-14T05:29:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7899999999999996e-06 2024-09-14T05:29:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400913+0000", 2024-09-14T05:29:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5329999999999996e-06 2024-09-14T05:29:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3463 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395478+0000", 2024-09-14T05:29:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0077149300000000001, 2024-09-14T05:29:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013152789, 2024-09-14T05:29:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3463 2024-09-14T05:29:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395478+0000", 2024-09-14T05:29:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395478+0000", 2024-09-14T05:29:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395478+0000", 2024-09-14T05:29:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.72e-07 2024-09-14T05:29:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395479+0000", 2024-09-14T05:29:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.22e-06 2024-09-14T05:29:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395483+0000", 2024-09-14T05:29:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4199999999999999e-06 2024-09-14T05:29:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395485+0000", 2024-09-14T05:29:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5639999999999999e-06 2024-09-14T05:29:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3590 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397305+0000", 2024-09-14T05:29:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0058871110000000004, 2024-09-14T05:29:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011361104, 2024-09-14T05:29:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3590 2024-09-14T05:29:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397305+0000", 2024-09-14T05:29:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397305+0000", 2024-09-14T05:29:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397306+0000", 2024-09-14T05:29:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-14T05:29:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397317+0000", 2024-09-14T05:29:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1018e-05 2024-09-14T05:29:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397324+0000", 2024-09-14T05:29:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6409999999999996e-06 2024-09-14T05:29:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397329+0000", 2024-09-14T05:29:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2209999999999996e-06 2024-09-14T05:29:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3673 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398069+0000", 2024-09-14T05:29:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0051232860000000003, 2024-09-14T05:29:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010645929, 2024-09-14T05:29:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3673 2024-09-14T05:29:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398069+0000", 2024-09-14T05:29:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398069+0000", 2024-09-14T05:29:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398070+0000", 2024-09-14T05:29:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0020000000000001e-06 2024-09-14T05:29:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398074+0000", 2024-09-14T05:29:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8630000000000001e-06 2024-09-14T05:29:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398081+0000", 2024-09-14T05:29:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0339999999999999e-06 2024-09-14T05:29:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398088+0000", 2024-09-14T05:29:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3810000000000004e-06 2024-09-14T05:29:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3706 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398900+0000", 2024-09-14T05:29:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0042926079999999998, 2024-09-14T05:29:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0098656030000000006, 2024-09-14T05:29:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3706 2024-09-14T05:29:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398900+0000", 2024-09-14T05:29:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398900+0000", 2024-09-14T05:29:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398900+0000", 2024-09-14T05:29:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8000000000000001e-07 2024-09-14T05:29:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398902+0000", 2024-09-14T05:29:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9980000000000002e-06 2024-09-14T05:29:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398908+0000", 2024-09-14T05:29:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7699999999999998e-06 2024-09-14T05:29:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398913+0000", 2024-09-14T05:29:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4909999999999999e-06 2024-09-14T05:29:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3789 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399555+0000", 2024-09-14T05:29:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0036376220000000001, 2024-09-14T05:29:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.009256841, 2024-09-14T05:29:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3789 2024-09-14T05:29:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399555+0000", 2024-09-14T05:29:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399555+0000", 2024-09-14T05:29:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399555+0000", 2024-09-14T05:29:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2800000000000002e-07 2024-09-14T05:29:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399557+0000", 2024-09-14T05:29:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.945e-06 2024-09-14T05:29:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399563+0000", 2024-09-14T05:29:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7799999999999997e-06 2024-09-14T05:29:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399568+0000", 2024-09-14T05:29:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7090000000000001e-06 2024-09-14T05:29:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3821 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400246+0000", 2024-09-14T05:29:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.002946864, 2024-09-14T05:29:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0086216339999999995, 2024-09-14T05:29:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3821 2024-09-14T05:29:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400246+0000", 2024-09-14T05:29:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400246+0000", 2024-09-14T05:29:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400246+0000", 2024-09-14T05:29:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0999999999999999e-07 2024-09-14T05:29:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400248+0000", 2024-09-14T05:29:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.108e-06 2024-09-14T05:29:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400254+0000", 2024-09-14T05:29:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9649999999999996e-06 2024-09-14T05:29:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400259+0000", 2024-09-14T05:29:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1479999999999997e-06 2024-09-14T05:29:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3904 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400919+0000", 2024-09-14T05:29:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0022739769999999999, 2024-09-14T05:29:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0079787060000000003, 2024-09-14T05:29:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3904 2024-09-14T05:29:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400919+0000", 2024-09-14T05:29:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400919+0000", 2024-09-14T05:29:32.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400919+0000", 2024-09-14T05:29:32.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3000000000000001e-07 2024-09-14T05:29:32.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400921+0000", 2024-09-14T05:29:32.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9520000000000001e-06 2024-09-14T05:29:32.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400927+0000", 2024-09-14T05:29:32.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8119999999999996e-06 2024-09-14T05:29:32.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400931+0000", 2024-09-14T05:29:32.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5460000000000002e-06 2024-09-14T05:29:32.034 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.034 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.034 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.035 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3464 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.035 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395505+0000", 2024-09-14T05:29:32.035 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0076879369999999997, 2024-09-14T05:29:32.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013419505999999999, 2024-09-14T05:29:32.035 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.035 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.035 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.035 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.035 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.035 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.035 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3464 2024-09-14T05:29:32.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.036 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395505+0000", 2024-09-14T05:29:32.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395505+0000", 2024-09-14T05:29:32.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395505+0000", 2024-09-14T05:29:32.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2100000000000002e-07 2024-09-14T05:29:32.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395506+0000", 2024-09-14T05:29:32.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4300000000000001e-06 2024-09-14T05:29:32.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395510+0000", 2024-09-14T05:29:32.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0249999999999996e-06 2024-09-14T05:29:32.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395513+0000", 2024-09-14T05:29:32.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6129999999999998e-06 2024-09-14T05:29:32.038 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.038 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.038 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.038 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3591 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.038 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397372+0000", 2024-09-14T05:29:32.038 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0058204750000000003, 2024-09-14T05:29:32.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011575787000000001, 2024-09-14T05:29:32.038 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.038 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.038 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.039 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.039 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.039 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.039 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3591 2024-09-14T05:29:32.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.039 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397372+0000", 2024-09-14T05:29:32.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397372+0000", 2024-09-14T05:29:32.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397373+0000", 2024-09-14T05:29:32.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9500000000000003e-07 2024-09-14T05:29:32.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397375+0000", 2024-09-14T05:29:32.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8679999999999999e-06 2024-09-14T05:29:32.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397383+0000", 2024-09-14T05:29:32.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0770000000000008e-06 2024-09-14T05:29:32.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397389+0000", 2024-09-14T05:29:32.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2410000000000003e-06 2024-09-14T05:29:32.041 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.041 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.041 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.042 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3674 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.042 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398111+0000", 2024-09-14T05:29:32.042 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0050819150000000002, 2024-09-14T05:29:32.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010860074000000001, 2024-09-14T05:29:32.042 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.042 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.042 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.042 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.042 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.042 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.042 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3674 2024-09-14T05:29:32.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.042 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398111+0000", 2024-09-14T05:29:32.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398111+0000", 2024-09-14T05:29:32.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398111+0000", 2024-09-14T05:29:32.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.23e-07 2024-09-14T05:29:32.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398115+0000", 2024-09-14T05:29:32.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7900000000000001e-06 2024-09-14T05:29:32.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398126+0000", 2024-09-14T05:29:32.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1164e-05 2024-09-14T05:29:32.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398134+0000", 2024-09-14T05:29:32.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1400000000000002e-06 2024-09-14T05:29:32.045 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.045 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.045 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.045 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3707 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.045 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398919+0000", 2024-09-14T05:29:32.045 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0042740030000000002, 2024-09-14T05:29:32.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010076313, 2024-09-14T05:29:32.045 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.045 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.045 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.046 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3707 2024-09-14T05:29:32.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.046 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398919+0000", 2024-09-14T05:29:32.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398919+0000", 2024-09-14T05:29:32.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398919+0000", 2024-09-14T05:29:32.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.72e-07 2024-09-14T05:29:32.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398921+0000", 2024-09-14T05:29:32.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.812e-06 2024-09-14T05:29:32.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398926+0000", 2024-09-14T05:29:32.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5659999999999997e-06 2024-09-14T05:29:32.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398931+0000", 2024-09-14T05:29:32.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7820000000000001e-06 2024-09-14T05:29:32.048 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.048 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.048 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.048 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3790 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.048 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399574+0000", 2024-09-14T05:29:32.049 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0036188689999999998, 2024-09-14T05:29:32.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094442299999999996, 2024-09-14T05:29:32.049 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.049 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.049 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.049 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3790 2024-09-14T05:29:32.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.049 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399574+0000", 2024-09-14T05:29:32.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399574+0000", 2024-09-14T05:29:32.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399574+0000", 2024-09-14T05:29:32.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1899999999999998e-07 2024-09-14T05:29:32.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399576+0000", 2024-09-14T05:29:32.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9e-06 2024-09-14T05:29:32.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399582+0000", 2024-09-14T05:29:32.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7409999999999998e-06 2024-09-14T05:29:32.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399587+0000", 2024-09-14T05:29:32.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8099999999999997e-06 2024-09-14T05:29:32.052 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.052 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.052 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.052 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3822 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.052 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400265+0000", 2024-09-14T05:29:32.052 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0029273229999999999, 2024-09-14T05:29:32.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0087766839999999999, 2024-09-14T05:29:32.052 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.052 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.052 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.052 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.052 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.053 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.053 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3822 2024-09-14T05:29:32.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.053 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400265+0000", 2024-09-14T05:29:32.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400265+0000", 2024-09-14T05:29:32.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400266+0000", 2024-09-14T05:29:32.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8299999999999998e-07 2024-09-14T05:29:32.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400267+0000", 2024-09-14T05:29:32.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-14T05:29:32.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400275+0000", 2024-09-14T05:29:32.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1609999999999997e-06 2024-09-14T05:29:32.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400279+0000", 2024-09-14T05:29:32.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9130000000000002e-06 2024-09-14T05:29:32.055 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.055 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.055 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.055 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3905 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.055 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400937+0000", 2024-09-14T05:29:32.055 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0022557789999999999, 2024-09-14T05:29:32.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081282139999999999, 2024-09-14T05:29:32.056 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.056 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.056 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.056 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.056 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.056 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.056 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3905 2024-09-14T05:29:32.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.056 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400937+0000", 2024-09-14T05:29:32.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400937+0000", 2024-09-14T05:29:32.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400937+0000", 2024-09-14T05:29:32.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9200000000000002e-07 2024-09-14T05:29:32.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400939+0000", 2024-09-14T05:29:32.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7179999999999999e-06 2024-09-14T05:29:32.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400945+0000", 2024-09-14T05:29:32.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6919999999999999e-06 2024-09-14T05:29:32.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400949+0000", 2024-09-14T05:29:32.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6410000000000001e-06 2024-09-14T05:29:32.059 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.059 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.059 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.059 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3465 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.059 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395516+0000", 2024-09-14T05:29:32.059 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0076760760000000004, 2024-09-14T05:29:32.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01358911, 2024-09-14T05:29:32.059 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.059 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.059 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.059 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.060 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.060 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.060 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3465 2024-09-14T05:29:32.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.060 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395516+0000", 2024-09-14T05:29:32.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395516+0000", 2024-09-14T05:29:32.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395517+0000", 2024-09-14T05:29:32.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-14T05:29:32.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395518+0000", 2024-09-14T05:29:32.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0809999999999999e-06 2024-09-14T05:29:32.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395521+0000", 2024-09-14T05:29:32.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.959e-06 2024-09-14T05:29:32.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395523+0000", 2024-09-14T05:29:32.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.491e-06 2024-09-14T05:29:32.062 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.062 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.062 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.062 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3592 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.063 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.397412+0000", 2024-09-14T05:29:32.063 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0057803910000000002, 2024-09-14T05:29:32.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011738915000000001, 2024-09-14T05:29:32.063 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.063 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.063 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.063 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.063 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.063 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.063 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3592 2024-09-14T05:29:32.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.063 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397412+0000", 2024-09-14T05:29:32.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397412+0000", 2024-09-14T05:29:32.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397413+0000", 2024-09-14T05:29:32.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1600000000000002e-07 2024-09-14T05:29:32.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397415+0000", 2024-09-14T05:29:32.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0509999999999999e-06 2024-09-14T05:29:32.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397425+0000", 2024-09-14T05:29:32.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0438999999999999e-05 2024-09-14T05:29:32.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.397430+0000", 2024-09-14T05:29:32.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8829999999999997e-06 2024-09-14T05:29:32.066 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.066 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.066 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.066 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3675 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.066 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398145+0000", 2024-09-14T05:29:32.066 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0050475629999999997, 2024-09-14T05:29:32.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011055107999999999, 2024-09-14T05:29:32.066 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.066 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.067 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.067 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.067 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.067 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.067 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3675 2024-09-14T05:29:32.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.067 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398145+0000", 2024-09-14T05:29:32.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398145+0000", 2024-09-14T05:29:32.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398146+0000", 2024-09-14T05:29:32.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9999999999999997e-07 2024-09-14T05:29:32.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398148+0000", 2024-09-14T05:29:32.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2910000000000002e-06 2024-09-14T05:29:32.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398154+0000", 2024-09-14T05:29:32.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5259999999999999e-06 2024-09-14T05:29:32.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398160+0000", 2024-09-14T05:29:32.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3290000000000001e-06 2024-09-14T05:29:32.069 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.069 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.069 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.070 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3708 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.070 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398937+0000", 2024-09-14T05:29:32.070 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0042555859999999996, 2024-09-14T05:29:32.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010308691, 2024-09-14T05:29:32.070 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.070 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.070 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.070 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.070 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.070 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.070 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3708 2024-09-14T05:29:32.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.071 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398937+0000", 2024-09-14T05:29:32.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398937+0000", 2024-09-14T05:29:32.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398937+0000", 2024-09-14T05:29:32.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.15e-07 2024-09-14T05:29:32.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398939+0000", 2024-09-14T05:29:32.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8589999999999999e-06 2024-09-14T05:29:32.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398945+0000", 2024-09-14T05:29:32.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.246e-06 2024-09-14T05:29:32.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398950+0000", 2024-09-14T05:29:32.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7389999999999999e-06 2024-09-14T05:29:32.073 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.073 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.073 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.073 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3791 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.073 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399592+0000", 2024-09-14T05:29:32.073 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00360049, 2024-09-14T05:29:32.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0097025040000000007, 2024-09-14T05:29:32.073 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.074 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.074 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.074 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.074 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.074 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.074 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3791 2024-09-14T05:29:32.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.074 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399592+0000", 2024-09-14T05:29:32.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399592+0000", 2024-09-14T05:29:32.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399593+0000", 2024-09-14T05:29:32.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9100000000000004e-07 2024-09-14T05:29:32.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399594+0000", 2024-09-14T05:29:32.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8679999999999999e-06 2024-09-14T05:29:32.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399600+0000", 2024-09-14T05:29:32.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8930000000000002e-06 2024-09-14T05:29:32.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399605+0000", 2024-09-14T05:29:32.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4279999999999998e-06 2024-09-14T05:29:32.076 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.077 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.077 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.077 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3823 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.077 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400286+0000", 2024-09-14T05:29:32.077 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0029068430000000001, 2024-09-14T05:29:32.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0090398249999999996, 2024-09-14T05:29:32.077 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.077 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.077 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.078 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3823 2024-09-14T05:29:32.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.078 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400286+0000", 2024-09-14T05:29:32.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400286+0000", 2024-09-14T05:29:32.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400286+0000", 2024-09-14T05:29:32.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1199999999999998e-07 2024-09-14T05:29:32.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400288+0000", 2024-09-14T05:29:32.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9479999999999998e-06 2024-09-14T05:29:32.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400294+0000", 2024-09-14T05:29:32.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9209999999999997e-06 2024-09-14T05:29:32.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400298+0000", 2024-09-14T05:29:32.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4669999999999998e-06 2024-09-14T05:29:32.080 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.080 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.080 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.080 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3906 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.080 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400955+0000", 2024-09-14T05:29:32.081 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0022376879999999998, 2024-09-14T05:29:32.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0083963179999999998, 2024-09-14T05:29:32.081 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.081 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.081 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.081 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3906 2024-09-14T05:29:32.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.081 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400955+0000", 2024-09-14T05:29:32.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400955+0000", 2024-09-14T05:29:32.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400955+0000", 2024-09-14T05:29:32.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.65e-07 2024-09-14T05:29:32.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400957+0000", 2024-09-14T05:29:32.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.863e-06 2024-09-14T05:29:32.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400963+0000", 2024-09-14T05:29:32.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9050000000000002e-06 2024-09-14T05:29:32.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400968+0000", 2024-09-14T05:29:32.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6110000000000004e-06 2024-09-14T05:29:32.084 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.084 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.084 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.084 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3466 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.084 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395544+0000", 2024-09-14T05:29:32.084 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0076489569999999996, 2024-09-14T05:29:32.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013834577000000001, 2024-09-14T05:29:32.084 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.084 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.084 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.084 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.085 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.085 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.085 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3466 2024-09-14T05:29:32.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.085 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395544+0000", 2024-09-14T05:29:32.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395544+0000", 2024-09-14T05:29:32.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395544+0000", 2024-09-14T05:29:32.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5699999999999998e-07 2024-09-14T05:29:32.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395545+0000", 2024-09-14T05:29:32.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3820000000000001e-06 2024-09-14T05:29:32.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395552+0000", 2024-09-14T05:29:32.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.798e-06 2024-09-14T05:29:32.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395555+0000", 2024-09-14T05:29:32.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9119999999999998e-06 2024-09-14T05:29:32.087 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.087 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.087 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.087 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3676 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.088 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398167+0000", 2024-09-14T05:29:32.088 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0050258589999999997, 2024-09-14T05:29:32.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011258059000000001, 2024-09-14T05:29:32.088 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.088 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.088 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.088 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.088 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.088 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.088 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3676 2024-09-14T05:29:32.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.088 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398167+0000", 2024-09-14T05:29:32.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398167+0000", 2024-09-14T05:29:32.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398167+0000", 2024-09-14T05:29:32.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8100000000000003e-07 2024-09-14T05:29:32.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398169+0000", 2024-09-14T05:29:32.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0549999999999998e-06 2024-09-14T05:29:32.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398175+0000", 2024-09-14T05:29:32.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2079999999999997e-06 2024-09-14T05:29:32.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398180+0000", 2024-09-14T05:29:32.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0479999999999998e-06 2024-09-14T05:29:32.091 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.091 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.091 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.091 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3709 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.091 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398956+0000", 2024-09-14T05:29:32.091 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0042363410000000002, 2024-09-14T05:29:32.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010509271000000001, 2024-09-14T05:29:32.091 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.091 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.091 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.091 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.092 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.092 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.092 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3709 2024-09-14T05:29:32.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.092 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398956+0000", 2024-09-14T05:29:32.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398956+0000", 2024-09-14T05:29:32.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398957+0000", 2024-09-14T05:29:32.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9000000000000002e-07 2024-09-14T05:29:32.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398958+0000", 2024-09-14T05:29:32.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.854e-06 2024-09-14T05:29:32.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398964+0000", 2024-09-14T05:29:32.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5470000000000004e-06 2024-09-14T05:29:32.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398968+0000", 2024-09-14T05:29:32.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5009999999999998e-06 2024-09-14T05:29:32.094 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.094 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.094 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.094 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3792 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.094 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399611+0000", 2024-09-14T05:29:32.095 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0035819020000000001, 2024-09-14T05:29:32.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0099049229999999995, 2024-09-14T05:29:32.095 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.095 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.095 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.095 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.095 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.095 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.095 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3792 2024-09-14T05:29:32.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.095 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399611+0000", 2024-09-14T05:29:32.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399611+0000", 2024-09-14T05:29:32.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399611+0000", 2024-09-14T05:29:32.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1300000000000001e-07 2024-09-14T05:29:32.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399613+0000", 2024-09-14T05:29:32.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8509999999999999e-06 2024-09-14T05:29:32.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399618+0000", 2024-09-14T05:29:32.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6030000000000004e-06 2024-09-14T05:29:32.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399623+0000", 2024-09-14T05:29:32.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.515e-06 2024-09-14T05:29:32.097 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.098 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.098 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.098 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3824 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.098 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400304+0000", 2024-09-14T05:29:32.098 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0028884169999999999, 2024-09-14T05:29:32.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0092599069999999995, 2024-09-14T05:29:32.098 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.098 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.098 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.098 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.098 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.098 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.099 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3824 2024-09-14T05:29:32.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.099 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400304+0000", 2024-09-14T05:29:32.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400304+0000", 2024-09-14T05:29:32.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400305+0000", 2024-09-14T05:29:32.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0900000000000002e-07 2024-09-14T05:29:32.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400310+0000", 2024-09-14T05:29:32.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.242e-06 2024-09-14T05:29:32.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400316+0000", 2024-09-14T05:29:32.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.823e-06 2024-09-14T05:29:32.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400320+0000", 2024-09-14T05:29:32.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.797e-06 2024-09-14T05:29:32.101 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.101 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.101 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.101 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3907 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.101 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400973+0000", 2024-09-14T05:29:32.101 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0022194879999999999, 2024-09-14T05:29:32.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0086186730000000003, 2024-09-14T05:29:32.102 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.102 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.102 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.102 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3907 2024-09-14T05:29:32.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.102 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400973+0000", 2024-09-14T05:29:32.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400973+0000", 2024-09-14T05:29:32.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400973+0000", 2024-09-14T05:29:32.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9499999999999998e-07 2024-09-14T05:29:32.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400975+0000", 2024-09-14T05:29:32.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.809e-06 2024-09-14T05:29:32.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400981+0000", 2024-09-14T05:29:32.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0000000000000002e-06 2024-09-14T05:29:32.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400986+0000", 2024-09-14T05:29:32.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6240000000000001e-06 2024-09-14T05:29:32.104 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.104 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.104 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.105 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3467 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.105 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395559+0000", 2024-09-14T05:29:32.105 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0076339800000000003, 2024-09-14T05:29:32.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014060168, 2024-09-14T05:29:32.105 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.105 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.105 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.105 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.105 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.105 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.105 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3467 2024-09-14T05:29:32.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.105 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395559+0000", 2024-09-14T05:29:32.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395559+0000", 2024-09-14T05:29:32.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395559+0000", 2024-09-14T05:29:32.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7799999999999997e-07 2024-09-14T05:29:32.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395567+0000", 2024-09-14T05:29:32.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0749999999999998e-06 2024-09-14T05:29:32.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395572+0000", 2024-09-14T05:29:32.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8439999999999997e-06 2024-09-14T05:29:32.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395575+0000", 2024-09-14T05:29:32.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9100000000000001e-06 2024-09-14T05:29:32.107 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.108 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.108 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.108 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3562 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.108 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.396332+0000", 2024-09-14T05:29:32.108 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0068600349999999996, 2024-09-14T05:29:32.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013309385999999999, 2024-09-14T05:29:32.108 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.108 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.108 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.108 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.108 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.109 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.109 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3562 2024-09-14T05:29:32.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.109 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396332+0000", 2024-09-14T05:29:32.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396332+0000", 2024-09-14T05:29:32.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396333+0000", 2024-09-14T05:29:32.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5899999999999996e-07 2024-09-14T05:29:32.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396335+0000", 2024-09-14T05:29:32.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8419999999999999e-06 2024-09-14T05:29:32.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396341+0000", 2024-09-14T05:29:32.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9529999999999996e-06 2024-09-14T05:29:32.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396346+0000", 2024-09-14T05:29:32.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.198e-06 2024-09-14T05:29:32.111 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.111 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.111 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.112 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3677 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.112 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398187+0000", 2024-09-14T05:29:32.112 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0050054950000000004, 2024-09-14T05:29:32.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011479563999999999, 2024-09-14T05:29:32.112 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.112 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.112 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.112 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.112 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.112 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.112 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3677 2024-09-14T05:29:32.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.113 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398187+0000", 2024-09-14T05:29:32.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398187+0000", 2024-09-14T05:29:32.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398188+0000", 2024-09-14T05:29:32.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.37e-07 2024-09-14T05:29:32.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398198+0000", 2024-09-14T05:29:32.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0321e-05 2024-09-14T05:29:32.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398222+0000", 2024-09-14T05:29:32.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4309000000000001e-05 2024-09-14T05:29:32.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398228+0000", 2024-09-14T05:29:32.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7389999999999996e-06 2024-09-14T05:29:32.115 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.115 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.115 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.115 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3727 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.115 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398974+0000", 2024-09-14T05:29:32.115 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00421806, 2024-09-14T05:29:32.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010716615000000001, 2024-09-14T05:29:32.115 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.115 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.116 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.116 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.116 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.116 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.116 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3727 2024-09-14T05:29:32.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.116 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398974+0000", 2024-09-14T05:29:32.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398974+0000", 2024-09-14T05:29:32.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398975+0000", 2024-09-14T05:29:32.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7800000000000002e-07 2024-09-14T05:29:32.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398977+0000", 2024-09-14T05:29:32.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8390000000000001e-06 2024-09-14T05:29:32.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398982+0000", 2024-09-14T05:29:32.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6110000000000001e-06 2024-09-14T05:29:32.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398987+0000", 2024-09-14T05:29:32.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.843e-06 2024-09-14T05:29:32.118 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.118 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.118 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.119 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3793 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.119 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399629+0000", 2024-09-14T05:29:32.119 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0035631059999999999, 2024-09-14T05:29:32.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010086499000000001, 2024-09-14T05:29:32.119 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.119 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.119 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.119 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.119 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.119 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.119 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3793 2024-09-14T05:29:32.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.119 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399629+0000", 2024-09-14T05:29:32.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399629+0000", 2024-09-14T05:29:32.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399630+0000", 2024-09-14T05:29:32.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.27e-07 2024-09-14T05:29:32.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399632+0000", 2024-09-14T05:29:32.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0279999999999999e-06 2024-09-14T05:29:32.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399638+0000", 2024-09-14T05:29:32.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6459999999999998e-06 2024-09-14T05:29:32.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399642+0000", 2024-09-14T05:29:32.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.775e-06 2024-09-14T05:29:32.122 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.122 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.122 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.122 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3825 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.122 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400337+0000", 2024-09-14T05:29:32.122 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0028554380000000001, 2024-09-14T05:29:32.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094020739999999998, 2024-09-14T05:29:32.122 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.122 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.122 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.123 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.123 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.123 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.123 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3825 2024-09-14T05:29:32.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.123 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400337+0000", 2024-09-14T05:29:32.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400337+0000", 2024-09-14T05:29:32.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400338+0000", 2024-09-14T05:29:32.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8400000000000004e-07 2024-09-14T05:29:32.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400340+0000", 2024-09-14T05:29:32.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.17e-06 2024-09-14T05:29:32.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400347+0000", 2024-09-14T05:29:32.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3459999999999996e-06 2024-09-14T05:29:32.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400352+0000", 2024-09-14T05:29:32.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0640000000000001e-06 2024-09-14T05:29:32.125 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.125 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.125 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.126 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3908 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.126 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400992+0000", 2024-09-14T05:29:32.126 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.002200998, 2024-09-14T05:29:32.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0087720539999999996, 2024-09-14T05:29:32.126 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.126 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.126 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.126 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.126 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.126 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.126 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3908 2024-09-14T05:29:32.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.126 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400992+0000", 2024-09-14T05:29:32.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400992+0000", 2024-09-14T05:29:32.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400992+0000", 2024-09-14T05:29:32.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0200000000000003e-07 2024-09-14T05:29:32.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400994+0000", 2024-09-14T05:29:32.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.708e-06 2024-09-14T05:29:32.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400999+0000", 2024-09-14T05:29:32.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7540000000000003e-06 2024-09-14T05:29:32.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401011+0000", 2024-09-14T05:29:32.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1618e-05 2024-09-14T05:29:32.129 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.129 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.129 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.129 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3468 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.129 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395589+0000", 2024-09-14T05:29:32.129 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0076035629999999998, 2024-09-14T05:29:32.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014199579, 2024-09-14T05:29:32.129 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.129 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.130 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.130 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.130 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.130 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.130 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3468 2024-09-14T05:29:32.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.130 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395589+0000", 2024-09-14T05:29:32.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395589+0000", 2024-09-14T05:29:32.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395589+0000", 2024-09-14T05:29:32.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-14T05:29:32.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395591+0000", 2024-09-14T05:29:32.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5379999999999999e-06 2024-09-14T05:29:32.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395595+0000", 2024-09-14T05:29:32.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9929999999999997e-06 2024-09-14T05:29:32.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395597+0000", 2024-09-14T05:29:32.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.599e-06 2024-09-14T05:29:32.132 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.132 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.132 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.133 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3563 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.133 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.396362+0000", 2024-09-14T05:29:32.133 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0068301569999999999, 2024-09-14T05:29:32.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013451083000000001, 2024-09-14T05:29:32.133 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.133 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.133 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.133 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.133 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.133 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.133 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3563 2024-09-14T05:29:32.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.133 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396362+0000", 2024-09-14T05:29:32.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396362+0000", 2024-09-14T05:29:32.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396363+0000", 2024-09-14T05:29:32.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0500000000000003e-07 2024-09-14T05:29:32.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396365+0000", 2024-09-14T05:29:32.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3709999999999998e-06 2024-09-14T05:29:32.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396373+0000", 2024-09-14T05:29:32.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3069999999999997e-06 2024-09-14T05:29:32.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396377+0000", 2024-09-14T05:29:32.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8289999999999998e-06 2024-09-14T05:29:32.136 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.136 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.136 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.136 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3678 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.136 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398242+0000", 2024-09-14T05:29:32.136 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0049501650000000003, 2024-09-14T05:29:32.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011613244999999999, 2024-09-14T05:29:32.136 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.136 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.137 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.137 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.137 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.137 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.137 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3678 2024-09-14T05:29:32.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.137 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398242+0000", 2024-09-14T05:29:32.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398242+0000", 2024-09-14T05:29:32.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398243+0000", 2024-09-14T05:29:32.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.68e-07 2024-09-14T05:29:32.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398245+0000", 2024-09-14T05:29:32.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4380000000000002e-06 2024-09-14T05:29:32.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398252+0000", 2024-09-14T05:29:32.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8279999999999997e-06 2024-09-14T05:29:32.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398257+0000", 2024-09-14T05:29:32.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-14T05:29:32.139 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.139 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.139 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.140 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3728 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.140 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398993+0000", 2024-09-14T05:29:32.140 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0041995119999999999, 2024-09-14T05:29:32.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010914205, 2024-09-14T05:29:32.140 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.140 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.140 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.140 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3728 2024-09-14T05:29:32.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.140 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398993+0000", 2024-09-14T05:29:32.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398993+0000", 2024-09-14T05:29:32.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398994+0000", 2024-09-14T05:29:32.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6499999999999999e-07 2024-09-14T05:29:32.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398995+0000", 2024-09-14T05:29:32.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.866e-06 2024-09-14T05:29:32.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399001+0000", 2024-09-14T05:29:32.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9889999999999997e-06 2024-09-14T05:29:32.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399006+0000", 2024-09-14T05:29:32.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6079999999999998e-06 2024-09-14T05:29:32.143 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.143 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.143 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.143 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3794 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.143 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399648+0000", 2024-09-14T05:29:32.143 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0035443530000000001, 2024-09-14T05:29:32.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010306245, 2024-09-14T05:29:32.143 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.143 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.143 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.144 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3794 2024-09-14T05:29:32.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.144 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399648+0000", 2024-09-14T05:29:32.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399648+0000", 2024-09-14T05:29:32.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399649+0000", 2024-09-14T05:29:32.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8999999999999996e-07 2024-09-14T05:29:32.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399651+0000", 2024-09-14T05:29:32.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.821e-06 2024-09-14T05:29:32.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399657+0000", 2024-09-14T05:29:32.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000002e-06 2024-09-14T05:29:32.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399662+0000", 2024-09-14T05:29:32.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9899999999999997e-06 2024-09-14T05:29:32.146 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.146 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.146 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.147 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3843 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.147 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400359+0000", 2024-09-14T05:29:32.147 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0028338780000000002, 2024-09-14T05:29:32.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0096451660000000002, 2024-09-14T05:29:32.147 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.147 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.147 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.147 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.147 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.147 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.147 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3843 2024-09-14T05:29:32.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.148 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400359+0000", 2024-09-14T05:29:32.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400359+0000", 2024-09-14T05:29:32.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400359+0000", 2024-09-14T05:29:32.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7100000000000002e-07 2024-09-14T05:29:32.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400361+0000", 2024-09-14T05:29:32.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.902e-06 2024-09-14T05:29:32.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400367+0000", 2024-09-14T05:29:32.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8279999999999999e-06 2024-09-14T05:29:32.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400372+0000", 2024-09-14T05:29:32.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7400000000000004e-06 2024-09-14T05:29:32.150 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.150 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.150 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.150 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3909 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.150 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401018+0000", 2024-09-14T05:29:32.150 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0021740739999999998, 2024-09-14T05:29:32.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0090134629999999993, 2024-09-14T05:29:32.150 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.151 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.151 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.151 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.151 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.151 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.151 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3909 2024-09-14T05:29:32.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.151 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401018+0000", 2024-09-14T05:29:32.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401018+0000", 2024-09-14T05:29:32.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401019+0000", 2024-09-14T05:29:32.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1800000000000001e-07 2024-09-14T05:29:32.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401021+0000", 2024-09-14T05:29:32.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8959999999999999e-06 2024-09-14T05:29:32.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401028+0000", 2024-09-14T05:29:32.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3239999999999997e-06 2024-09-14T05:29:32.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401033+0000", 2024-09-14T05:29:32.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1320000000000002e-06 2024-09-14T05:29:32.153 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.153 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.153 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.154 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3469 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.154 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395601+0000", 2024-09-14T05:29:32.154 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0075916509999999996, 2024-09-14T05:29:32.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014455661, 2024-09-14T05:29:32.154 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.154 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.154 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.154 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.154 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.154 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.154 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3469 2024-09-14T05:29:32.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.155 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395601+0000", 2024-09-14T05:29:32.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395601+0000", 2024-09-14T05:29:32.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395601+0000", 2024-09-14T05:29:32.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1199999999999999e-07 2024-09-14T05:29:32.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395602+0000", 2024-09-14T05:29:32.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.122e-06 2024-09-14T05:29:32.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395605+0000", 2024-09-14T05:29:32.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9270000000000001e-06 2024-09-14T05:29:32.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395608+0000", 2024-09-14T05:29:32.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1580000000000001e-06 2024-09-14T05:29:32.157 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.157 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.157 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.157 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3564 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.157 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.396401+0000", 2024-09-14T05:29:32.157 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.006791371, 2024-09-14T05:29:32.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013680454, 2024-09-14T05:29:32.157 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.158 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.158 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.158 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.158 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.158 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.158 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3564 2024-09-14T05:29:32.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.158 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396401+0000", 2024-09-14T05:29:32.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396401+0000", 2024-09-14T05:29:32.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396402+0000", 2024-09-14T05:29:32.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9299999999999997e-07 2024-09-14T05:29:32.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396404+0000", 2024-09-14T05:29:32.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2029999999999999e-06 2024-09-14T05:29:32.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396411+0000", 2024-09-14T05:29:32.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5200000000000003e-06 2024-09-14T05:29:32.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396416+0000", 2024-09-14T05:29:32.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3210000000000004e-06 2024-09-14T05:29:32.160 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.160 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.161 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.161 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3679 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.161 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398279+0000", 2024-09-14T05:29:32.161 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0049130700000000003, 2024-09-14T05:29:32.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011826796000000001, 2024-09-14T05:29:32.161 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.161 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.161 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.161 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.161 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.161 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.162 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3679 2024-09-14T05:29:32.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.162 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398279+0000", 2024-09-14T05:29:32.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398279+0000", 2024-09-14T05:29:32.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398280+0000", 2024-09-14T05:29:32.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-14T05:29:32.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398283+0000", 2024-09-14T05:29:32.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3939999999999999e-06 2024-09-14T05:29:32.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398289+0000", 2024-09-14T05:29:32.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6789999999999999e-06 2024-09-14T05:29:32.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398294+0000", 2024-09-14T05:29:32.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3090000000000003e-06 2024-09-14T05:29:32.164 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.164 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.164 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.164 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3729 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.164 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399012+0000", 2024-09-14T05:29:32.165 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00418054, 2024-09-14T05:29:32.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011120026, 2024-09-14T05:29:32.165 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.165 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.165 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.165 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.165 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.165 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.165 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3729 2024-09-14T05:29:32.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.165 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399012+0000", 2024-09-14T05:29:32.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399012+0000", 2024-09-14T05:29:32.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399012+0000", 2024-09-14T05:29:32.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2500000000000001e-07 2024-09-14T05:29:32.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399014+0000", 2024-09-14T05:29:32.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7239999999999999e-06 2024-09-14T05:29:32.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399020+0000", 2024-09-14T05:29:32.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6969999999999998e-06 2024-09-14T05:29:32.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399025+0000", 2024-09-14T05:29:32.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.809e-06 2024-09-14T05:29:32.167 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.168 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.168 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.168 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3795 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.168 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399679+0000", 2024-09-14T05:29:32.168 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0035132449999999999, 2024-09-14T05:29:32.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010475641000000001, 2024-09-14T05:29:32.168 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.168 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.168 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.168 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.168 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.169 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.169 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3795 2024-09-14T05:29:32.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.169 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399679+0000", 2024-09-14T05:29:32.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399679+0000", 2024-09-14T05:29:32.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399680+0000", 2024-09-14T05:29:32.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6899999999999998e-07 2024-09-14T05:29:32.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399682+0000", 2024-09-14T05:29:32.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.244e-06 2024-09-14T05:29:32.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399689+0000", 2024-09-14T05:29:32.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.579e-06 2024-09-14T05:29:32.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399694+0000", 2024-09-14T05:29:32.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.096e-06 2024-09-14T05:29:32.171 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.171 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.171 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.171 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3844 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.171 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400377+0000", 2024-09-14T05:29:32.171 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.002815222, 2024-09-14T05:29:32.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0098047020000000002, 2024-09-14T05:29:32.172 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.172 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.172 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.172 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.172 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.172 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.172 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3844 2024-09-14T05:29:32.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.172 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400377+0000", 2024-09-14T05:29:32.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400377+0000", 2024-09-14T05:29:32.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400378+0000", 2024-09-14T05:29:32.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.89e-07 2024-09-14T05:29:32.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400379+0000", 2024-09-14T05:29:32.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6249999999999999e-06 2024-09-14T05:29:32.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400385+0000", 2024-09-14T05:29:32.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.071e-06 2024-09-14T05:29:32.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400390+0000", 2024-09-14T05:29:32.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1999999999999996e-06 2024-09-14T05:29:32.174 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.174 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.175 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.175 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3910 3.1e 3.4500c29e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.175 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401041+0000", 2024-09-14T05:29:32.175 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0021512910000000001, 2024-09-14T05:29:32.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091861110000000003, 2024-09-14T05:29:32.175 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.175 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.175 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.175 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.175 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.175 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.176 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3910 2024-09-14T05:29:32.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.176 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401041+0000", 2024-09-14T05:29:32.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401041+0000", 2024-09-14T05:29:32.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401042+0000", 2024-09-14T05:29:32.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.01e-07 2024-09-14T05:29:32.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401044+0000", 2024-09-14T05:29:32.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9659999999999999e-06 2024-09-14T05:29:32.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401051+0000", 2024-09-14T05:29:32.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1269999999999997e-06 2024-09-14T05:29:32.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401056+0000", 2024-09-14T05:29:32.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0919999999999997e-06 2024-09-14T05:29:32.178 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.178 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.178 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.178 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3470 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.178 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395643+0000", 2024-09-14T05:29:32.179 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.007549427, 2024-09-14T05:29:32.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014636082999999999, 2024-09-14T05:29:32.179 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.179 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.179 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.179 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.179 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.179 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.179 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3470 2024-09-14T05:29:32.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.179 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395643+0000", 2024-09-14T05:29:32.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395643+0000", 2024-09-14T05:29:32.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395643+0000", 2024-09-14T05:29:32.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0899999999999997e-07 2024-09-14T05:29:32.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395645+0000", 2024-09-14T05:29:32.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.209e-06 2024-09-14T05:29:32.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395650+0000", 2024-09-14T05:29:32.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2989999999999996e-06 2024-09-14T05:29:32.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395653+0000", 2024-09-14T05:29:32.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.79e-06 2024-09-14T05:29:32.182 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.182 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.182 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.182 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3565 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.182 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.396423+0000", 2024-09-14T05:29:32.182 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0067698469999999998, 2024-09-14T05:29:32.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013896445, 2024-09-14T05:29:32.182 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.182 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.182 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.182 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.182 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.183 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.183 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3565 2024-09-14T05:29:32.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.183 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396423+0000", 2024-09-14T05:29:32.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396423+0000", 2024-09-14T05:29:32.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396423+0000", 2024-09-14T05:29:32.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1600000000000001e-07 2024-09-14T05:29:32.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396425+0000", 2024-09-14T05:29:32.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9570000000000001e-06 2024-09-14T05:29:32.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396431+0000", 2024-09-14T05:29:32.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8069999999999996e-06 2024-09-14T05:29:32.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396436+0000", 2024-09-14T05:29:32.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6240000000000001e-06 2024-09-14T05:29:32.185 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.185 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.185 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.186 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3680 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.186 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398303+0000", 2024-09-14T05:29:32.186 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.004889259, 2024-09-14T05:29:32.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012043641000000001, 2024-09-14T05:29:32.186 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.186 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.186 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.186 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.186 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.186 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.186 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3680 2024-09-14T05:29:32.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.187 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398303+0000", 2024-09-14T05:29:32.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398303+0000", 2024-09-14T05:29:32.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398304+0000", 2024-09-14T05:29:32.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1799999999999995e-07 2024-09-14T05:29:32.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398306+0000", 2024-09-14T05:29:32.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1040000000000001e-06 2024-09-14T05:29:32.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398316+0000", 2024-09-14T05:29:32.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0468e-05 2024-09-14T05:29:32.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398323+0000", 2024-09-14T05:29:32.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.268e-06 2024-09-14T05:29:32.189 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.189 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.189 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.189 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3730 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.189 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399031+0000", 2024-09-14T05:29:32.189 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0041618929999999998, 2024-09-14T05:29:32.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011353714000000001, 2024-09-14T05:29:32.190 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.190 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.190 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.190 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.190 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.190 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.190 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3730 2024-09-14T05:29:32.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.190 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399031+0000", 2024-09-14T05:29:32.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399031+0000", 2024-09-14T05:29:32.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399031+0000", 2024-09-14T05:29:32.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2099999999999996e-07 2024-09-14T05:29:32.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399036+0000", 2024-09-14T05:29:32.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1229999999999999e-06 2024-09-14T05:29:32.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399042+0000", 2024-09-14T05:29:32.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9569999999999999e-06 2024-09-14T05:29:32.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399049+0000", 2024-09-14T05:29:32.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.392e-06 2024-09-14T05:29:32.193 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.193 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.193 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.193 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3796 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.193 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399700+0000", 2024-09-14T05:29:32.193 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0034922270000000001, 2024-09-14T05:29:32.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010733581000000001, 2024-09-14T05:29:32.193 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.193 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.194 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.194 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.194 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.194 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.194 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3796 2024-09-14T05:29:32.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.194 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399700+0000", 2024-09-14T05:29:32.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399700+0000", 2024-09-14T05:29:32.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399701+0000", 2024-09-14T05:29:32.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9500000000000003e-07 2024-09-14T05:29:32.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399703+0000", 2024-09-14T05:29:32.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.001e-06 2024-09-14T05:29:32.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399708+0000", 2024-09-14T05:29:32.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6710000000000004e-06 2024-09-14T05:29:32.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399713+0000", 2024-09-14T05:29:32.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8559999999999998e-06 2024-09-14T05:29:32.196 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.196 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.196 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.197 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3845 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.197 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400395+0000", 2024-09-14T05:29:32.197 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0027977089999999998, 2024-09-14T05:29:32.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010078441, 2024-09-14T05:29:32.197 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.197 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.197 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.197 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3845 2024-09-14T05:29:32.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.198 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400395+0000", 2024-09-14T05:29:32.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400395+0000", 2024-09-14T05:29:32.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400395+0000", 2024-09-14T05:29:32.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6600000000000002e-07 2024-09-14T05:29:32.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400397+0000", 2024-09-14T05:29:32.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6300000000000001e-06 2024-09-14T05:29:32.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400402+0000", 2024-09-14T05:29:32.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.485e-06 2024-09-14T05:29:32.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400407+0000", 2024-09-14T05:29:32.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8640000000000004e-06 2024-09-14T05:29:32.200 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.200 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.200 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.200 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3911 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.200 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401067+0000", 2024-09-14T05:29:32.200 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0021254389999999998, 2024-09-14T05:29:32.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094348709999999992, 2024-09-14T05:29:32.201 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.201 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.201 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.201 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.201 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.201 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.201 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3911 2024-09-14T05:29:32.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.201 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401067+0000", 2024-09-14T05:29:32.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401067+0000", 2024-09-14T05:29:32.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401068+0000", 2024-09-14T05:29:32.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7199999999999999e-07 2024-09-14T05:29:32.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401070+0000", 2024-09-14T05:29:32.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1509999999999998e-06 2024-09-14T05:29:32.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401077+0000", 2024-09-14T05:29:32.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5979999999999999e-06 2024-09-14T05:29:32.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401082+0000", 2024-09-14T05:29:32.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7160000000000002e-06 2024-09-14T05:29:32.203 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.203 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.204 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.204 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3471 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.204 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395656+0000", 2024-09-14T05:29:32.204 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0075363629999999999, 2024-09-14T05:29:32.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014870411, 2024-09-14T05:29:32.204 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.204 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.204 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.204 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.204 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.204 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.204 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3471 2024-09-14T05:29:32.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.205 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395656+0000", 2024-09-14T05:29:32.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395656+0000", 2024-09-14T05:29:32.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395656+0000", 2024-09-14T05:29:32.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3099999999999999e-07 2024-09-14T05:29:32.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395657+0000", 2024-09-14T05:29:32.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0380000000000001e-06 2024-09-14T05:29:32.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395661+0000", 2024-09-14T05:29:32.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0260000000000001e-06 2024-09-14T05:29:32.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395665+0000", 2024-09-14T05:29:32.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6909999999999999e-06 2024-09-14T05:29:32.207 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.207 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.207 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.207 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3566 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.207 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.396451+0000", 2024-09-14T05:29:32.207 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0067419430000000002, 2024-09-14T05:29:32.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014100665, 2024-09-14T05:29:32.207 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.207 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.208 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.208 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3566 2024-09-14T05:29:32.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.208 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396451+0000", 2024-09-14T05:29:32.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396451+0000", 2024-09-14T05:29:32.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396451+0000", 2024-09-14T05:29:32.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-14T05:29:32.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396460+0000", 2024-09-14T05:29:32.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1509999999999993e-06 2024-09-14T05:29:32.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396467+0000", 2024-09-14T05:29:32.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9619999999999996e-06 2024-09-14T05:29:32.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396473+0000", 2024-09-14T05:29:32.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.293e-06 2024-09-14T05:29:32.210 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.210 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.211 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.211 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3681 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.211 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398332+0000", 2024-09-14T05:29:32.211 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0048604449999999997, 2024-09-14T05:29:32.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012243343, 2024-09-14T05:29:32.211 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.211 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.211 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.211 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.211 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.211 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.211 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3681 2024-09-14T05:29:32.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.212 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398332+0000", 2024-09-14T05:29:32.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398332+0000", 2024-09-14T05:29:32.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398333+0000", 2024-09-14T05:29:32.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7700000000000005e-07 2024-09-14T05:29:32.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398335+0000", 2024-09-14T05:29:32.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.368e-06 2024-09-14T05:29:32.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398343+0000", 2024-09-14T05:29:32.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3180000000000007e-06 2024-09-14T05:29:32.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398348+0000", 2024-09-14T05:29:32.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1939999999999997e-06 2024-09-14T05:29:32.214 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.214 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.214 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.214 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3731 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.214 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399055+0000", 2024-09-14T05:29:32.214 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.004137384, 2024-09-14T05:29:32.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011543114, 2024-09-14T05:29:32.214 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.214 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.214 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.215 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.215 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.215 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.215 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3731 2024-09-14T05:29:32.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.215 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399055+0000", 2024-09-14T05:29:32.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399055+0000", 2024-09-14T05:29:32.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399056+0000", 2024-09-14T05:29:32.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-07 2024-09-14T05:29:32.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399058+0000", 2024-09-14T05:29:32.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9429999999999999e-06 2024-09-14T05:29:32.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399064+0000", 2024-09-14T05:29:32.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0519999999999998e-06 2024-09-14T05:29:32.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399068+0000", 2024-09-14T05:29:32.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7049999999999998e-06 2024-09-14T05:29:32.217 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.217 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.217 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.217 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3797 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.217 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399719+0000", 2024-09-14T05:29:32.218 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0034733339999999998, 2024-09-14T05:29:32.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010903417, 2024-09-14T05:29:32.218 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.218 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.218 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.218 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.218 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.218 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.218 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3797 2024-09-14T05:29:32.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.218 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399719+0000", 2024-09-14T05:29:32.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399719+0000", 2024-09-14T05:29:32.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399720+0000", 2024-09-14T05:29:32.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-14T05:29:32.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399722+0000", 2024-09-14T05:29:32.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.933e-06 2024-09-14T05:29:32.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399728+0000", 2024-09-14T05:29:32.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4609999999999996e-06 2024-09-14T05:29:32.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399733+0000", 2024-09-14T05:29:32.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7809999999999996e-06 2024-09-14T05:29:32.221 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.221 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.221 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.221 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3846 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.221 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400413+0000", 2024-09-14T05:29:32.221 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0027795659999999998, 2024-09-14T05:29:32.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010232227999999999, 2024-09-14T05:29:32.222 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.222 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.222 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.222 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.222 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.222 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.222 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3846 2024-09-14T05:29:32.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.222 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400413+0000", 2024-09-14T05:29:32.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400413+0000", 2024-09-14T05:29:32.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400413+0000", 2024-09-14T05:29:32.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4099999999999999e-07 2024-09-14T05:29:32.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400415+0000", 2024-09-14T05:29:32.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.889e-06 2024-09-14T05:29:32.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400421+0000", 2024-09-14T05:29:32.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7760000000000003e-06 2024-09-14T05:29:32.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400426+0000", 2024-09-14T05:29:32.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7729999999999999e-06 2024-09-14T05:29:32.224 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.224 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.224 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.225 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3912 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.225 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401089+0000", 2024-09-14T05:29:32.225 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0021037450000000002, 2024-09-14T05:29:32.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0095820969999999995, 2024-09-14T05:29:32.225 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.225 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.225 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.225 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.225 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.225 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.226 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3912 2024-09-14T05:29:32.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.226 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401089+0000", 2024-09-14T05:29:32.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401089+0000", 2024-09-14T05:29:32.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401089+0000", 2024-09-14T05:29:32.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3500000000000002e-07 2024-09-14T05:29:32.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401091+0000", 2024-09-14T05:29:32.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0990000000000001e-06 2024-09-14T05:29:32.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401098+0000", 2024-09-14T05:29:32.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.562e-06 2024-09-14T05:29:32.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401102+0000", 2024-09-14T05:29:32.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5109999999999997e-06 2024-09-14T05:29:32.228 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.228 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.228 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.228 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3472 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.228 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.395668+0000", 2024-09-14T05:29:32.228 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0075240669999999997, 2024-09-14T05:29:32.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015045507, 2024-09-14T05:29:32.228 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.228 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.228 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.229 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.229 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.229 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.229 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3472 2024-09-14T05:29:32.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.229 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395668+0000", 2024-09-14T05:29:32.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395668+0000", 2024-09-14T05:29:32.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395669+0000", 2024-09-14T05:29:32.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-14T05:29:32.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395670+0000", 2024-09-14T05:29:32.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.057e-06 2024-09-14T05:29:32.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395673+0000", 2024-09-14T05:29:32.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0730000000000001e-06 2024-09-14T05:29:32.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.395675+0000", 2024-09-14T05:29:32.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4820000000000001e-06 2024-09-14T05:29:32.231 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.231 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.231 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.232 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3567 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.232 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.396490+0000", 2024-09-14T05:29:32.232 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0067026969999999996, 2024-09-14T05:29:32.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014274797000000001, 2024-09-14T05:29:32.232 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.232 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.232 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.232 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.232 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.232 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.232 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3567 2024-09-14T05:29:32.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.232 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396490+0000", 2024-09-14T05:29:32.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396490+0000", 2024-09-14T05:29:32.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396491+0000", 2024-09-14T05:29:32.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7899999999999998e-07 2024-09-14T05:29:32.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396493+0000", 2024-09-14T05:29:32.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1050000000000002e-06 2024-09-14T05:29:32.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396499+0000", 2024-09-14T05:29:32.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7900000000000002e-06 2024-09-14T05:29:32.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.396505+0000", 2024-09-14T05:29:32.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4700000000000001e-06 2024-09-14T05:29:32.235 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.235 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.235 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.235 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3682 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.235 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.398355+0000", 2024-09-14T05:29:32.235 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0048373799999999996, 2024-09-14T05:29:32.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012456214, 2024-09-14T05:29:32.235 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.235 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.235 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.236 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.236 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.236 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.236 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3682 2024-09-14T05:29:32.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.236 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398355+0000", 2024-09-14T05:29:32.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398355+0000", 2024-09-14T05:29:32.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398356+0000", 2024-09-14T05:29:32.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9399999999999995e-07 2024-09-14T05:29:32.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398358+0000", 2024-09-14T05:29:32.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0779999999999998e-06 2024-09-14T05:29:32.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398364+0000", 2024-09-14T05:29:32.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1689999999999997e-06 2024-09-14T05:29:32.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.398369+0000", 2024-09-14T05:29:32.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8720000000000001e-06 2024-09-14T05:29:32.238 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.238 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.238 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.239 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3732 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.239 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399074+0000", 2024-09-14T05:29:32.239 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0041181810000000003, 2024-09-14T05:29:32.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011796703, 2024-09-14T05:29:32.239 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.239 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.239 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.239 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.239 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.239 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.239 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3732 2024-09-14T05:29:32.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.240 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399074+0000", 2024-09-14T05:29:32.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399074+0000", 2024-09-14T05:29:32.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399075+0000", 2024-09-14T05:29:32.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9100000000000004e-07 2024-09-14T05:29:32.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399077+0000", 2024-09-14T05:29:32.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7179999999999999e-06 2024-09-14T05:29:32.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399082+0000", 2024-09-14T05:29:32.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5979999999999996e-06 2024-09-14T05:29:32.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399087+0000", 2024-09-14T05:29:32.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8609999999999997e-06 2024-09-14T05:29:32.242 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.242 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.242 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.242 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3798 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.242 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.399744+0000", 2024-09-14T05:29:32.242 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0034480650000000002, 2024-09-14T05:29:32.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011163388, 2024-09-14T05:29:32.243 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.243 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.243 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.243 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.243 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.243 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.243 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3798 2024-09-14T05:29:32.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.243 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399744+0000", 2024-09-14T05:29:32.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399744+0000", 2024-09-14T05:29:32.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399745+0000", 2024-09-14T05:29:32.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2e-07 2024-09-14T05:29:32.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399747+0000", 2024-09-14T05:29:32.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1110000000000002e-06 2024-09-14T05:29:32.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399754+0000", 2024-09-14T05:29:32.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5429999999999999e-06 2024-09-14T05:29:32.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.399759+0000", 2024-09-14T05:29:32.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1630000000000004e-06 2024-09-14T05:29:32.245 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.245 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.246 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.246 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3847 3.1e 3.d76fe27e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.246 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.400431+0000", 2024-09-14T05:29:32.246 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0027614079999999999, 2024-09-14T05:29:32.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010502272999999999, 2024-09-14T05:29:32.246 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.246 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.246 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.246 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.246 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.246 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.246 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3847 2024-09-14T05:29:32.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.247 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400431+0000", 2024-09-14T05:29:32.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400431+0000", 2024-09-14T05:29:32.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400432+0000", 2024-09-14T05:29:32.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5499999999999998e-07 2024-09-14T05:29:32.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400433+0000", 2024-09-14T05:29:32.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5770000000000001e-06 2024-09-14T05:29:32.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400439+0000", 2024-09-14T05:29:32.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-14T05:29:32.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.400443+0000", 2024-09-14T05:29:32.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4390000000000002e-06 2024-09-14T05:29:32.249 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.249 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.249 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.249 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3913 3.1f 3.8263151f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:32.249 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401109+0000", 2024-09-14T05:29:32.249 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0020838710000000002, 2024-09-14T05:29:32.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0098477219999999997, 2024-09-14T05:29:32.250 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:32.250 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:32.250 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:32.250 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:32.250 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:32.250 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:32.250 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3913 2024-09-14T05:29:32.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.250 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:32.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:32.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401109+0000", 2024-09-14T05:29:32.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:32.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401109+0000", 2024-09-14T05:29:32.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:32.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:32.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401109+0000", 2024-09-14T05:29:32.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3599999999999999e-07 2024-09-14T05:29:32.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:32.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401111+0000", 2024-09-14T05:29:32.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.889e-06 2024-09-14T05:29:32.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:32.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401117+0000", 2024-09-14T05:29:32.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7660000000000004e-06 2024-09-14T05:29:32.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:32.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:32.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:32.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401121+0000", 2024-09-14T05:29:32.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3680000000000004e-06 2024-09-14T05:29:32.252 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.252 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:32.252 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.253 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:32.253 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-14T05:29:32.253 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 236 2024-09-14T05:29:32.253 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:32.253 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:29:32.255 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:29:32.261 INFO:teuthology.orchestra.run.smithi071.stdout:212 read 0x5d7ef thru 0x6ad11 (0xd523 bytes) 2024-09-14T05:29:32.261 INFO:teuthology.orchestra.run.smithi071.stdout:213 trunc from 0x3fe9fe to 0x6c5d0db 2024-09-14T05:29:32.261 INFO:teuthology.orchestra.run.smithi071.stdout:214 read 0x37731cc thru 0x377cce1 (0x9b16 bytes) 2024-09-14T05:29:32.261 INFO:teuthology.orchestra.run.smithi071.stdout:215 read 0x63ce3b6 thru 0x63ce9b1 (0x5fc bytes) 2024-09-14T05:29:32.261 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.261 INFO:teuthology.orchestra.run.smithi071.stdout:217 writesame 0xabef038 thru 0xabfc07a data_size 0x1(0xd043 bytes) 2024-09-14T05:29:32.261 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.261 INFO:teuthology.orchestra.run.smithi071.stdout:219 write 0x7f27b62 thru 0x7f3387b (0xbd1a bytes) 2024-09-14T05:29:32.262 INFO:teuthology.orchestra.run.smithi071.stdout:220 writesame 0x879eff7 thru 0x87a77d5 data_size 0x1(0x87df bytes) 2024-09-14T05:29:32.262 INFO:teuthology.orchestra.run.smithi071.stdout:221 trunc from 0xabfc07b to 0x33e447d 2024-09-14T05:29:32.262 INFO:teuthology.orchestra.run.smithi071.stdout:222 write 0x6bace5 thru 0x6c1394 (0x66b0 bytes) 2024-09-14T05:29:32.262 INFO:teuthology.orchestra.run.smithi071.stdout:223 punch from 0x178889a to 0x178dc0c, (0x5372 bytes) 2024-09-14T05:29:32.262 INFO:teuthology.orchestra.run.smithi071.stdout:224 read 0x2100f24 thru 0x210a3b7 (0x9494 bytes) 2024-09-14T05:29:32.262 INFO:teuthology.orchestra.run.smithi071.stdout:225 read 0xf9d736 thru 0xfaaeb5 (0xd780 bytes) 2024-09-14T05:29:32.262 INFO:teuthology.orchestra.run.smithi071.stdout:226 writesame 0xdcf03f9 thru 0xdcfbf97 data_size 0x1(0xbb9f bytes) 2024-09-14T05:29:32.262 INFO:teuthology.orchestra.run.smithi071.stdout:227 writesame 0x28d6bd7 thru 0x28d9f57 data_size 0xf(0x3381 bytes) 2024-09-14T05:29:32.262 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.262 INFO:teuthology.orchestra.run.smithi071.stdout:229 read 0x3f6e778 thru 0x3f7e21f (0xfaa8 bytes) 2024-09-14T05:29:32.262 INFO:teuthology.orchestra.run.smithi071.stdout:230 read 0x70e426f thru 0x70e9b86 (0x5918 bytes) 2024-09-14T05:29:32.262 INFO:teuthology.orchestra.run.smithi071.stdout:231 read 0xa52f877 thru 0xa53eed2 (0xf65c bytes) 2024-09-14T05:29:32.262 INFO:teuthology.orchestra.run.smithi071.stdout:233 write 0x961a744 thru 0x9624ae6 (0xa3a3 bytes) 2024-09-14T05:29:32.263 INFO:teuthology.orchestra.run.smithi071.stdout:234 punch from 0x5059dbb to 0x506085b, (0x6aa0 bytes) 2024-09-14T05:29:32.263 INFO:teuthology.orchestra.run.smithi071.stdout:235 write 0xe8c2b3 thru 0xe95865 (0x95b3 bytes) 2024-09-14T05:29:32.263 INFO:teuthology.orchestra.run.smithi071.stdout:236 writesame 0x7b741ef thru 0x7b7e0b9 data_size 0xd(0x9ecb bytes) 2024-09-14T05:29:32.263 INFO:teuthology.orchestra.run.smithi071.stdout:237 write 0xb3001bd thru 0xb30ecad (0xeaf1 bytes) 2024-09-14T05:29:32.263 INFO:teuthology.orchestra.run.smithi071.stdout:238 read 0xa756b63 thru 0xa7588a0 (0x1d3e bytes) 2024-09-14T05:29:32.263 INFO:teuthology.orchestra.run.smithi071.stdout:240 read 0x1604d84 thru 0x1610cdc (0xbf59 bytes) 2024-09-14T05:29:32.263 INFO:teuthology.orchestra.run.smithi071.stdout:241 write 0xe706de8 thru 0xe70f571 (0x878a bytes) 2024-09-14T05:29:32.263 INFO:teuthology.orchestra.run.smithi071.stdout:242 writesame 0xdd5e869 thru 0xdd6288d data_size 0x1(0x4025 bytes) 2024-09-14T05:29:32.263 INFO:teuthology.orchestra.run.smithi071.stdout:243 writesame 0x440f4f7 thru 0x441b650 data_size 0x1(0xc15a bytes) 2024-09-14T05:29:32.263 INFO:teuthology.orchestra.run.smithi071.stdout:245 read 0x3134cea thru 0x3135ec8 (0x11df bytes) 2024-09-14T05:29:32.263 INFO:teuthology.orchestra.run.smithi071.stdout:247 writesame 0x9162e39 thru 0x916338a data_size 0x3(0x552 bytes) 2024-09-14T05:29:32.263 INFO:teuthology.orchestra.run.smithi071.stdout:248 punch from 0x16b8e7c to 0x16bf67e, (0x6802 bytes) 2024-09-14T05:29:32.263 INFO:teuthology.orchestra.run.smithi071.stdout:249 read 0x7999137 thru 0x799ef6e (0x5e38 bytes) 2024-09-14T05:29:32.263 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.264 INFO:teuthology.orchestra.run.smithi071.stdout:254 read 0x7c33670 thru 0x7c34723 (0x10b4 bytes) 2024-09-14T05:29:32.264 INFO:teuthology.orchestra.run.smithi071.stdout:255 read 0x5fd48ca thru 0x5fdf527 (0xac5e bytes) 2024-09-14T05:29:32.264 INFO:teuthology.orchestra.run.smithi071.stdout:257 writesame 0x23c5d12 thru 0x23d3a8a data_size 0x1(0xdd79 bytes) 2024-09-14T05:29:32.264 INFO:teuthology.orchestra.run.smithi071.stdout:259 trunc from 0xe70f572 to 0x6569320 2024-09-14T05:29:32.264 INFO:teuthology.orchestra.run.smithi071.stdout:260 writesame 0x62b5204 thru 0x62c26e2 data_size 0x3f(0xd4df bytes) 2024-09-14T05:29:32.264 INFO:teuthology.orchestra.run.smithi071.stdout:262 write 0xa25143c thru 0xa260171 (0xed36 bytes) 2024-09-14T05:29:32.264 INFO:teuthology.orchestra.run.smithi071.stdout:263 trunc from 0xa260172 to 0xaf37340 2024-09-14T05:29:32.264 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.264 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.264 INFO:teuthology.orchestra.run.smithi071.stdout:266 write 0x3cdfabb thru 0x3ce216a (0x26b0 bytes) 2024-09-14T05:29:32.264 INFO:teuthology.orchestra.run.smithi071.stdout:267 read 0x19e2e59 thru 0x19ef963 (0xcb0b bytes) 2024-09-14T05:29:32.264 INFO:teuthology.orchestra.run.smithi071.stdout:268 punch from 0x954f629 to 0x9552420, (0x2df7 bytes) 2024-09-14T05:29:32.264 INFO:teuthology.orchestra.run.smithi071.stdout:270 write 0x2d1f125 thru 0x2d25101 (0x5fdd bytes) 2024-09-14T05:29:32.265 INFO:teuthology.orchestra.run.smithi071.stdout:271 punch from 0x50e05e4 to 0x50ecd77, (0xc793 bytes) 2024-09-14T05:29:32.265 INFO:teuthology.orchestra.run.smithi071.stdout:274 punch from 0xa56b85 to 0xa6089c, (0x9d17 bytes) 2024-09-14T05:29:32.265 INFO:teuthology.orchestra.run.smithi071.stdout:276 write 0x9c08492 thru 0x9c12d49 (0xa8b8 bytes) 2024-09-14T05:29:32.265 INFO:teuthology.orchestra.run.smithi071.stdout:277 read 0x3ef5463 thru 0x3f03a8d (0xe62b bytes) 2024-09-14T05:29:32.265 INFO:teuthology.orchestra.run.smithi071.stdout:279 trunc from 0xaf37340 to 0x14cae94 2024-09-14T05:29:32.265 INFO:teuthology.orchestra.run.smithi071.stdout:280 writesame 0x31aa2a9 thru 0x31b602c data_size 0x4e(0xbd84 bytes) 2024-09-14T05:29:32.265 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.265 INFO:teuthology.orchestra.run.smithi071.stdout:282 punch from 0xcf1788 to 0xcfb108, (0x9980 bytes) 2024-09-14T05:29:32.265 INFO:teuthology.orchestra.run.smithi071.stdout:283 write 0x4b77f31 thru 0x4b84781 (0xc851 bytes) 2024-09-14T05:29:32.265 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.265 INFO:teuthology.orchestra.run.smithi071.stdout:285 punch from 0x7d5eb7 to 0x7d699f, (0xae8 bytes) 2024-09-14T05:29:32.265 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.265 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.266 INFO:teuthology.orchestra.run.smithi071.stdout:290 write 0x9c73ac thru 0x9d3140 (0xbd95 bytes) 2024-09-14T05:29:32.266 INFO:teuthology.orchestra.run.smithi071.stdout:291 read 0x2227ec0 thru 0x2235b27 (0xdc68 bytes) 2024-09-14T05:29:32.266 INFO:teuthology.orchestra.run.smithi071.stdout:292 write 0x3f3b7f6 thru 0x3f3c9fa (0x1205 bytes) 2024-09-14T05:29:32.266 INFO:teuthology.orchestra.run.smithi071.stdout:293 writesame 0xddb5c77 thru 0xddc5368 data_size 0x49(0xf6f2 bytes) 2024-09-14T05:29:32.266 INFO:teuthology.orchestra.run.smithi071.stdout:294 read 0x76d8aee thru 0x76e03a7 (0x78ba bytes) 2024-09-14T05:29:32.266 INFO:teuthology.orchestra.run.smithi071.stdout:295 punch from 0x506a11a to 0x5071311, (0x71f7 bytes) 2024-09-14T05:29:32.266 INFO:teuthology.orchestra.run.smithi071.stdout:296 writesame 0x8c427c6 thru 0x8c4e086 data_size 0x1(0xb8c1 bytes) 2024-09-14T05:29:32.266 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.266 INFO:teuthology.orchestra.run.smithi071.stdout:299 write 0x41b5a24 thru 0x41c4670 (0xec4d bytes) 2024-09-14T05:29:32.266 INFO:teuthology.orchestra.run.smithi071.stdout:300 writesame 0x6b0c38 thru 0x6c06f8 data_size 0x1(0xfac1 bytes) 2024-09-14T05:29:32.266 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-14T05:29:32.267 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.267 INFO:teuthology.orchestra.run.smithi071.stdout:302 punch from 0x4723062 to 0x472ee1a, (0xbdb8 bytes) 2024-09-14T05:29:32.267 INFO:teuthology.orchestra.run.smithi071.stdout:303 write 0x733f190 thru 0x7343903 (0x4774 bytes) 2024-09-14T05:29:32.272 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.273 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.273 INFO:teuthology.orchestra.run.smithi071.stdout:307 read 0x685b3bf thru 0x686840a (0xd04c bytes) 2024-09-14T05:29:32.273 INFO:teuthology.orchestra.run.smithi071.stdout:308 write 0x485f83b thru 0x486b5ff (0xbdc5 bytes) 2024-09-14T05:29:32.273 INFO:teuthology.orchestra.run.smithi071.stdout:310 read 0x3bc16bb thru 0x3bca59c (0x8ee2 bytes) 2024-09-14T05:29:32.273 INFO:teuthology.orchestra.run.smithi071.stdout:313 punch from 0x550c5f9 to 0x55199e0, (0xd3e7 bytes) 2024-09-14T05:29:32.273 INFO:teuthology.orchestra.run.smithi071.stdout:317 write 0xe2bba06 thru 0xe2c08d1 (0x4ecc bytes) 2024-09-14T05:29:32.273 INFO:teuthology.orchestra.run.smithi071.stdout:318 write 0xe9788b7 thru 0xe9846dc (0xbe26 bytes) 2024-09-14T05:29:32.273 INFO:teuthology.orchestra.run.smithi071.stdout:319 read 0x61f88bd thru 0x62038a1 (0xafe5 bytes) 2024-09-14T05:29:32.273 INFO:teuthology.orchestra.run.smithi071.stdout:320 read 0x35470a8 thru 0x354b6e3 (0x463c bytes) 2024-09-14T05:29:32.273 INFO:teuthology.orchestra.run.smithi071.stdout:321 write 0x70a6429 thru 0x70a9229 (0x2e01 bytes) 2024-09-14T05:29:32.273 INFO:teuthology.orchestra.run.smithi071.stdout:322 punch from 0x987b2cc to 0x987e87a, (0x35ae bytes) 2024-09-14T05:29:32.273 INFO:teuthology.orchestra.run.smithi071.stdout:323 write 0x1d1674a thru 0x1d1f313 (0x8bca bytes) 2024-09-14T05:29:32.274 INFO:teuthology.orchestra.run.smithi071.stdout:324 writesame 0x792cbd1 thru 0x792e26a data_size 0x2(0x169a bytes) 2024-09-14T05:29:32.274 INFO:teuthology.orchestra.run.smithi071.stdout:325 writesame 0x6dc97b thru 0x6e533a data_size 0x26(0x89c0 bytes) 2024-09-14T05:29:32.274 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.274 INFO:teuthology.orchestra.run.smithi071.stdout:328 writesame 0x953f30 thru 0x961d8e data_size 0x1d(0xde5f bytes) 2024-09-14T05:29:32.274 INFO:teuthology.orchestra.run.smithi071.stdout:329 writesame 0x847c1e9 thru 0x847f82a data_size 0xf(0x3642 bytes) 2024-09-14T05:29:32.274 INFO:teuthology.orchestra.run.smithi071.stdout:330 punch from 0x2a28b27 to 0x2a34e19, (0xc2f2 bytes) 2024-09-14T05:29:32.274 INFO:teuthology.orchestra.run.smithi071.stdout:331 read 0x782c4a1 thru 0x783675f (0xa2bf bytes) 2024-09-14T05:29:32.274 INFO:teuthology.orchestra.run.smithi071.stdout:334 trunc from 0xe9846dd to 0x78348bd 2024-09-14T05:29:32.274 INFO:teuthology.orchestra.run.smithi071.stdout:336 read 0x3274184 thru 0x327b63e (0x74bb bytes) 2024-09-14T05:29:32.274 INFO:teuthology.orchestra.run.smithi071.stdout:337 read 0x3b34dfa thru 0x3b403b9 (0xb5c0 bytes) 2024-09-14T05:29:32.274 INFO:teuthology.orchestra.run.smithi071.stdout:339 write 0x1c549a thru 0x1c7b10 (0x2677 bytes) 2024-09-14T05:29:32.274 INFO:teuthology.orchestra.run.smithi071.stdout:340 punch from 0x3926c92 to 0x392c54d, (0x58bb bytes) 2024-09-14T05:29:32.274 INFO:teuthology.orchestra.run.smithi071.stdout:342 writesame 0xcc2f80a thru 0xcc3af4c data_size 0xb(0xb743 bytes) 2024-09-14T05:29:32.275 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.275 INFO:teuthology.orchestra.run.smithi071.stdout:345 writesame 0x8b5ca3d thru 0x8b6a0f6 data_size 0x2e(0xd6ba bytes) 2024-09-14T05:29:32.275 INFO:teuthology.orchestra.run.smithi071.stdout:347 read 0x10c20a3 thru 0x10d18a6 (0xf804 bytes) 2024-09-14T05:29:32.275 INFO:teuthology.orchestra.run.smithi071.stdout:348 write 0xbc29ea5 thru 0xbc2d5c1 (0x371d bytes) 2024-09-14T05:29:32.275 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.275 INFO:teuthology.orchestra.run.smithi071.stdout:350 read 0x10b4c6e thru 0x10b53fc (0x78f bytes) 2024-09-14T05:29:32.275 INFO:teuthology.orchestra.run.smithi071.stdout:351 punch from 0x52ec270 to 0x52f5347, (0x90d7 bytes) 2024-09-14T05:29:32.275 INFO:teuthology.orchestra.run.smithi071.stdout:352 write 0xea76533 thru 0xea78c07 (0x26d5 bytes) 2024-09-14T05:29:32.275 INFO:teuthology.orchestra.run.smithi071.stdout:353 write 0xa267324 thru 0xa275931 (0xe60e bytes) 2024-09-14T05:29:32.275 INFO:teuthology.orchestra.run.smithi071.stdout:354 read 0xd993b5 thru 0xd9fe6c (0x6ab8 bytes) 2024-09-14T05:29:32.275 INFO:teuthology.orchestra.run.smithi071.stdout:356 writesame 0xd908eee thru 0xd90eb28 data_size 0x1(0x5c3b bytes) 2024-09-14T05:29:32.275 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.275 INFO:teuthology.orchestra.run.smithi071.stdout:359 write 0xad91d0 thru 0xae4a6a (0xb89b bytes) 2024-09-14T05:29:32.275 INFO:teuthology.orchestra.run.smithi071.stdout:360 read 0xa9e8ef thru 0xaa15b5 (0x2cc7 bytes) 2024-09-14T05:29:32.275 INFO:teuthology.orchestra.run.smithi071.stdout:361 punch from 0x4e55ac2 to 0x4e658e9, (0xfe27 bytes) 2024-09-14T05:29:32.276 INFO:teuthology.orchestra.run.smithi071.stdout:362 read 0x9ba16cc thru 0x9bacb12 (0xb447 bytes) 2024-09-14T05:29:32.276 INFO:teuthology.orchestra.run.smithi071.stdout:363 read 0x5825745 thru 0x58261bc (0xa78 bytes) 2024-09-14T05:29:32.276 INFO:teuthology.orchestra.run.smithi071.stdout:364 read 0x2ed8e2b thru 0x2ed987e (0xa54 bytes) 2024-09-14T05:29:32.276 INFO:teuthology.orchestra.run.smithi071.stdout:365 trunc from 0xea78c08 to 0x16307b4 2024-09-14T05:29:32.276 INFO:teuthology.orchestra.run.smithi071.stdout:367 punch from 0xe2ddd to 0xefc6d, (0xce90 bytes) 2024-09-14T05:29:32.276 INFO:teuthology.orchestra.run.smithi071.stdout:368 punch from 0xbe3da5 to 0xbedb64, (0x9dbf bytes) 2024-09-14T05:29:32.276 INFO:teuthology.orchestra.run.smithi071.stdout:369 read 0x489bcf thru 0x49383c (0x9c6e bytes) 2024-09-14T05:29:32.276 INFO:teuthology.orchestra.run.smithi071.stdout:370 read 0xdf36c2 thru 0xdffefc (0xc83b bytes) 2024-09-14T05:29:32.276 INFO:teuthology.orchestra.run.smithi071.stdout:371 writesame 0x7599f87 thru 0x759cef1 data_size 0x1(0x2f6b bytes) 2024-09-14T05:29:32.276 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.276 INFO:teuthology.orchestra.run.smithi071.stdout:373 read 0x1315c83 thru 0x131c99c (0x6d1a bytes) 2024-09-14T05:29:32.276 INFO:teuthology.orchestra.run.smithi071.stdout:374 trunc from 0x759cef2 to 0x521c801 2024-09-14T05:29:32.276 INFO:teuthology.orchestra.run.smithi071.stdout:375 writesame 0x38a41ff thru 0x38b0651 data_size 0xb(0xc453 bytes) 2024-09-14T05:29:32.276 INFO:teuthology.orchestra.run.smithi071.stdout:376 read 0x4c8f2fb thru 0x4c9ebc7 (0xf8cd bytes) 2024-09-14T05:29:32.277 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:32.253+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:32.277 INFO:teuthology.orchestra.run.smithi071.stdout:378 trunc from 0x521c801 to 0x8f1c1a1 2024-09-14T05:29:32.278 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.278 INFO:teuthology.orchestra.run.smithi071.stdout:380 write 0xb40dd9 thru 0xb48f34 (0x815c bytes) 2024-09-14T05:29:32.278 INFO:teuthology.orchestra.run.smithi071.stdout:381 read 0x89fae47 thru 0x8a0759f (0xc759 bytes) 2024-09-14T05:29:32.278 INFO:teuthology.orchestra.run.smithi071.stdout:382 writesame 0x11e2293 thru 0x11ef196 data_size 0x2(0xcf04 bytes) 2024-09-14T05:29:32.282 INFO:teuthology.orchestra.run.smithi071.stdout:383 trunc from 0x8f1c1a1 to 0x1058683 2024-09-14T05:29:32.296 INFO:teuthology.orchestra.run.smithi071.stdout:384 read 0xfd588b thru 0xfe1b9a (0xc310 bytes) 2024-09-14T05:29:32.296 INFO:teuthology.orchestra.run.smithi071.stdout:386 read 0x7bc530 thru 0x7c5937 (0x9408 bytes) 2024-09-14T05:29:32.296 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:32.297 INFO:teuthology.orchestra.run.smithi071.stdout:388 read 0x41be01 thru 0x422de5 (0x6fe5 bytes) 2024-09-14T05:29:32.308 INFO:teuthology.orchestra.run.smithi071.stdout:390 write 0x5c17a75 thru 0x5c18771 (0xcfd bytes) 2024-09-14T05:29:32.315 INFO:teuthology.orchestra.run.smithi071.stdout:391 write 0x18f684b thru 0x18fa3d1 (0x3b87 bytes) 2024-09-14T05:29:32.318 INFO:teuthology.orchestra.run.smithi071.stdout:397 trunc from 0x5c18772 to 0x51a2501 2024-09-14T05:29:32.335 INFO:teuthology.orchestra.run.smithi071.stdout:399 write 0x8f380ce thru 0x8f46aa5 (0xe9d8 bytes) 2024-09-14T05:29:32.341 INFO:teuthology.orchestra.run.smithi071.stdout:400 read 0x453aa86 thru 0x45451c0 (0xa73b bytes) 2024-09-14T05:29:32.353 INFO:teuthology.orchestra.run.smithi071.stdout:401 writesame 0xe44deef thru 0xe45a95b data_size 0xb(0xca6d bytes) 2024-09-14T05:29:32.355 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:32.354+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:32.368 INFO:teuthology.orchestra.run.smithi071.stdout:402 trunc from 0xe45a95c to 0x1844c11 2024-09-14T05:29:32.368 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:32.369 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:32.369 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:29:32.369 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:29:32.369 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:32.378 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:29:32.392 INFO:teuthology.orchestra.run.smithi071.stdout:403 write 0xbdb17b thru 0xbe6076 (0xaefc bytes) 2024-09-14T05:29:32.394 INFO:teuthology.orchestra.run.smithi071.stdout:404 clone 5 order 20 su 262144 sc 12 2024-09-14T05:29:32.416 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:29:32.417 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:29:32.433 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:29:32.433 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:29:32.450 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:29:32.450 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:29:32.455 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:32.455+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:32.465 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:29:32.465 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:29:32.482 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:29:32.482 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:29:32.498 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:29:32.498 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:29:32.515 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:29:32.515 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:29:32.524 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:32.524 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:32.531 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:29:32.531 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:29:32.533 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:29:32.583 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:32.582+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:32.635 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:32.636 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:32.644 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:29:32.658 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:32.657+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:32.746 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:32.746 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:32.755 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:29:32.800 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:32.799+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:32.859 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:32.859 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:32.859 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:32.859+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:32.867 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:29:32.960 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:32.960+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:33.010 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:33.010 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:33.018 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:29:33.061 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:33.060+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:33.149 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone4 from 0x1844c11 (overlap 0x3fe9fe) to 0x332198 2024-09-14T05:29:33.162 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:33.161+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:33.173 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:33.173 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:33.183 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:29:33.236 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-14T05:29:33.237 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:33.237 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:33.251 INFO:teuthology.orchestra.run.smithi071.stdout:408 trunc from 0x1844c11 to 0xd2186d1 2024-09-14T05:29:33.259 INFO:teuthology.orchestra.run.smithi071.stdout:410 write 0x944d043 thru 0x944d5d8 (0x596 bytes) 2024-09-14T05:29:33.261 INFO:teuthology.orchestra.run.smithi071.stdout:412 write 0x8a9ca8e thru 0x8aa8977 (0xbeea bytes) 2024-09-14T05:29:33.264 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:33.264 INFO:teuthology.orchestra.run.smithi071.stdout:415 flatten 2024-09-14T05:29:33.266 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:33.266+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:33.325 INFO:teuthology.orchestra.run.smithi071.stdout:416 write 0xde37b78 thru 0xde42d96 (0xb21f bytes) 2024-09-14T05:29:33.340 INFO:teuthology.orchestra.run.smithi071.stdout:417 trunc from 0xde42d97 to 0x408158e 2024-09-14T05:29:33.344 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:33.344 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:33.350 INFO:teuthology.orchestra.run.smithi071.stdout:418 punch from 0x18ad98e to 0x18b2ee8, (0x555a bytes) 2024-09-14T05:29:33.353 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:29:33.359 INFO:teuthology.orchestra.run.smithi071.stdout:419 trunc from 0x408158e to 0xb7e9794 2024-09-14T05:29:33.374 INFO:teuthology.orchestra.run.smithi071.stdout:420 trunc from 0xb7e9794 to 0x37d6fa0 2024-09-14T05:29:33.390 INFO:teuthology.orchestra.run.smithi071.stdout:421 trunc from 0x37d6fa0 to 0xb40f09f 2024-09-14T05:29:33.394 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:33.394+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:33.395 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T05:29:33.395 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T05:29:33.395 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-14T05:29:33.395 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T05:29:33.395 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T05:29:33.396 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-14T05:29:33.396 INFO:teuthology.orchestra.run.smithi071.stdout:422 writesame 0x9430155 thru 0x943e772 data_size 0xa(0xe61e bytes) 2024-09-14T05:29:33.400 INFO:teuthology.orchestra.run.smithi071.stdout:423 write 0x652a252 thru 0x652a320 (0xcf bytes) 2024-09-14T05:29:33.403 INFO:teuthology.orchestra.run.smithi071.stdout:424 read 0x6c1b1f3 thru 0x6c2876e (0xd57c bytes) 2024-09-14T05:29:33.404 INFO:teuthology.orchestra.run.smithi071.stdout:426 write 0x47cf923 thru 0x47d633d (0x6a1b bytes) 2024-09-14T05:29:33.406 INFO:teuthology.orchestra.run.smithi071.stdout:427 writesame 0x78c19d8 thru 0x78c6d7c data_size 0x31(0x53a5 bytes) 2024-09-14T05:29:33.419 INFO:teuthology.orchestra.run.smithi071.stdout:428 write 0xe3a2231 thru 0xe3acbbc (0xa98c bytes) 2024-09-14T05:29:33.422 INFO:teuthology.orchestra.run.smithi071.stdout:429 write 0xaceadd3 thru 0xacf226c (0x749a bytes) 2024-09-14T05:29:33.425 INFO:teuthology.orchestra.run.smithi071.stdout:430 read 0x1e833ca thru 0x1e913e6 (0xe01d bytes) 2024-09-14T05:29:33.425 INFO:teuthology.orchestra.run.smithi071.stdout:432 write 0x5878cc1 thru 0x588578d (0xcacd bytes) 2024-09-14T05:29:33.445 INFO:teuthology.orchestra.run.smithi071.stdout:433 trunc from 0xe3acbbd to 0xad0d12 2024-09-14T05:29:33.459 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:33.463 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:33.463+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:33.473 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:33.473 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:33.477 INFO:teuthology.orchestra.run.smithi071.stdout:435 write 0xb07cb5b thru 0xb086dcd (0xa273 bytes) 2024-09-14T05:29:33.483 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:29:33.486 INFO:teuthology.orchestra.run.smithi071.stdout:437 trunc from 0xb086dce to 0x84f0819 2024-09-14T05:29:33.496 INFO:teuthology.orchestra.run.smithi071.stdout:438 write 0x7ff5c2e thru 0x7ffae05 (0x51d8 bytes) 2024-09-14T05:29:33.499 INFO:teuthology.orchestra.run.smithi071.stdout:439 punch from 0x7ae90a1 to 0x7af04ba, (0x7419 bytes) 2024-09-14T05:29:33.501 INFO:teuthology.orchestra.run.smithi071.stdout:441 trunc from 0x84f0819 to 0xa91cc79 2024-09-14T05:29:33.503 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:33.503 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T05:29:33.503 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:33.507 INFO:teuthology.orchestra.run.smithi071.stdout:442 write 0x888bb6c thru 0x8891d82 (0x6217 bytes) 2024-09-14T05:29:33.510 INFO:teuthology.orchestra.run.smithi071.stdout:445 clone 6 order 20 su 16384 sc 3 2024-09-14T05:29:33.564 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:33.563+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:33.589 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:33.590 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:33.598 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:29:33.682 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:33.682+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:33.741 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:33.741 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:33.750 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:29:33.791 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:33.790+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:33.866 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:33.865+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:33.892 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:33.892 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:33.900 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:29:33.985 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:33.984+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:34.053 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:34.053 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:34.062 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:29:34.068 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:34.067+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:34.168 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:34.168+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:34.200 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-14T05:29:34.209 INFO:teuthology.orchestra.run.smithi071.stdout:446 read 0xa55f23e thru 0xa56c3bf (0xd182 bytes) 2024-09-14T05:29:34.210 INFO:teuthology.orchestra.run.smithi071.stdout:447 read 0x99408ef thru 0x9943623 (0x2d35 bytes) 2024-09-14T05:29:34.210 INFO:teuthology.orchestra.run.smithi071.stdout:448 write 0x82c2c29 thru 0x82ca333 (0x770b bytes) 2024-09-14T05:29:34.212 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:34.212 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:34.222 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:29:34.223 INFO:teuthology.orchestra.run.smithi071.stdout:449 trunc from 0xa91cc79 to 0xaa71d12 2024-09-14T05:29:34.229 INFO:teuthology.orchestra.run.smithi071.stdout:450 write 0x5a53f6b thru 0x5a5e380 (0xa416 bytes) 2024-09-14T05:29:34.239 INFO:teuthology.orchestra.run.smithi071.stdout:452 punch from 0x662f0d5 to 0x663079c, (0x16c7 bytes) 2024-09-14T05:29:34.239 INFO:teuthology.orchestra.run.smithi071.stdout:454 writesame 0x3a50799 thru 0x3a5ee9b data_size 0x3(0xe703 bytes) 2024-09-14T05:29:34.255 INFO:teuthology.orchestra.run.smithi071.stdout:455 write 0x5f7bba thru 0x602735 (0xab7c bytes) 2024-09-14T05:29:34.269 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T05:29:34.269 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:34.269+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:34.284 INFO:teuthology.orchestra.run.smithi071.stdout:456 write 0x3557538 thru 0x3559156 (0x1c1f bytes) 2024-09-14T05:29:34.296 INFO:teuthology.orchestra.run.smithi071.stdout:458 write 0xd256cd8 thru 0xd25d46f (0x6798 bytes) 2024-09-14T05:29:34.301 INFO:teuthology.orchestra.run.smithi071.stdout:459 writesame 0xcef989e thru 0xcf00286 data_size 0x1(0x69e9 bytes) 2024-09-14T05:29:34.312 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:34.312 INFO:teuthology.orchestra.run.smithi071.stdout:463 punch from 0x644ef3c to 0x6450106, (0x11ca bytes) 2024-09-14T05:29:34.312 INFO:teuthology.orchestra.run.smithi071.stdout:464 writesame 0xa1548ca thru 0xa156375 data_size 0x6(0x1aac bytes) 2024-09-14T05:29:34.316 INFO:teuthology.orchestra.run.smithi071.stdout:466 punch from 0xb183e1c to 0xb1851fa, (0x13de bytes) 2024-09-14T05:29:34.316 INFO:teuthology.orchestra.run.smithi071.stdout:469 punch from 0x90bd3d to 0x910135, (0x43f8 bytes) 2024-09-14T05:29:34.316 INFO:teuthology.orchestra.run.smithi071.stdout:470 write 0x76ce458 thru 0x76da254 (0xbdfd bytes) 2024-09-14T05:29:34.335 INFO:teuthology.orchestra.run.smithi071.stdout:471 trunc from 0xd25d470 to 0x221290e 2024-09-14T05:29:34.344 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:34.345 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:34.354 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:29:34.358 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:34.358 INFO:teuthology.orchestra.run.smithi071.stdout:473 write 0x20cff2c thru 0x20dc974 (0xca49 bytes) 2024-09-14T05:29:34.370 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:34.369+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:34.386 INFO:teuthology.orchestra.run.smithi071.stdout:474 write 0xbeb883d thru 0xbec65dd (0xdda1 bytes) 2024-09-14T05:29:34.401 INFO:teuthology.orchestra.run.smithi071.stdout:475 writesame 0xd642691 thru 0xd64b358 data_size 0x35(0x8cc8 bytes) 2024-09-14T05:29:34.401 INFO:teuthology.orchestra.run.smithi071.stdout:478 write 0xa40ef31 thru 0xa413c08 (0x4cd8 bytes) 2024-09-14T05:29:34.407 INFO:teuthology.orchestra.run.smithi071.stdout:479 punch from 0x1863750 to 0x186b283, (0x7b33 bytes) 2024-09-14T05:29:34.407 INFO:teuthology.orchestra.run.smithi071.stdout:481 write 0x8d3fb0b thru 0x8d4917b (0x9671 bytes) 2024-09-14T05:29:34.415 INFO:teuthology.orchestra.run.smithi071.stdout:482 punch from 0xd3e9166 to 0xd3f0e63, (0x7cfd bytes) 2024-09-14T05:29:34.415 INFO:teuthology.orchestra.run.smithi071.stdout:483 punch from 0xa02fc69 to 0xa03672a, (0x6ac1 bytes) 2024-09-14T05:29:34.428 INFO:teuthology.orchestra.run.smithi071.stdout:484 trunc from 0xd64b359 to 0x2547a44 2024-09-14T05:29:34.458 INFO:teuthology.orchestra.run.smithi071.stdout:486 write 0x83b292f thru 0x83b4283 (0x1955 bytes) 2024-09-14T05:29:34.467 INFO:teuthology.orchestra.run.smithi071.stdout:487 write 0x8c4ba96 thru 0x8c4bdf1 (0x35c bytes) 2024-09-14T05:29:34.471 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:34.470+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:34.472 INFO:teuthology.orchestra.run.smithi071.stdout:488 writesame 0x36170c6 thru 0x3621b92 data_size 0x4b(0xaacd bytes) 2024-09-14T05:29:34.476 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:34.476 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:34.484 INFO:teuthology.orchestra.run.smithi071.stdout:489 trunc from 0x8c4bdf2 to 0xb3fa76d 2024-09-14T05:29:34.487 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:29:34.492 INFO:teuthology.orchestra.run.smithi071.stdout:491 read 0x5a0859e thru 0x5a0d985 (0x53e8 bytes) 2024-09-14T05:29:34.492 INFO:teuthology.orchestra.run.smithi071.stdout:493 read 0x8f2bef2 thru 0x8f367cb (0xa8da bytes) 2024-09-14T05:29:34.492 INFO:teuthology.orchestra.run.smithi071.stdout:495 read 0x9377703 thru 0x9381bf9 (0xa4f7 bytes) 2024-09-14T05:29:34.492 INFO:teuthology.orchestra.run.smithi071.stdout:497 write 0x5d9d6c3 thru 0x5da14e6 (0x3e24 bytes) 2024-09-14T05:29:34.495 INFO:teuthology.orchestra.run.smithi071.stdout:498 write 0xa95fc53 thru 0xa964707 (0x4ab5 bytes) 2024-09-14T05:29:34.500 INFO:teuthology.orchestra.run.smithi071.stdout:501 writesame 0x19164b4 thru 0x191ac0d data_size 0x1(0x475a bytes) 2024-09-14T05:29:34.508 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:34.508 INFO:teuthology.orchestra.run.smithi071.stdout:503 write 0x28da780 thru 0x28dfe16 (0x5697 bytes) 2024-09-14T05:29:34.522 INFO:teuthology.orchestra.run.smithi071.stdout:504 trunc from 0xb3fa76d to 0x2935c5a 2024-09-14T05:29:34.542 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:29:34.543 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:34.543 INFO:teuthology.orchestra.run.smithi071.stdout:506 write 0x28faa91 thru 0x28fce7e (0x23ee bytes) 2024-09-14T05:29:34.567 INFO:teuthology.orchestra.run.smithi071.stdout:509 write 0xe87f360 thru 0xe885de0 (0x6a81 bytes) 2024-09-14T05:29:34.571 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:34.571+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:34.574 INFO:teuthology.orchestra.run.smithi071.stdout:510 writesame 0x471dc81 thru 0x4729943 data_size 0x17(0xbcc3 bytes) 2024-09-14T05:29:34.580 INFO:teuthology.orchestra.run.smithi071.stdout:511 read 0xdb4efa4 thru 0xdb58a19 (0x9a76 bytes) 2024-09-14T05:29:34.580 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:34.580 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:34.580 INFO:teuthology.orchestra.run.smithi071.stdout:516 read 0x6406ba7 thru 0x640db07 (0x6f61 bytes) 2024-09-14T05:29:34.580 INFO:teuthology.orchestra.run.smithi071.stdout:518 punch from 0xd8b8990 to 0xd8bbb59, (0x31c9 bytes) 2024-09-14T05:29:34.580 INFO:teuthology.orchestra.run.smithi071.stdout:519 write 0x569ca22 thru 0x56a4c85 (0x8264 bytes) 2024-09-14T05:29:34.589 INFO:teuthology.orchestra.run.smithi071.stdout:522 read 0xb888972 thru 0xb88d946 (0x4fd5 bytes) 2024-09-14T05:29:34.589 INFO:teuthology.orchestra.run.smithi071.stdout:524 write 0xb278a11 thru 0xb2859a5 (0xcf95 bytes) 2024-09-14T05:29:34.594 INFO:teuthology.orchestra.run.smithi071.stdout:527 writesame 0x91834c thru 0x92414c data_size 0x1(0xbe01 bytes) 2024-09-14T05:29:34.616 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:34.617 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:34.621 INFO:teuthology.orchestra.run.smithi071.stdout:528 write 0xdf47d91 thru 0xdf56dd9 (0xf049 bytes) 2024-09-14T05:29:34.626 INFO:teuthology.orchestra.run.smithi071.stdout:529 punch from 0x80a2284 to 0x80a69e5, (0x4761 bytes) 2024-09-14T05:29:34.626 INFO:teuthology.orchestra.run.smithi071.stdout:531 write 0x66eb2ec thru 0x66ef5f4 (0x4309 bytes) 2024-09-14T05:29:34.627 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:29:34.631 INFO:teuthology.orchestra.run.smithi071.stdout:532 read 0x7472617 thru 0x7482493 (0xfe7d bytes) 2024-09-14T05:29:34.631 INFO:teuthology.orchestra.run.smithi071.stdout:533 write 0x5ce6c35 thru 0x5ceec37 (0x8003 bytes) 2024-09-14T05:29:34.638 INFO:teuthology.orchestra.run.smithi071.stdout:534 clone 7 order 21 su 524288 sc 11 2024-09-14T05:29:34.672 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:34.672+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:34.681 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:29:34.681 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:29:34.700 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:29:34.700 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:29:34.716 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:29:34.716 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:29:34.732 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:29:34.732 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:29:34.737 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:34.737 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:34.737 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:29:34.737 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:34.746 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:29:34.749 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:29:34.749 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:29:34.766 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:29:34.766 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:29:34.773 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:34.773+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:34.783 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:29:34.783 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:29:34.799 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:29:34.800 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:29:34.858 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:34.858 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:34.858 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:34.858 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:34.858 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:34.867 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:29:34.874 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:34.873+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:34.970 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4235.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-14T05:29:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:29:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 21.129282609000001, 2024-09-14T05:29:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12887758399999999, 2024-09-14T05:29:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:29:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4235", 2024-09-14T05:29:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/3211559243", 2024-09-14T05:29:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 2 2024-09-14T05:29:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:29:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:29:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839833+0000", 2024-09-14T05:29:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:29:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839838+0000", 2024-09-14T05:29:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2440000000000001e-06 2024-09-14T05:29:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839848+0000", 2024-09-14T05:29:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0030000000000001e-05 2024-09-14T05:29:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839854+0000", 2024-09-14T05:29:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3750000000000002e-06 2024-09-14T05:29:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839908+0000", 2024-09-14T05:29:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3965999999999998e-05 2024-09-14T05:29:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:29:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839914+0000", 2024-09-14T05:29:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6139999999999997e-06 2024-09-14T05:29:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968558+0000", 2024-09-14T05:29:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12864361199999999 2024-09-14T05:29:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968710+0000", 2024-09-14T05:29:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015177300000000001 2024-09-14T05:29:34.976 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:34.976 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:34.976 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:34.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.976 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1088 3.11 3:8fa4ce7b:::rbd_data.112dd25fb4c.0000000000000059:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:34.976 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.146612+0000", 2024-09-14T05:29:34.976 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.8225030140000005, 2024-09-14T05:29:34.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0076673100000000001, 2024-09-14T05:29:34.976 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:34.976 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:34.976 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:34.976 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:34.976 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:34.977 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:34.977 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1088 2024-09-14T05:29:34.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.977 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:34.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:34.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.146612+0000", 2024-09-14T05:29:34.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:34.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:34.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.146612+0000", 2024-09-14T05:29:34.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:34.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:34.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.146612+0000", 2024-09-14T05:29:34.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0299999999999999e-07 2024-09-14T05:29:34.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:34.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.146615+0000", 2024-09-14T05:29:34.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.118e-06 2024-09-14T05:29:34.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:34.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.146620+0000", 2024-09-14T05:29:34.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6789999999999996e-06 2024-09-14T05:29:34.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:34.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.146625+0000", 2024-09-14T05:29:34.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1390000000000003e-06 2024-09-14T05:29:34.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:34.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.146950+0000", 2024-09-14T05:29:34.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000325349 2024-09-14T05:29:34.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:34.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.147046+0000", 2024-09-14T05:29:34.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5536000000000001e-05 2024-09-14T05:29:34.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:34.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.154279+0000", 2024-09-14T05:29:34.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0072329860000000003 2024-09-14T05:29:34.980 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:34.981 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:34.981 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:34.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.981 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1091 3.11 3:8904a053:::rbd_data.112dd25fb4c.000000000000005c:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:34.981 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.148300+0000", 2024-09-14T05:29:34.981 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.8208148619999998, 2024-09-14T05:29:34.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.008774347, 2024-09-14T05:29:34.981 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:34.981 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:34.981 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:34.981 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:34.981 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:34.981 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:34.982 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1091 2024-09-14T05:29:34.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.982 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:34.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:34.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.148300+0000", 2024-09-14T05:29:34.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:34.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:34.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.148300+0000", 2024-09-14T05:29:34.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:34.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:34.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.148301+0000", 2024-09-14T05:29:34.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2399999999999997e-07 2024-09-14T05:29:34.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:34.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.148304+0000", 2024-09-14T05:29:34.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.597e-06 2024-09-14T05:29:34.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:34.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.148313+0000", 2024-09-14T05:29:34.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4579999999999993e-06 2024-09-14T05:29:34.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:34.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.148319+0000", 2024-09-14T05:29:34.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2110000000000003e-06 2024-09-14T05:29:34.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:34.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.148425+0000", 2024-09-14T05:29:34.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000105963 2024-09-14T05:29:34.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:34.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.148505+0000", 2024-09-14T05:29:34.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0552999999999999e-05 2024-09-14T05:29:34.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:34.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.157074+0000", 2024-09-14T05:29:34.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0085687409999999995 2024-09-14T05:29:34.985 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:34.985 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:34.985 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:34.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.986 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1099 3.11 3:8cc65777:::rbd_data.112dd25fb4c.0000000000000058:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:34.986 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.149585+0000", 2024-09-14T05:29:34.986 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.819529191, 2024-09-14T05:29:34.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0076382430000000003, 2024-09-14T05:29:34.986 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:34.986 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:34.986 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:34.986 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:34.986 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:34.986 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:34.986 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1099 2024-09-14T05:29:34.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.986 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:34.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:34.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.149585+0000", 2024-09-14T05:29:34.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:34.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:34.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.149585+0000", 2024-09-14T05:29:34.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:34.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:34.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.149586+0000", 2024-09-14T05:29:34.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7599999999999997e-07 2024-09-14T05:29:34.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:34.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.149590+0000", 2024-09-14T05:29:34.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5939999999999998e-06 2024-09-14T05:29:34.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:34.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.149598+0000", 2024-09-14T05:29:34.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8490000000000006e-06 2024-09-14T05:29:34.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:34.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.149603+0000", 2024-09-14T05:29:34.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7010000000000001e-06 2024-09-14T05:29:34.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:34.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.149652+0000", 2024-09-14T05:29:34.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9157999999999999e-05 2024-09-14T05:29:34.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:34.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.149723+0000", 2024-09-14T05:29:34.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0553e-05 2024-09-14T05:29:34.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:34.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.149924+0000", 2024-09-14T05:29:34.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020064200000000001 2024-09-14T05:29:34.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:34.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.154290+0000", 2024-09-14T05:29:34.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0043660690000000002 2024-09-14T05:29:34.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:34.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.157120+0000", 2024-09-14T05:29:34.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0028306859999999998 2024-09-14T05:29:34.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:34.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.157180+0000", 2024-09-14T05:29:34.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9936999999999999e-05 2024-09-14T05:29:34.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:34.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.157224+0000", 2024-09-14T05:29:34.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3377999999999997e-05 2024-09-14T05:29:34.992 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:34.992 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:34.992 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:34.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.992 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1154 3.9 3:919c95c5:::rbd_data.112dd25fb4c.0000000000000072:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:34.993 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.174308+0000", 2024-09-14T05:29:34.993 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.7948063679999997, 2024-09-14T05:29:34.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010732152999999999, 2024-09-14T05:29:34.993 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:34.993 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:34.993 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:34.993 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:34.993 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:34.993 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:34.993 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1154 2024-09-14T05:29:34.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.993 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:34.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:34.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.174308+0000", 2024-09-14T05:29:34.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:34.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:34.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.174308+0000", 2024-09-14T05:29:34.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:34.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:34.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.174309+0000", 2024-09-14T05:29:34.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8599999999999997e-07 2024-09-14T05:29:34.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:34.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.174313+0000", 2024-09-14T05:29:34.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2250000000000002e-06 2024-09-14T05:29:34.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:34.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.174326+0000", 2024-09-14T05:29:34.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2758000000000001e-05 2024-09-14T05:29:34.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:34.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.174334+0000", 2024-09-14T05:29:34.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0760000000000003e-06 2024-09-14T05:29:34.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:34.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.174383+0000", 2024-09-14T05:29:34.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8918999999999997e-05 2024-09-14T05:29:34.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:34.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.174459+0000", 2024-09-14T05:29:34.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6352000000000004e-05 2024-09-14T05:29:34.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-14T05:29:34.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.174653+0000", 2024-09-14T05:29:34.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019318499999999999 2024-09-14T05:29:34.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:34.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.178124+0000", 2024-09-14T05:29:34.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0034713080000000002 2024-09-14T05:29:34.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:34.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.184948+0000", 2024-09-14T05:29:34.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0068236659999999999 2024-09-14T05:29:34.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:34.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.184994+0000", 2024-09-14T05:29:34.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6001000000000002e-05 2024-09-14T05:29:34.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:34.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.185040+0000", 2024-09-14T05:29:34.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6777000000000003e-05 2024-09-14T05:29:34.999 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:34.999 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:34.999 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:34.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:34.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:34.999 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1164 3.14 3:29d40917:::rbd_data.112dd25fb4c.0000000000000071:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:34.999 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.177293+0000", 2024-09-14T05:29:34.999 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.7918219369999999, 2024-09-14T05:29:34.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.011789762000000001, 2024-09-14T05:29:34.999 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:34.999 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:34.999 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:35.000 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:35.000 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:35.000 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.000 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1164 2024-09-14T05:29:35.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.000 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:35.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:35.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177293+0000", 2024-09-14T05:29:35.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:35.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177293+0000", 2024-09-14T05:29:35.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:35.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177293+0000", 2024-09-14T05:29:35.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.09e-07 2024-09-14T05:29:35.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:35.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177297+0000", 2024-09-14T05:29:35.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7589999999999999e-06 2024-09-14T05:29:35.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:35.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177307+0000", 2024-09-14T05:29:35.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8539999999999992e-06 2024-09-14T05:29:35.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177312+0000", 2024-09-14T05:29:35.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9269999999999996e-06 2024-09-14T05:29:35.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:35.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177359+0000", 2024-09-14T05:29:35.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6796999999999999e-05 2024-09-14T05:29:35.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:35.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177437+0000", 2024-09-14T05:29:35.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7891000000000004e-05 2024-09-14T05:29:35.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:35.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177575+0000", 2024-09-14T05:29:35.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000138176 2024-09-14T05:29:35.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:35.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.185301+0000", 2024-09-14T05:29:35.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0077263139999999998 2024-09-14T05:29:35.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:35.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.188985+0000", 2024-09-14T05:29:35.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0036843039999999998 2024-09-14T05:29:35.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:35.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.189039+0000", 2024-09-14T05:29:35.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3919000000000003e-05 2024-09-14T05:29:35.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:35.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.189082+0000", 2024-09-14T05:29:35.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3012000000000001e-05 2024-09-14T05:29:35.005 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.005 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:35.005 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.005 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1201 3.9 3:93781545:::rbd_data.112dd25fb4c.000000000000007a:head [call rbd.sparse_copyup in=14115b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:35.005 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.197062+0000", 2024-09-14T05:29:35.005 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.7720528639999999, 2024-09-14T05:29:35.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.013609586, 2024-09-14T05:29:35.006 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:35.006 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:35.006 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:35.006 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:35.006 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:35.006 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.006 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1201 2024-09-14T05:29:35.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.006 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:35.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:35.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197062+0000", 2024-09-14T05:29:35.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:35.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197062+0000", 2024-09-14T05:29:35.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:35.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197062+0000", 2024-09-14T05:29:35.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9400000000000005e-07 2024-09-14T05:29:35.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:35.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197079+0000", 2024-09-14T05:29:35.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6118999999999999e-05 2024-09-14T05:29:35.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:35.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197087+0000", 2024-09-14T05:29:35.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3170000000000002e-06 2024-09-14T05:29:35.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197093+0000", 2024-09-14T05:29:35.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.246e-06 2024-09-14T05:29:35.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:35.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197136+0000", 2024-09-14T05:29:35.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3362000000000003e-05 2024-09-14T05:29:35.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:35.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197185+0000", 2024-09-14T05:29:35.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8374000000000001e-05 2024-09-14T05:29:35.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-14T05:29:35.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197305+0000", 2024-09-14T05:29:35.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119782 2024-09-14T05:29:35.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:35.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202953+0000", 2024-09-14T05:29:35.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056481090000000001 2024-09-14T05:29:35.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:35.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.210445+0000", 2024-09-14T05:29:35.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0074919499999999998 2024-09-14T05:29:35.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:35.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.210521+0000", 2024-09-14T05:29:35.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6520000000000006e-05 2024-09-14T05:29:35.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:35.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.210671+0000", 2024-09-14T05:29:35.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000150113 2024-09-14T05:29:35.011 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.011 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:35.011 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.012 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1202 3.9 3:93781545:::rbd_data.112dd25fb4c.000000000000007a:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:35.012 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.197108+0000", 2024-09-14T05:29:35.012 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.7720065800000002, 2024-09-14T05:29:35.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017664335999999999, 2024-09-14T05:29:35.012 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:35.012 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:35.012 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:35.012 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:35.012 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:35.012 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.012 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1202 2024-09-14T05:29:35.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.012 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:35.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:35.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197108+0000", 2024-09-14T05:29:35.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:35.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197108+0000", 2024-09-14T05:29:35.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:35.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197109+0000", 2024-09-14T05:29:35.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.13e-07 2024-09-14T05:29:35.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:35.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197111+0000", 2024-09-14T05:29:35.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.711e-06 2024-09-14T05:29:35.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:35.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197120+0000", 2024-09-14T05:29:35.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7409999999999998e-06 2024-09-14T05:29:35.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197125+0000", 2024-09-14T05:29:35.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4990000000000002e-06 2024-09-14T05:29:35.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:35.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.197993+0000", 2024-09-14T05:29:35.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000867307 2024-09-14T05:29:35.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:35.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.198063+0000", 2024-09-14T05:29:35.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0652000000000001e-05 2024-09-14T05:29:35.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-14T05:29:35.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.198256+0000", 2024-09-14T05:29:35.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019209399999999999 2024-09-14T05:29:35.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:35.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203008+0000", 2024-09-14T05:29:35.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0047523670000000004 2024-09-14T05:29:35.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:35.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.214661+0000", 2024-09-14T05:29:35.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.011653035000000001 2024-09-14T05:29:35.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:35.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.214708+0000", 2024-09-14T05:29:35.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6737000000000003e-05 2024-09-14T05:29:35.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:35.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.214772+0000", 2024-09-14T05:29:35.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4679999999999997e-05 2024-09-14T05:29:35.017 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.017 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:35.017 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.017 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1208 3.d 3:b1b1b80b:::rbd_data.112dd25fb4c.000000000000008a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:35.017 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.200746+0000", 2024-09-14T05:29:35.018 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.7683686029999999, 2024-09-14T05:29:35.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0098531460000000001, 2024-09-14T05:29:35.018 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:35.018 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:35.018 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:35.018 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:35.018 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:35.018 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.018 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1208 2024-09-14T05:29:35.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.018 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:35.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:35.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200746+0000", 2024-09-14T05:29:35.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:35.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200746+0000", 2024-09-14T05:29:35.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:35.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200747+0000", 2024-09-14T05:29:35.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7400000000000002e-07 2024-09-14T05:29:35.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:35.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200757+0000", 2024-09-14T05:29:35.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9299999999999998e-06 2024-09-14T05:29:35.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:35.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200768+0000", 2024-09-14T05:29:35.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0869999999999999e-05 2024-09-14T05:29:35.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200772+0000", 2024-09-14T05:29:35.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9400000000000001e-06 2024-09-14T05:29:35.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:35.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200816+0000", 2024-09-14T05:29:35.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3730999999999999e-05 2024-09-14T05:29:35.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:35.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.200902+0000", 2024-09-14T05:29:35.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5317999999999999e-05 2024-09-14T05:29:35.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:35.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.210599+0000", 2024-09-14T05:29:35.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0096975829999999992 2024-09-14T05:29:35.022 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.022 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:35.022 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.022 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1214 3.13 3:ce6c3324:::rbd_data.112dd25fb4c.0000000000000085:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:35.022 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.202983+0000", 2024-09-14T05:29:35.022 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.7661317199999997, 2024-09-14T05:29:35.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.011801188000000001, 2024-09-14T05:29:35.023 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:35.023 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:35.023 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:35.023 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:35.023 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:35.023 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.023 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1214 2024-09-14T05:29:35.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.023 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:35.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:35.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202983+0000", 2024-09-14T05:29:35.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:35.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202983+0000", 2024-09-14T05:29:35.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:35.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202984+0000", 2024-09-14T05:29:35.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0669999999999999e-06 2024-09-14T05:29:35.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:35.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202989+0000", 2024-09-14T05:29:35.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1200000000000001e-06 2024-09-14T05:29:35.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:35.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203001+0000", 2024-09-14T05:29:35.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2191999999999999e-05 2024-09-14T05:29:35.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203009+0000", 2024-09-14T05:29:35.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.259e-06 2024-09-14T05:29:35.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:35.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203068+0000", 2024-09-14T05:29:35.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8675000000000003e-05 2024-09-14T05:29:35.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:35.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203149+0000", 2024-09-14T05:29:35.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0358999999999994e-05 2024-09-14T05:29:35.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-14T05:29:35.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203306+0000", 2024-09-14T05:29:35.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000157529 2024-09-14T05:29:35.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:35.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.210920+0000", 2024-09-14T05:29:35.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0076139249999999997 2024-09-14T05:29:35.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:35.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.214697+0000", 2024-09-14T05:29:35.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0037769430000000001 2024-09-14T05:29:35.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:35.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.214747+0000", 2024-09-14T05:29:35.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0136000000000002e-05 2024-09-14T05:29:35.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:35.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.214784+0000", 2024-09-14T05:29:35.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6983000000000001e-05 2024-09-14T05:29:35.028 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.028 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:35.028 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.028 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1224 3.d 3:b1b1b80b:::rbd_data.112dd25fb4c.000000000000008a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:35.028 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.211081+0000", 2024-09-14T05:29:35.029 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.7580332429999999, 2024-09-14T05:29:35.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0081961900000000008, 2024-09-14T05:29:35.029 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:35.029 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:35.029 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:35.029 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:35.029 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:35.029 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.029 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1224 2024-09-14T05:29:35.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.029 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:35.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:35.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211081+0000", 2024-09-14T05:29:35.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:35.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211081+0000", 2024-09-14T05:29:35.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:35.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211083+0000", 2024-09-14T05:29:35.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2130000000000001e-06 2024-09-14T05:29:35.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:35.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211090+0000", 2024-09-14T05:29:35.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2559999999999996e-06 2024-09-14T05:29:35.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:35.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211100+0000", 2024-09-14T05:29:35.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7329999999999998e-06 2024-09-14T05:29:35.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211106+0000", 2024-09-14T05:29:35.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.579e-06 2024-09-14T05:29:35.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:35.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211157+0000", 2024-09-14T05:29:35.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0426999999999997e-05 2024-09-14T05:29:35.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:35.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211227+0000", 2024-09-14T05:29:35.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0238000000000001e-05 2024-09-14T05:29:35.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-14T05:29:35.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211358+0000", 2024-09-14T05:29:35.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000131407 2024-09-14T05:29:35.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:35.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.214743+0000", 2024-09-14T05:29:35.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0033846739999999998 2024-09-14T05:29:35.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:35.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.219184+0000", 2024-09-14T05:29:35.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0044408640000000001 2024-09-14T05:29:35.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:35.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.219237+0000", 2024-09-14T05:29:35.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2834000000000003e-05 2024-09-14T05:29:35.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:35.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.219278+0000", 2024-09-14T05:29:35.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0964999999999998e-05 2024-09-14T05:29:35.035 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.035 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:35.035 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.035 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "pg_update_log_missing(3.17 epoch 30/24 rep_tid 108 entries 30'12 (0'0) error 3:e9c60235:::rbd_data.112dd25fb4c.00000000000000a1:head by client.4397.0:1262 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'10)", 2024-09-14T05:29:35.035 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.227798+0000", 2024-09-14T05:29:35.035 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.741317016, 2024-09-14T05:29:35.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0076164079999999999, 2024-09-14T05:29:35.035 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:29:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.227798+0000", 2024-09-14T05:29:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.227798+0000", 2024-09-14T05:29:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:35.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.227798+0000", 2024-09-14T05:29:35.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6700000000000003e-07 2024-09-14T05:29:35.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:35.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.227804+0000", 2024-09-14T05:29:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2570000000000004e-06 2024-09-14T05:29:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.227819+0000", 2024-09-14T05:29:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4738e-05 2024-09-14T05:29:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.227828+0000", 2024-09-14T05:29:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9129999999999992e-06 2024-09-14T05:29:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:35.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.227890+0000", 2024-09-14T05:29:35.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1939000000000004e-05 2024-09-14T05:29:35.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:35.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.235414+0000", 2024-09-14T05:29:35.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.007523994 2024-09-14T05:29:35.039 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.039 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:35.039 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.040 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "pg_update_log_missing(3.17 epoch 30/24 rep_tid 110 entries 30'14 (0'0) error 3:eebf0247:::rbd_data.112dd25fb4c.00000000000000a3:head by client.4397.0:1266 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'10)", 2024-09-14T05:29:35.040 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.231661+0000", 2024-09-14T05:29:35.040 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.7374537139999999, 2024-09-14T05:29:35.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0078005110000000004, 2024-09-14T05:29:35.040 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:35.040 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:35.040 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:29:35.040 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:35.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:35.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231661+0000", 2024-09-14T05:29:35.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:35.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231661+0000", 2024-09-14T05:29:35.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:35.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231662+0000", 2024-09-14T05:29:35.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2900000000000003e-07 2024-09-14T05:29:35.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:35.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231669+0000", 2024-09-14T05:29:35.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7689999999999996e-06 2024-09-14T05:29:35.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:35.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231690+0000", 2024-09-14T05:29:35.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0509e-05 2024-09-14T05:29:35.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231704+0000", 2024-09-14T05:29:35.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4569000000000001e-05 2024-09-14T05:29:35.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:35.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.231781+0000", 2024-09-14T05:29:35.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6983999999999993e-05 2024-09-14T05:29:35.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:35.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.239461+0000", 2024-09-14T05:29:35.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.007679951 2024-09-14T05:29:35.043 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.043 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:35.043 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.044 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1598 3.14 3:2ae9c1cd:::rbd_data.112dd25fb4c.0000000000000143:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:35.044 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.364021+0000", 2024-09-14T05:29:35.044 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.6050936790000003, 2024-09-14T05:29:35.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0080707320000000006, 2024-09-14T05:29:35.044 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:35.044 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:35.044 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:35.044 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:35.044 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:35.044 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.044 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1598 2024-09-14T05:29:35.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.045 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:35.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.045 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:35.045 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364021+0000", 2024-09-14T05:29:35.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.045 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:35.045 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364021+0000", 2024-09-14T05:29:35.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.045 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:35.045 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364022+0000", 2024-09-14T05:29:35.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9000000000000005e-07 2024-09-14T05:29:35.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:35.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364035+0000", 2024-09-14T05:29:35.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3302000000000001e-05 2024-09-14T05:29:35.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:35.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364049+0000", 2024-09-14T05:29:35.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3329999999999999e-05 2024-09-14T05:29:35.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364056+0000", 2024-09-14T05:29:35.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5050000000000001e-06 2024-09-14T05:29:35.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:35.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364124+0000", 2024-09-14T05:29:35.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7739999999999999e-05 2024-09-14T05:29:35.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:35.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364247+0000", 2024-09-14T05:29:35.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012286900000000001 2024-09-14T05:29:35.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.048 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:35.048 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.372092+0000", 2024-09-14T05:29:35.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0078449959999999999 2024-09-14T05:29:35.048 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.048 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:35.048 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.048 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1599 3.9 3:977a0fb9:::rbd_data.112dd25fb4c.000000000000013f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:35.048 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.364092+0000", 2024-09-14T05:29:35.048 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.6050227889999995, 2024-09-14T05:29:35.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.008035805, 2024-09-14T05:29:35.048 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:35.049 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:35.049 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:35.049 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:35.049 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:35.049 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.049 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1599 2024-09-14T05:29:35.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.049 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:35.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:35.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364092+0000", 2024-09-14T05:29:35.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:35.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364092+0000", 2024-09-14T05:29:35.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:35.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364092+0000", 2024-09-14T05:29:35.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.68e-07 2024-09-14T05:29:35.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:35.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364096+0000", 2024-09-14T05:29:35.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8029999999999998e-06 2024-09-14T05:29:35.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:35.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364105+0000", 2024-09-14T05:29:35.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4069999999999993e-06 2024-09-14T05:29:35.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364110+0000", 2024-09-14T05:29:35.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4999999999999999e-06 2024-09-14T05:29:35.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:35.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364169+0000", 2024-09-14T05:29:35.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8517999999999998e-05 2024-09-14T05:29:35.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:35.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364240+0000", 2024-09-14T05:29:35.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0947999999999999e-05 2024-09-14T05:29:35.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-14T05:29:35.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364456+0000", 2024-09-14T05:29:35.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021666599999999999 2024-09-14T05:29:35.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.053 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:35.053 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.365279+0000", 2024-09-14T05:29:35.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000822287 2024-09-14T05:29:35.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.053 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:35.053 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.372030+0000", 2024-09-14T05:29:35.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0067517740000000003 2024-09-14T05:29:35.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.053 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:35.053 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.372089+0000", 2024-09-14T05:29:35.054 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8218999999999999e-05 2024-09-14T05:29:35.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.054 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:35.054 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.372128+0000", 2024-09-14T05:29:35.054 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9014999999999998e-05 2024-09-14T05:29:35.054 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.054 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:35.054 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.054 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1603 3.d 3:b4928c19:::rbd_data.112dd25fb4c.000000000000013d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:35.054 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.364654+0000", 2024-09-14T05:29:35.055 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.6044605589999996, 2024-09-14T05:29:35.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.007721197, 2024-09-14T05:29:35.055 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:35.055 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:35.055 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:35.055 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:35.055 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:35.055 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.055 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1603 2024-09-14T05:29:35.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.055 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:35.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:35.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364654+0000", 2024-09-14T05:29:35.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:35.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364654+0000", 2024-09-14T05:29:35.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:35.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364655+0000", 2024-09-14T05:29:35.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4099999999999997e-07 2024-09-14T05:29:35.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:35.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364659+0000", 2024-09-14T05:29:35.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1640000000000003e-06 2024-09-14T05:29:35.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:35.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364668+0000", 2024-09-14T05:29:35.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1819999999999995e-06 2024-09-14T05:29:35.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364675+0000", 2024-09-14T05:29:35.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4180000000000002e-06 2024-09-14T05:29:35.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.058 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.058 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:35.058 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364721+0000", 2024-09-14T05:29:35.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6706000000000003e-05 2024-09-14T05:29:35.058 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.058 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.058 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:35.058 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.364793+0000", 2024-09-14T05:29:35.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1619000000000007e-05 2024-09-14T05:29:35.058 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.058 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.058 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-14T05:29:35.058 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.365006+0000", 2024-09-14T05:29:35.059 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000212945 2024-09-14T05:29:35.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.059 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:35.059 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.372010+0000", 2024-09-14T05:29:35.059 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0070042739999999996 2024-09-14T05:29:35.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.059 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:35.059 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.372240+0000", 2024-09-14T05:29:35.059 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023001 2024-09-14T05:29:35.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.059 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:35.060 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.372299+0000", 2024-09-14T05:29:35.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8436000000000001e-05 2024-09-14T05:29:35.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:35.060 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.372375+0000", 2024-09-14T05:29:35.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6601999999999996e-05 2024-09-14T05:29:35.060 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.060 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:35.060 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.060 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1819 3.14 3:2f3733d0:::rbd_data.112dd25fb4c.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:35.061 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.456147+0000", 2024-09-14T05:29:35.061 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.5129675030000005, 2024-09-14T05:29:35.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0080915729999999995, 2024-09-14T05:29:35.061 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:35.061 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:35.061 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:35.061 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:35.061 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:35.061 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.061 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1819 2024-09-14T05:29:35.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.061 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:35.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:35.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.456147+0000", 2024-09-14T05:29:35.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:35.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.456147+0000", 2024-09-14T05:29:35.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:35.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.456149+0000", 2024-09-14T05:29:35.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.266e-06 2024-09-14T05:29:35.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.063 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:35.063 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.456157+0000", 2024-09-14T05:29:35.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8339999999999999e-06 2024-09-14T05:29:35.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.063 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:35.063 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.456177+0000", 2024-09-14T05:29:35.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9834999999999999e-05 2024-09-14T05:29:35.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.063 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.063 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.456186+0000", 2024-09-14T05:29:35.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.465e-06 2024-09-14T05:29:35.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:35.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.456246+0000", 2024-09-14T05:29:35.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0004999999999999e-05 2024-09-14T05:29:35.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:35.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.456384+0000", 2024-09-14T05:29:35.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000137236 2024-09-14T05:29:35.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:35.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.456581+0000", 2024-09-14T05:29:35.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019773600000000001 2024-09-14T05:29:35.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:35.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.457656+0000", 2024-09-14T05:29:35.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001074565 2024-09-14T05:29:35.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:35.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.464156+0000", 2024-09-14T05:29:35.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.006500273 2024-09-14T05:29:35.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:35.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.464202+0000", 2024-09-14T05:29:35.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5939999999999997e-05 2024-09-14T05:29:35.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:35.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.464239+0000", 2024-09-14T05:29:35.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6418000000000003e-05 2024-09-14T05:29:35.068 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.068 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:35.068 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.068 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:4328 3.11 3:8bc7a24c:::rbd_data.112d3d611e54.0000000000000023:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.069 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:31.725351+0000", 2024-09-14T05:29:35.069 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.2437635399999998, 2024-09-14T05:29:35.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.014564745, 2024-09-14T05:29:35.069 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:35.069 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:35.069 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:35.069 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:35.069 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:35.069 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.069 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 4328 2024-09-14T05:29:35.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.069 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:35.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:35.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.725351+0000", 2024-09-14T05:29:35.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:35.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.725351+0000", 2024-09-14T05:29:35.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:35.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.725352+0000", 2024-09-14T05:29:35.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4199999999999995e-07 2024-09-14T05:29:35.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:35.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.725356+0000", 2024-09-14T05:29:35.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.907e-06 2024-09-14T05:29:35.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:35.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.725365+0000", 2024-09-14T05:29:35.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.611e-06 2024-09-14T05:29:35.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.725370+0000", 2024-09-14T05:29:35.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9130000000000002e-06 2024-09-14T05:29:35.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:35.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.725418+0000", 2024-09-14T05:29:35.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7435999999999998e-05 2024-09-14T05:29:35.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:35.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.725494+0000", 2024-09-14T05:29:35.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6302000000000002e-05 2024-09-14T05:29:35.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.073 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:35.073 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.728024+0000", 2024-09-14T05:29:35.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0025301109999999998 2024-09-14T05:29:35.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.073 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:35.073 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.739815+0000", 2024-09-14T05:29:35.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.011791127 2024-09-14T05:29:35.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.073 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:35.074 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.739847+0000", 2024-09-14T05:29:35.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1498000000000002e-05 2024-09-14T05:29:35.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.074 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.074 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:35.074 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.739887+0000", 2024-09-14T05:29:35.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0658999999999998e-05 2024-09-14T05:29:35.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.074 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.074 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:35.074 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.739916+0000", 2024-09-14T05:29:35.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8439e-05 2024-09-14T05:29:35.074 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.074 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:35.075 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.075 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:4416 3.a 3:5647e17c:::rbd_data.112d3d611e54.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.075 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:31.821682+0000", 2024-09-14T05:29:35.075 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.1474323740000001, 2024-09-14T05:29:35.075 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010661550000000001, 2024-09-14T05:29:35.075 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:35.075 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:35.075 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:35.075 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:35.075 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:35.076 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.076 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 4416 2024-09-14T05:29:35.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.076 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:35.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.076 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:35.076 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.821682+0000", 2024-09-14T05:29:35.076 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.076 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:35.076 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.821682+0000", 2024-09-14T05:29:35.076 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.077 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.077 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:35.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.821684+0000", 2024-09-14T05:29:35.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4309999999999999e-06 2024-09-14T05:29:35.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.077 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.077 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:35.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.821692+0000", 2024-09-14T05:29:35.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.541e-06 2024-09-14T05:29:35.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.077 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.077 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:35.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.821705+0000", 2024-09-14T05:29:35.078 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2627e-05 2024-09-14T05:29:35.078 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.078 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.078 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.078 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.821710+0000", 2024-09-14T05:29:35.078 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6960000000000002e-06 2024-09-14T05:29:35.078 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.078 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.078 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:35.078 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.821759+0000", 2024-09-14T05:29:35.078 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8226999999999998e-05 2024-09-14T05:29:35.078 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.078 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.079 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:35.079 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.821850+0000", 2024-09-14T05:29:35.079 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1666e-05 2024-09-14T05:29:35.079 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.079 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.079 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-14T05:29:35.079 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.822838+0000", 2024-09-14T05:29:35.079 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00098742699999999992 2024-09-14T05:29:35.079 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.079 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.079 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:35.079 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.829108+0000", 2024-09-14T05:29:35.079 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0062706749999999999 2024-09-14T05:29:35.080 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.080 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.080 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:35.080 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.832272+0000", 2024-09-14T05:29:35.080 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0031637119999999999 2024-09-14T05:29:35.080 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.080 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.080 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:35.080 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.832309+0000", 2024-09-14T05:29:35.080 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7117999999999999e-05 2024-09-14T05:29:35.080 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.080 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.080 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:35.080 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.832344+0000", 2024-09-14T05:29:35.081 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4430000000000001e-05 2024-09-14T05:29:35.081 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.081 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:35.081 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.081 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.081 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.081 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:4577 3.9 3:96d1aa93:::rbd_data.112d3d611e54.0000000000000021:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.081 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:32.016233+0000", 2024-09-14T05:29:35.081 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 2.9528818609999998, 2024-09-14T05:29:35.081 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.011185417, 2024-09-14T05:29:35.081 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:35.081 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:35.081 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:35.082 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:35.082 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:35.082 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.082 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 4577 2024-09-14T05:29:35.082 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.082 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:35.082 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.082 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:35.082 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.016233+0000", 2024-09-14T05:29:35.082 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.082 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.082 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.082 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:35.083 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.016233+0000", 2024-09-14T05:29:35.083 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:35.083 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.083 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.083 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:35.083 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.016234+0000", 2024-09-14T05:29:35.083 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.319e-06 2024-09-14T05:29:35.083 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.083 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.083 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:35.083 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.016239+0000", 2024-09-14T05:29:35.083 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8489999999999997e-06 2024-09-14T05:29:35.083 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.084 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.084 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:35.084 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.016250+0000", 2024-09-14T05:29:35.084 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0736e-05 2024-09-14T05:29:35.084 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.084 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.084 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.084 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.016255+0000", 2024-09-14T05:29:35.084 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1749999999999996e-06 2024-09-14T05:29:35.084 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.084 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.084 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:35.084 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.016295+0000", 2024-09-14T05:29:35.085 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0463000000000002e-05 2024-09-14T05:29:35.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.085 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.085 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:35.085 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.016350+0000", 2024-09-14T05:29:35.085 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4761e-05 2024-09-14T05:29:35.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.085 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.085 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-14T05:29:35.085 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.017961+0000", 2024-09-14T05:29:35.085 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001611101 2024-09-14T05:29:35.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.085 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.086 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:35.086 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.027303+0000", 2024-09-14T05:29:35.086 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0093416860000000001 2024-09-14T05:29:35.086 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.086 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.086 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:35.086 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.027319+0000", 2024-09-14T05:29:35.086 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6676000000000001e-05 2024-09-14T05:29:35.086 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.086 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.086 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:35.086 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.027372+0000", 2024-09-14T05:29:35.086 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2988999999999997e-05 2024-09-14T05:29:35.086 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:35.087 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:35.087 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:35.087 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.027418+0000", 2024-09-14T05:29:35.087 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5661999999999998e-05 2024-09-14T05:29:35.087 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.087 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:35.087 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.087 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:35.087 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:35.087 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:35.089 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:29:35.090 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-14T05:29:35.090 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:35.088+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:35.126 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone6 from 0xe885de1 (overlap 0x221290e) to 0x14718a2 2024-09-14T05:29:35.189 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:35.189+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:35.205 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:35.205 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:35.205 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:29:35.205 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:35.211 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-14T05:29:35.214 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:29:35.291 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:35.290+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:35.319 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:35.319 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:35.319 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:29:35.319 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:29:35.319 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:35.327 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:29:35.391 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:35.391+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:35.399 INFO:teuthology.orchestra.run.smithi071.stdout:535 read 0x8e9916a thru 0x8ea8bd3 (0xfa6a bytes) 2024-09-14T05:29:35.399 INFO:teuthology.orchestra.run.smithi071.stdout:537 read 0x123e3cd thru 0x124e36d (0xffa1 bytes) 2024-09-14T05:29:35.407 INFO:teuthology.orchestra.run.smithi071.stdout:540 trunc from 0xe885de1 to 0x7e13deb 2024-09-14T05:29:35.428 INFO:teuthology.orchestra.run.smithi071.stdout:541 write 0x39899ef thru 0x3989e40 (0x452 bytes) 2024-09-14T05:29:35.432 INFO:teuthology.orchestra.run.smithi071.stdout:543 read 0x703a88d thru 0x704836d (0xdae1 bytes) 2024-09-14T05:29:35.432 INFO:teuthology.orchestra.run.smithi071.stdout:544 read 0x45c28f9 thru 0x45cdc82 (0xb38a bytes) 2024-09-14T05:29:35.433 INFO:teuthology.orchestra.run.smithi071.stdout:545 punch from 0x59ff722 to 0x5a0eac0, (0xf39e bytes) 2024-09-14T05:29:35.433 INFO:teuthology.orchestra.run.smithi071.stdout:546 write 0x5150c20 thru 0x51589ad (0x7d8e bytes) 2024-09-14T05:29:35.438 INFO:teuthology.orchestra.run.smithi071.stdout:547 punch from 0x4601afe to 0x460a666, (0x8b68 bytes) 2024-09-14T05:29:35.438 INFO:teuthology.orchestra.run.smithi071.stdout:549 read 0x57c628e thru 0x57d34e6 (0xd259 bytes) 2024-09-14T05:29:35.438 INFO:teuthology.orchestra.run.smithi071.stdout:552 read 0x50629f6 thru 0x5064a3a (0x2045 bytes) 2024-09-14T05:29:35.438 INFO:teuthology.orchestra.run.smithi071.stdout:553 write 0x274b62d thru 0x2750584 (0x4f58 bytes) 2024-09-14T05:29:35.441 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:35.441 INFO:teuthology.orchestra.run.smithi071.stdout:555 punch from 0x6a1aabc to 0x6a252a2, (0xa7e6 bytes) 2024-09-14T05:29:35.447 INFO:teuthology.orchestra.run.smithi071.stdout:556 trunc from 0x7e13deb to 0xd0289f2 2024-09-14T05:29:35.453 INFO:teuthology.orchestra.run.smithi071.stdout:557 read 0xa245dd8 thru 0xa24cd6b (0x6f94 bytes) 2024-09-14T05:29:35.453 INFO:teuthology.orchestra.run.smithi071.stdout:558 read 0xb239ab0 thru 0xb242f40 (0x9491 bytes) 2024-09-14T05:29:35.453 INFO:teuthology.orchestra.run.smithi071.stdout:559 read 0x876185c thru 0x8762cbb (0x1460 bytes) 2024-09-14T05:29:35.453 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:35.453 INFO:teuthology.orchestra.run.smithi071.stdout:561 read 0x22a26fc thru 0x22aefac (0xc8b1 bytes) 2024-09-14T05:29:35.461 INFO:teuthology.orchestra.run.smithi071.stdout:564 write 0xd5d875e thru 0xd5d938c (0xc2f bytes) 2024-09-14T05:29:35.465 INFO:teuthology.orchestra.run.smithi071.stdout:565 write 0x97203bd thru 0x972c510 (0xc154 bytes) 2024-09-14T05:29:35.469 INFO:teuthology.orchestra.run.smithi071.stdout:566 read 0xbbc32fc thru 0xbbcbf6c (0x8c71 bytes) 2024-09-14T05:29:35.474 INFO:teuthology.orchestra.run.smithi071.stdout:567 trunc from 0xd5d938d to 0x94972c6 2024-09-14T05:29:35.476 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:35.476 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:29:35.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:29:35.476 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:29:35.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.476 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3523 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6062080~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.476 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:29:35.476 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.9604385459999998, 2024-09-14T05:29:35.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096911030999999995, 2024-09-14T05:29:35.477 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.477 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.477 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.477 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3523 2024-09-14T05:29:35.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.477 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:29:35.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:29:35.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514359+0000", 2024-09-14T05:29:35.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9299999999999999e-07 2024-09-14T05:29:35.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514361+0000", 2024-09-14T05:29:35.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:29:35.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514366+0000", 2024-09-14T05:29:35.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1660000000000002e-06 2024-09-14T05:29:35.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514371+0000", 2024-09-14T05:29:35.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.648e-06 2024-09-14T05:29:35.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.610954+0000", 2024-09-14T05:29:35.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096582170999999994 2024-09-14T05:29:35.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611002+0000", 2024-09-14T05:29:35.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7923999999999997e-05 2024-09-14T05:29:35.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611269+0000", 2024-09-14T05:29:35.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026789499999999998 2024-09-14T05:29:35.482 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.482 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.482 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.482 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3524 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6553600~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.482 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:29:35.482 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.9604193539999999, 2024-09-14T05:29:35.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097263347999999999, 2024-09-14T05:29:35.483 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.483 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.483 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.483 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3524 2024-09-14T05:29:35.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.483 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:29:35.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:29:35.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:29:35.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:29:35.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514380+0000", 2024-09-14T05:29:35.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6500000000000001e-06 2024-09-14T05:29:35.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514386+0000", 2024-09-14T05:29:35.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7360000000000004e-06 2024-09-14T05:29:35.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514391+0000", 2024-09-14T05:29:35.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2830000000000003e-06 2024-09-14T05:29:35.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611295+0000", 2024-09-14T05:29:35.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096904814000000006 2024-09-14T05:29:35.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611343+0000", 2024-09-14T05:29:35.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7506999999999998e-05 2024-09-14T05:29:35.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611641+0000", 2024-09-14T05:29:35.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029798800000000001 2024-09-14T05:29:35.487 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.487 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.487 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.487 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3525 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7045120~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.487 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:29:35.487 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.9603974669999999, 2024-09-14T05:29:35.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097644422999999994, 2024-09-14T05:29:35.487 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.487 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.487 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.488 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3525 2024-09-14T05:29:35.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.488 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:29:35.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:29:35.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514400+0000", 2024-09-14T05:29:35.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.77e-07 2024-09-14T05:29:35.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514402+0000", 2024-09-14T05:29:35.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6530000000000001e-06 2024-09-14T05:29:35.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514411+0000", 2024-09-14T05:29:35.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7429999999999997e-06 2024-09-14T05:29:35.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514417+0000", 2024-09-14T05:29:35.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2249999999999997e-06 2024-09-14T05:29:35.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611668+0000", 2024-09-14T05:29:35.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097250681000000005 2024-09-14T05:29:35.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611716+0000", 2024-09-14T05:29:35.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7336000000000002e-05 2024-09-14T05:29:35.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612044+0000", 2024-09-14T05:29:35.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032840799999999998 2024-09-14T05:29:35.491 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.491 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.492 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.492 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3526 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7536640~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.492 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:29:35.492 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.9603729830000001, 2024-09-14T05:29:35.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097943884999999994, 2024-09-14T05:29:35.492 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.492 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.492 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.492 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.492 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.492 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.492 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3526 2024-09-14T05:29:35.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.493 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:29:35.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:29:35.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:29:35.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0400000000000002e-07 2024-09-14T05:29:35.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514426+0000", 2024-09-14T05:29:35.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:29:35.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514432+0000", 2024-09-14T05:29:35.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.105e-06 2024-09-14T05:29:35.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514438+0000", 2024-09-14T05:29:35.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5149999999999997e-06 2024-09-14T05:29:35.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612072+0000", 2024-09-14T05:29:35.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097633947999999998 2024-09-14T05:29:35.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612120+0000", 2024-09-14T05:29:35.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8458999999999998e-05 2024-09-14T05:29:35.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612368+0000", 2024-09-14T05:29:35.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024761999999999999 2024-09-14T05:29:35.496 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.496 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.496 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.496 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3527 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 8028160~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.496 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:29:35.496 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.96035227, 2024-09-14T05:29:35.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098244955999999994, 2024-09-14T05:29:35.497 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.497 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.497 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.497 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.497 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.497 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.497 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3527 2024-09-14T05:29:35.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.497 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:29:35.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:29:35.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519842+0000", 2024-09-14T05:29:35.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053975230000000004 2024-09-14T05:29:35.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519851+0000", 2024-09-14T05:29:35.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5599999999999994e-06 2024-09-14T05:29:35.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519858+0000", 2024-09-14T05:29:35.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2409999999999998e-06 2024-09-14T05:29:35.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519863+0000", 2024-09-14T05:29:35.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:29:35.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612395+0000", 2024-09-14T05:29:35.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092532552000000004 2024-09-14T05:29:35.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612442+0000", 2024-09-14T05:29:35.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6486000000000002e-05 2024-09-14T05:29:35.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612690+0000", 2024-09-14T05:29:35.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024791599999999999 2024-09-14T05:29:35.501 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.501 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.501 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.501 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3603 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5177344~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.501 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:29:35.501 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.9547998250000003, 2024-09-14T05:29:35.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096796436, 2024-09-14T05:29:35.501 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.501 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.501 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.502 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3603 2024-09-14T05:29:35.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.502 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:29:35.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:29:35.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:29:35.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1899999999999999e-07 2024-09-14T05:29:35.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519998+0000", 2024-09-14T05:29:35.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5000000000000001e-07 2024-09-14T05:29:35.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520002+0000", 2024-09-14T05:29:35.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7299999999999999e-06 2024-09-14T05:29:35.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520004+0000", 2024-09-14T05:29:35.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.429e-06 2024-09-14T05:29:35.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616493+0000", 2024-09-14T05:29:35.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096488345000000003 2024-09-14T05:29:35.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616544+0000", 2024-09-14T05:29:35.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1724000000000001e-05 2024-09-14T05:29:35.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616794+0000", 2024-09-14T05:29:35.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000249139 2024-09-14T05:29:35.505 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.505 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.505 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.506 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3604 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5668864~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.506 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:29:35.506 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.9547851870000001, 2024-09-14T05:29:35.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108222999999994, 2024-09-14T05:29:35.506 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.506 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.506 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.506 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.506 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.506 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.506 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3604 2024-09-14T05:29:35.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.506 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:29:35.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:29:35.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:29:35.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7000000000000001e-07 2024-09-14T05:29:35.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520013+0000", 2024-09-14T05:29:35.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.389e-06 2024-09-14T05:29:35.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520018+0000", 2024-09-14T05:29:35.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3019999999999996e-06 2024-09-14T05:29:35.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520026+0000", 2024-09-14T05:29:35.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6170000000000007e-06 2024-09-14T05:29:35.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616816+0000", 2024-09-14T05:29:35.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096789766999999999 2024-09-14T05:29:35.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616871+0000", 2024-09-14T05:29:35.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4811000000000001e-05 2024-09-14T05:29:35.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617120+0000", 2024-09-14T05:29:35.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024906700000000001 2024-09-14T05:29:35.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.510 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.510 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3605 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6160384~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.510 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:29:35.510 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.954766727, 2024-09-14T05:29:35.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097423945999999997, 2024-09-14T05:29:35.511 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.511 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.511 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.511 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3605 2024-09-14T05:29:35.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.511 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:29:35.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:29:35.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:29:35.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3999999999999998e-07 2024-09-14T05:29:35.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520032+0000", 2024-09-14T05:29:35.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.083e-06 2024-09-14T05:29:35.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520036+0000", 2024-09-14T05:29:35.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.019e-06 2024-09-14T05:29:35.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520038+0000", 2024-09-14T05:29:35.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4849999999999999e-06 2024-09-14T05:29:35.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617147+0000", 2024-09-14T05:29:35.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108665999999996 2024-09-14T05:29:35.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617199+0000", 2024-09-14T05:29:35.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2558000000000001e-05 2024-09-14T05:29:35.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617454+0000", 2024-09-14T05:29:35.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025489499999999999 2024-09-14T05:29:35.514 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.514 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.514 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.515 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3606 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6651904~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.515 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:29:35.515 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.9547560050000001, 2024-09-14T05:29:35.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097745705000000002, 2024-09-14T05:29:35.515 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.515 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.515 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.515 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3606 2024-09-14T05:29:35.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.516 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:29:35.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:29:35.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:29:35.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-14T05:29:35.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520042+0000", 2024-09-14T05:29:35.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4799999999999997e-07 2024-09-14T05:29:35.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520046+0000", 2024-09-14T05:29:35.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8149999999999999e-06 2024-09-14T05:29:35.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520049+0000", 2024-09-14T05:29:35.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.576e-06 2024-09-14T05:29:35.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617476+0000", 2024-09-14T05:29:35.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097427558999999997 2024-09-14T05:29:35.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617522+0000", 2024-09-14T05:29:35.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5927999999999998e-05 2024-09-14T05:29:35.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617787+0000", 2024-09-14T05:29:35.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000264663 2024-09-14T05:29:35.519 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.519 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.519 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.519 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3607 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7143424~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.519 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:29:35.519 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.954745478, 2024-09-14T05:29:35.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097871125000000003, 2024-09-14T05:29:35.519 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.519 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.520 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.520 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3607 2024-09-14T05:29:35.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.520 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:29:35.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:29:35.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520052+0000", 2024-09-14T05:29:35.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.03e-07 2024-09-14T05:29:35.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520053+0000", 2024-09-14T05:29:35.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.85e-07 2024-09-14T05:29:35.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520056+0000", 2024-09-14T05:29:35.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9140000000000001e-06 2024-09-14T05:29:35.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520059+0000", 2024-09-14T05:29:35.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4600000000000002e-06 2024-09-14T05:29:35.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617809+0000", 2024-09-14T05:29:35.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097750216000000001 2024-09-14T05:29:35.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617864+0000", 2024-09-14T05:29:35.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5211999999999999e-05 2024-09-14T05:29:35.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617923+0000", 2024-09-14T05:29:35.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8235000000000001e-05 2024-09-14T05:29:35.524 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.524 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.524 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.524 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3608 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7634944~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.524 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:29:35.524 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.9547351700000002, 2024-09-14T05:29:35.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098199212999999994, 2024-09-14T05:29:35.524 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.525 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.525 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.525 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3608 2024-09-14T05:29:35.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.525 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:29:35.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:29:35.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525953+0000", 2024-09-14T05:29:35.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058911850000000002 2024-09-14T05:29:35.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525957+0000", 2024-09-14T05:29:35.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2379999999999999e-06 2024-09-14T05:29:35.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525967+0000", 2024-09-14T05:29:35.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5389999999999996e-06 2024-09-14T05:29:35.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525972+0000", 2024-09-14T05:29:35.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.541e-06 2024-09-14T05:29:35.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617945+0000", 2024-09-14T05:29:35.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091972597000000003 2024-09-14T05:29:35.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617992+0000", 2024-09-14T05:29:35.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7397e-05 2024-09-14T05:29:35.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.618261+0000", 2024-09-14T05:29:35.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026871600000000001 2024-09-14T05:29:35.528 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.528 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.528 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3640 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6586368~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:29:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.9486273280000002, 2024-09-14T05:29:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097068580000000002, 2024-09-14T05:29:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3640 2024-09-14T05:29:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:29:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:29:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:29:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.23e-07 2024-09-14T05:29:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526171+0000", 2024-09-14T05:29:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:29:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526175+0000", 2024-09-14T05:29:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8369999999999999e-06 2024-09-14T05:29:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526177+0000", 2024-09-14T05:29:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4959999999999999e-06 2024-09-14T05:29:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622847+0000", 2024-09-14T05:29:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096669790000000005 2024-09-14T05:29:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622902+0000", 2024-09-14T05:29:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5157000000000001e-05 2024-09-14T05:29:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623238+0000", 2024-09-14T05:29:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033603399999999997 2024-09-14T05:29:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3641 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7077888~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:29:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.9486168180000001, 2024-09-14T05:29:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097371141999999994, 2024-09-14T05:29:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3641 2024-09-14T05:29:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:29:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:29:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531851+0000", 2024-09-14T05:29:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005671265 2024-09-14T05:29:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531855+0000", 2024-09-14T05:29:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0849999999999999e-06 2024-09-14T05:29:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531867+0000", 2024-09-14T05:29:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1028e-05 2024-09-14T05:29:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531872+0000", 2024-09-14T05:29:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000003e-06 2024-09-14T05:29:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623251+0000", 2024-09-14T05:29:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091378980999999998 2024-09-14T05:29:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623295+0000", 2024-09-14T05:29:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3881999999999998e-05 2024-09-14T05:29:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623551+0000", 2024-09-14T05:29:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025636099999999998 2024-09-14T05:29:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3721 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:29:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.9427585430000001, 2024-09-14T05:29:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096806472000000005, 2024-09-14T05:29:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3721 2024-09-14T05:29:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:29:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:29:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532039+0000", 2024-09-14T05:29:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.05e-07 2024-09-14T05:29:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532040+0000", 2024-09-14T05:29:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9099999999999991e-07 2024-09-14T05:29:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532043+0000", 2024-09-14T05:29:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6610000000000002e-06 2024-09-14T05:29:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532046+0000", 2024-09-14T05:29:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4080000000000001e-06 2024-09-14T05:29:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628545+0000", 2024-09-14T05:29:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096499615999999996 2024-09-14T05:29:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628592+0000", 2024-09-14T05:29:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6907000000000003e-05 2024-09-14T05:29:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628845+0000", 2024-09-14T05:29:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025268399999999998 2024-09-14T05:29:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3722 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6193152~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:29:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.942748463, 2024-09-14T05:29:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097146587000000006, 2024-09-14T05:29:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3722 2024-09-14T05:29:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:29:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:29:35.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538191+0000", 2024-09-14T05:29:35.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061429969999999999 2024-09-14T05:29:35.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538200+0000", 2024-09-14T05:29:35.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7069999999999998e-06 2024-09-14T05:29:35.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538212+0000", 2024-09-14T05:29:35.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1504e-05 2024-09-14T05:29:35.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538219+0000", 2024-09-14T05:29:35.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7500000000000003e-06 2024-09-14T05:29:35.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628869+0000", 2024-09-14T05:29:35.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090649224 2024-09-14T05:29:35.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628919+0000", 2024-09-14T05:29:35.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0144e-05 2024-09-14T05:29:35.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.629195+0000", 2024-09-14T05:29:35.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027626099999999998 2024-09-14T05:29:35.546 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.546 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.546 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.547 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3758 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7110656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.547 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:29:35.547 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.9361542470000002, 2024-09-14T05:29:35.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096977544999999998, 2024-09-14T05:29:35.547 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.547 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.547 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.547 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3758 2024-09-14T05:29:35.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.548 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:29:35.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:29:35.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:29:35.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:29:35.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538645+0000", 2024-09-14T05:29:35.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.843e-06 2024-09-14T05:29:35.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538651+0000", 2024-09-14T05:29:35.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6169999999999998e-06 2024-09-14T05:29:35.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538655+0000", 2024-09-14T05:29:35.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5399999999999997e-06 2024-09-14T05:29:35.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635225+0000", 2024-09-14T05:29:35.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096570149999999993 2024-09-14T05:29:35.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635272+0000", 2024-09-14T05:29:35.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7199e-05 2024-09-14T05:29:35.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635620+0000", 2024-09-14T05:29:35.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034782600000000003 2024-09-14T05:29:35.551 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.551 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.551 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.551 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3759 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.551 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:29:35.551 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.9361363360000001, 2024-09-14T05:29:35.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097335331999999997, 2024-09-14T05:29:35.551 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.551 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.552 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.552 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.552 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.552 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.552 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3759 2024-09-14T05:29:35.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.552 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:29:35.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:29:35.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544013+0000", 2024-09-14T05:29:35.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053520829999999997 2024-09-14T05:29:35.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544016+0000", 2024-09-14T05:29:35.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6660000000000001e-06 2024-09-14T05:29:35.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544026+0000", 2024-09-14T05:29:35.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7089999999999997e-06 2024-09-14T05:29:35.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544043+0000", 2024-09-14T05:29:35.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833999999999998e-05 2024-09-14T05:29:35.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635646+0000", 2024-09-14T05:29:35.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091603544999999995 2024-09-14T05:29:35.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635696+0000", 2024-09-14T05:29:35.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9307000000000001e-05 2024-09-14T05:29:35.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635996+0000", 2024-09-14T05:29:35.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030018800000000001 2024-09-14T05:29:35.555 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.555 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.555 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.556 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3838 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6225920~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.556 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:29:35.556 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.9304746559999999, 2024-09-14T05:29:35.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096894235999999995, 2024-09-14T05:29:35.556 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.556 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.556 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.556 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.556 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.556 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.556 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3838 2024-09-14T05:29:35.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.557 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:29:35.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:29:35.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544323+0000", 2024-09-14T05:29:35.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-14T05:29:35.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544325+0000", 2024-09-14T05:29:35.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.75e-06 2024-09-14T05:29:35.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544331+0000", 2024-09-14T05:29:35.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5019999999999998e-06 2024-09-14T05:29:35.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544335+0000", 2024-09-14T05:29:35.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2989999999999998e-06 2024-09-14T05:29:35.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640869+0000", 2024-09-14T05:29:35.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096533351000000003 2024-09-14T05:29:35.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640919+0000", 2024-09-14T05:29:35.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999999e-05 2024-09-14T05:29:35.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641217+0000", 2024-09-14T05:29:35.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029717600000000002 2024-09-14T05:29:35.560 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.560 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.560 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.560 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3839 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.560 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:29:35.560 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.9304557180000002, 2024-09-14T05:29:35.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097240339999999995, 2024-09-14T05:29:35.560 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.560 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.560 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.561 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.561 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.561 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.561 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3839 2024-09-14T05:29:35.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.561 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:29:35.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:29:35.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549834+0000", 2024-09-14T05:29:35.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054927819999999999 2024-09-14T05:29:35.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549838+0000", 2024-09-14T05:29:35.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8689999999999997e-06 2024-09-14T05:29:35.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549848+0000", 2024-09-14T05:29:35.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0154e-05 2024-09-14T05:29:35.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549853+0000", 2024-09-14T05:29:35.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9740000000000001e-06 2024-09-14T05:29:35.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641245+0000", 2024-09-14T05:29:35.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091392245999999996 2024-09-14T05:29:35.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641295+0000", 2024-09-14T05:29:35.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0018999999999997e-05 2024-09-14T05:29:35.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641582+0000", 2024-09-14T05:29:35.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028629600000000001 2024-09-14T05:29:35.564 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.564 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.564 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.564 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3868 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:35.564 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:29:35.564 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.9247931400000002, 2024-09-14T05:29:35.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096859555999999999, 2024-09-14T05:29:35.565 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.565 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.565 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:35.565 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.565 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.565 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.565 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3868 2024-09-14T05:29:35.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.565 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:29:35.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:29:35.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556173+0000", 2024-09-14T05:29:35.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006169038 2024-09-14T05:29:35.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556177+0000", 2024-09-14T05:29:35.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0010000000000003e-06 2024-09-14T05:29:35.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556188+0000", 2024-09-14T05:29:35.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0675e-05 2024-09-14T05:29:35.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556199+0000", 2024-09-14T05:29:35.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.155e-05 2024-09-14T05:29:35.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646511+0000", 2024-09-14T05:29:35.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090312187000000002 2024-09-14T05:29:35.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646563+0000", 2024-09-14T05:29:35.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2219000000000003e-05 2024-09-14T05:29:35.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646863+0000", 2024-09-14T05:29:35.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029988600000000002 2024-09-14T05:29:35.568 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.568 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.568 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.569 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.569 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.569 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:35.571 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:29:35.571 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:35.569+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:35.572 INFO:teuthology.orchestra.run.smithi071.stdout:568 read 0x730fd68 thru 0x7319aeb (0x9d84 bytes) 2024-09-14T05:29:35.572 INFO:teuthology.orchestra.run.smithi071.stdout:569 write 0x858b106 thru 0x8595ea2 (0xad9d bytes) 2024-09-14T05:29:35.572 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:35.573 INFO:teuthology.orchestra.run.smithi071.stdout:572 trunc from 0x94972c6 to 0x8216334 2024-09-14T05:29:35.573 INFO:teuthology.orchestra.run.smithi071.stdout:574 write 0xe6da1ec thru 0xe6e0ae2 (0x68f7 bytes) 2024-09-14T05:29:35.573 INFO:teuthology.orchestra.run.smithi071.stdout:575 read 0x8327403 thru 0x832e40d (0x700b bytes) 2024-09-14T05:29:35.573 INFO:teuthology.orchestra.run.smithi071.stdout:576 write 0xec29f7e thru 0xec2eee8 (0x4f6b bytes) 2024-09-14T05:29:35.573 INFO:teuthology.orchestra.run.smithi071.stdout:578 read 0x44eda4e thru 0x44f768b (0x9c3e bytes) 2024-09-14T05:29:35.573 INFO:teuthology.orchestra.run.smithi071.stdout:580 write 0xc141b33 thru 0xc14849a (0x6968 bytes) 2024-09-14T05:29:35.573 INFO:teuthology.orchestra.run.smithi071.stdout:581 writesame 0x34c1cfb thru 0x34cf5ca data_size 0x8(0xd8d0 bytes) 2024-09-14T05:29:35.573 INFO:teuthology.orchestra.run.smithi071.stdout:582 read 0x591b47 thru 0x59bdca (0xa284 bytes) 2024-09-14T05:29:35.573 INFO:teuthology.orchestra.run.smithi071.stdout:583 punch from 0x371d7f8 to 0x37284de, (0xace6 bytes) 2024-09-14T05:29:35.573 INFO:teuthology.orchestra.run.smithi071.stdout:585 read 0xda1c36f thru 0xda1ec87 (0x2919 bytes) 2024-09-14T05:29:35.573 INFO:teuthology.orchestra.run.smithi071.stdout:586 write 0xa64fc5d thru 0xa65da52 (0xddf6 bytes) 2024-09-14T05:29:35.573 INFO:teuthology.orchestra.run.smithi071.stdout:587 writesame 0xaedacb0 thru 0xaedc54f data_size 0x10(0x18a0 bytes) 2024-09-14T05:29:35.573 INFO:teuthology.orchestra.run.smithi071.stdout:588 read 0xa6d5b93 thru 0xa6d854f (0x29bd bytes) 2024-09-14T05:29:35.573 INFO:teuthology.orchestra.run.smithi071.stdout:589 read 0xe173621 thru 0xe182f89 (0xf969 bytes) 2024-09-14T05:29:35.578 INFO:teuthology.orchestra.run.smithi071.stdout:590 trunc from 0xec2eee9 to 0xbe2232 2024-09-14T05:29:35.629 INFO:teuthology.orchestra.run.smithi071.stdout:591 read 0x8748c0 thru 0x87a94c (0x608d bytes) 2024-09-14T05:29:35.629 INFO:teuthology.orchestra.run.smithi071.stdout:592 read 0x3adafb thru 0x3b22d2 (0x47d8 bytes) 2024-09-14T05:29:35.629 INFO:teuthology.orchestra.run.smithi071.stdout:593 read 0x6e7b2e thru 0x6ebddd (0x42b0 bytes) 2024-09-14T05:29:35.630 INFO:teuthology.orchestra.run.smithi071.stdout:594 punch from 0x6c6675 to 0x6d6188, (0xfb13 bytes) 2024-09-14T05:29:35.652 INFO:teuthology.orchestra.run.smithi071.stdout:595 write 0xd86fdf9 thru 0xd879381 (0x9589 bytes) 2024-09-14T05:29:35.665 INFO:teuthology.orchestra.run.smithi071.stdout:596 trunc from 0xd879382 to 0x4c9f87b 2024-09-14T05:29:35.670 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:35.670+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:35.678 INFO:teuthology.orchestra.run.smithi071.stdout:597 write 0xfd70a7 thru 0xfe4e29 (0xdd83 bytes) 2024-09-14T05:29:35.701 INFO:teuthology.orchestra.run.smithi071.stdout:598 writesame 0xea7cd39 thru 0xea7e639 data_size 0x1(0x1901 bytes) 2024-09-14T05:29:35.701 INFO:teuthology.orchestra.run.smithi071.stdout:599 writesame 0x65e29ff thru 0x65ec159 data_size 0x1(0x975b bytes) 2024-09-14T05:29:35.718 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:35.718 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:35.718 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:29:35.718 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:35.724 INFO:teuthology.orchestra.run.smithi071.stdout:602 read 0x7c291f0 thru 0x7c3807e (0xee8f bytes) 2024-09-14T05:29:35.724 INFO:teuthology.orchestra.run.smithi071.stdout:603 read 0xc546ea8 thru 0xc54f05d (0x81b6 bytes) 2024-09-14T05:29:35.724 INFO:teuthology.orchestra.run.smithi071.stdout:605 write 0xbe5cf84 thru 0xbe6aadc (0xdb59 bytes) 2024-09-14T05:29:35.727 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:29:35.728 INFO:teuthology.orchestra.run.smithi071.stdout:606 read 0x7692cf8 thru 0x769f066 (0xc36f bytes) 2024-09-14T05:29:35.728 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:35.728 INFO:teuthology.orchestra.run.smithi071.stdout:609 punch from 0x551adae to 0x5525e13, (0xb065 bytes) 2024-09-14T05:29:35.728 INFO:teuthology.orchestra.run.smithi071.stdout:611 punch from 0xb1d8094 to 0xb1e71ff, (0xf16b bytes) 2024-09-14T05:29:35.728 INFO:teuthology.orchestra.run.smithi071.stdout:612 write 0xee8ba5 thru 0xee94ad (0x909 bytes) 2024-09-14T05:29:35.729 INFO:teuthology.orchestra.run.smithi071.stdout:613 write 0xca9182a thru 0xca9738f (0x5b66 bytes) 2024-09-14T05:29:35.733 INFO:teuthology.orchestra.run.smithi071.stdout:615 punch from 0x71b716c to 0x71bc2ae, (0x5142 bytes) 2024-09-14T05:29:35.733 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:35.733 INFO:teuthology.orchestra.run.smithi071.stdout:618 writesame 0x24df898 thru 0x24e416b data_size 0x4f(0x48d4 bytes) 2024-09-14T05:29:35.736 INFO:teuthology.orchestra.run.smithi071.stdout:619 writesame 0x23cff45 thru 0x23dc040 data_size 0x8a(0xc0fc bytes) 2024-09-14T05:29:35.740 INFO:teuthology.orchestra.run.smithi071.stdout:620 write 0x80001de thru 0x80053cf (0x51f2 bytes) 2024-09-14T05:29:35.745 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:35.745 INFO:teuthology.orchestra.run.smithi071.stdout:622 writesame 0xbb3aa86 thru 0xbb44ce7 data_size 0x5(0xa262 bytes) 2024-09-14T05:29:35.756 INFO:teuthology.orchestra.run.smithi071.stdout:626 trunc from 0xea7e63a to 0x98bc41b 2024-09-14T05:29:35.771 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:35.770+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:35.777 INFO:teuthology.orchestra.run.smithi071.stdout:627 write 0xb22378a thru 0xb22732b (0x3ba2 bytes) 2024-09-14T05:29:35.782 INFO:teuthology.orchestra.run.smithi071.stdout:628 read 0x8153f7c thru 0x8154784 (0x809 bytes) 2024-09-14T05:29:35.783 INFO:teuthology.orchestra.run.smithi071.stdout:629 writesame 0x3c07464 thru 0x3c09e0a data_size 0x1(0x29a7 bytes) 2024-09-14T05:29:35.789 INFO:teuthology.orchestra.run.smithi071.stdout:633 writesame 0x9d3ab4f thru 0x9d3f319 data_size 0x1(0x47cb bytes) 2024-09-14T05:29:35.805 INFO:teuthology.orchestra.run.smithi071.stdout:634 write 0xc3af356 thru 0xc3b1a0b (0x26b6 bytes) 2024-09-14T05:29:35.823 INFO:teuthology.orchestra.run.smithi071.stdout:636 trunc from 0xc3b1a0c to 0x2e290 2024-09-14T05:29:35.853 INFO:teuthology.orchestra.run.smithi071.stdout:637 punch from 0x1343a to 0x1d087, (0x9c4d bytes) 2024-09-14T05:29:35.853 INFO:teuthology.orchestra.run.smithi071.stdout:638 trunc from 0x2e290 to 0xa5767a 2024-09-14T05:29:35.858 INFO:teuthology.orchestra.run.smithi071.stdout:639 read 0x37c6e4 thru 0x38af06 (0xe823 bytes) 2024-09-14T05:29:35.859 INFO:teuthology.orchestra.run.smithi071.stdout:640 punch from 0x9dc414 to 0x9e27af, (0x639b bytes) 2024-09-14T05:29:35.872 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:35.871+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:35.874 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:35.874 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:35.874 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:29:35.874 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:29:35.875 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:35.881 INFO:teuthology.orchestra.run.smithi071.stdout:641 writesame 0xeb22f3f thru 0xeb2fb5c data_size 0x9(0xcc1e bytes) 2024-09-14T05:29:35.883 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:29:35.890 INFO:teuthology.orchestra.run.smithi071.stdout:642 trunc from 0xeb2fb5d to 0x6cb0569 2024-09-14T05:29:35.897 INFO:teuthology.orchestra.run.smithi071.stdout:643 write 0x30f7ce7 thru 0x3104e4d (0xd167 bytes) 2024-09-14T05:29:35.901 INFO:teuthology.orchestra.run.smithi071.stdout:644 write 0x3e6269a thru 0x3e6bdc7 (0x972e bytes) 2024-09-14T05:29:35.903 INFO:teuthology.orchestra.run.smithi071.stdout:645 trunc from 0x6cb0569 to 0x6b9a147 2024-09-14T05:29:35.921 INFO:teuthology.orchestra.run.smithi071.stdout:646 write 0xc9d877f thru 0xc9df1f3 (0x6a75 bytes) 2024-09-14T05:29:35.924 INFO:teuthology.orchestra.run.smithi071.stdout:648 read 0x4b91214 thru 0x4b9bd33 (0xab20 bytes) 2024-09-14T05:29:35.924 INFO:teuthology.orchestra.run.smithi071.stdout:650 writesame 0xc7eab56 thru 0xc7ec77c data_size 0x1(0x1c27 bytes) 2024-09-14T05:29:35.928 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:35.928 INFO:teuthology.orchestra.run.smithi071.stdout:652 write 0x917f0ee thru 0x91875ae (0x84c1 bytes) 2024-09-14T05:29:35.934 INFO:teuthology.orchestra.run.smithi071.stdout:653 trunc from 0xc9df1f4 to 0xe3a3d33 2024-09-14T05:29:35.940 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:35.940 INFO:teuthology.orchestra.run.smithi071.stdout:655 read 0xf5d862 thru 0xf5ed31 (0x14d0 bytes) 2024-09-14T05:29:35.940 INFO:teuthology.orchestra.run.smithi071.stdout:657 read 0x2470874 thru 0x2472508 (0x1c95 bytes) 2024-09-14T05:29:35.940 INFO:teuthology.orchestra.run.smithi071.stdout:658 read 0x25fe386 thru 0x2601f8b (0x3c06 bytes) 2024-09-14T05:29:35.940 INFO:teuthology.orchestra.run.smithi071.stdout:659 read 0x24f2730 thru 0x24fad66 (0x8637 bytes) 2024-09-14T05:29:35.940 INFO:teuthology.orchestra.run.smithi071.stdout:660 read 0x12484bb thru 0x12526db (0xa221 bytes) 2024-09-14T05:29:35.940 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:35.940 INFO:teuthology.orchestra.run.smithi071.stdout:662 clone 8 order 20 su 131072 sc 10 2024-09-14T05:29:35.972 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:35.972+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:35.990 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:35.990 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:29:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:29:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:29:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1989 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 16326080~54496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:29:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.4277131340000002, 2024-09-14T05:29:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096664322999999996, 2024-09-14T05:29:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1989 2024-09-14T05:29:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:29:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:29:35.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:35.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:35.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561455+0000", 2024-09-14T05:29:35.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1699999999999997e-07 2024-09-14T05:29:35.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:35.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590703+0000", 2024-09-14T05:29:35.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029248225999999999 2024-09-14T05:29:35.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:35.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590732+0000", 2024-09-14T05:29:35.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8679000000000001e-05 2024-09-14T05:29:35.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:35.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590759+0000", 2024-09-14T05:29:35.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7319999999999999e-05 2024-09-14T05:29:35.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:35.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590835+0000", 2024-09-14T05:29:35.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5543999999999994e-05 2024-09-14T05:29:35.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:35.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590956+0000", 2024-09-14T05:29:35.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012168 2024-09-14T05:29:35.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 1", 2024-09-14T05:29:35.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.591261+0000", 2024-09-14T05:29:35.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030484799999999998 2024-09-14T05:29:35.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:35.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.637682+0000", 2024-09-14T05:29:35.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046420550999999997 2024-09-14T05:29:35.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:35.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.657912+0000", 2024-09-14T05:29:35.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020230011999999999 2024-09-14T05:29:35.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:35.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658007+0000", 2024-09-14T05:29:35.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5814000000000001e-05 2024-09-14T05:29:35.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:35.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658118+0000", 2024-09-14T05:29:35.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110832 2024-09-14T05:29:35.997 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.997 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:35.997 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:35.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.997 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2033 3.1e 3:7e47f6eb:::rbd_data.112d1f36d097.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30853661~63531] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:35.997 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:29:35.997 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.3164384680000003, 2024-09-14T05:29:35.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065167630000000004, 2024-09-14T05:29:35.997 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:35.998 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:35.998 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:35.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:35.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:35.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:35.998 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2033 2024-09-14T05:29:35.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:35.998 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:35.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:35.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:35.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:29:36.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:29:36.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672730+0000", 2024-09-14T05:29:36.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.393e-06 2024-09-14T05:29:36.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691289+0000", 2024-09-14T05:29:36.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018558991 2024-09-14T05:29:36.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691316+0000", 2024-09-14T05:29:36.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7498999999999999e-05 2024-09-14T05:29:36.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691331+0000", 2024-09-14T05:29:36.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4652e-05 2024-09-14T05:29:36.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691397+0000", 2024-09-14T05:29:36.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6112000000000007e-05 2024-09-14T05:29:36.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691509+0000", 2024-09-14T05:29:36.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111599 2024-09-14T05:29:36.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:36.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.692253+0000", 2024-09-14T05:29:36.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074456200000000004 2024-09-14T05:29:36.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:36.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.719140+0000", 2024-09-14T05:29:36.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026886542999999999 2024-09-14T05:29:36.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:36.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737789+0000", 2024-09-14T05:29:36.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018649203999999999 2024-09-14T05:29:36.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737846+0000", 2024-09-14T05:29:36.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6501e-05 2024-09-14T05:29:36.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737896+0000", 2024-09-14T05:29:36.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0574000000000001e-05 2024-09-14T05:29:36.004 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.004 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.004 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.005 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3008 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13729792~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.005 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:29:36.005 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6855145980000001, 2024-09-14T05:29:36.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061007651000000003, 2024-09-14T05:29:36.005 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.005 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.005 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.005 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.005 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.005 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.005 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3008 2024-09-14T05:29:36.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.006 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:29:36.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:29:36.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303653+0000", 2024-09-14T05:29:36.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0699999999999999e-07 2024-09-14T05:29:36.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303654+0000", 2024-09-14T05:29:36.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-07 2024-09-14T05:29:36.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303657+0000", 2024-09-14T05:29:36.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2890000000000002e-06 2024-09-14T05:29:36.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303661+0000", 2024-09-14T05:29:36.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3290000000000004e-06 2024-09-14T05:29:36.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364233+0000", 2024-09-14T05:29:36.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060572134 2024-09-14T05:29:36.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364315+0000", 2024-09-14T05:29:36.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1549000000000001e-05 2024-09-14T05:29:36.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364660+0000", 2024-09-14T05:29:36.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034525300000000001 2024-09-14T05:29:36.009 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.009 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.009 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.010 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3009 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14221312~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.010 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:29:36.010 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6855005079999996, 2024-09-14T05:29:36.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061445063000000001, 2024-09-14T05:29:36.010 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.010 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.010 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.010 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.010 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.010 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.010 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3009 2024-09-14T05:29:36.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.010 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:29:36.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:29:36.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:29:36.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:29:36.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303669+0000", 2024-09-14T05:29:36.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7150000000000001e-06 2024-09-14T05:29:36.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303675+0000", 2024-09-14T05:29:36.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5339999999999996e-06 2024-09-14T05:29:36.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303681+0000", 2024-09-14T05:29:36.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3639999999999998e-06 2024-09-14T05:29:36.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364705+0000", 2024-09-14T05:29:36.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061024123 2024-09-14T05:29:36.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364786+0000", 2024-09-14T05:29:36.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1378999999999995e-05 2024-09-14T05:29:36.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365112+0000", 2024-09-14T05:29:36.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032558700000000001 2024-09-14T05:29:36.014 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.014 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.014 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.014 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3010 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14712832~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.014 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:29:36.014 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6854796439999999, 2024-09-14T05:29:36.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061834630000000002, 2024-09-14T05:29:36.015 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.015 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.015 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.015 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3010 2024-09-14T05:29:36.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.015 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:29:36.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:29:36.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303688+0000", 2024-09-14T05:29:36.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:29:36.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303690+0000", 2024-09-14T05:29:36.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9460000000000001e-06 2024-09-14T05:29:36.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303696+0000", 2024-09-14T05:29:36.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2230000000000004e-06 2024-09-14T05:29:36.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303700+0000", 2024-09-14T05:29:36.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7290000000000002e-06 2024-09-14T05:29:36.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365163+0000", 2024-09-14T05:29:36.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061463093000000003 2024-09-14T05:29:36.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365214+0000", 2024-09-14T05:29:36.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0977999999999999e-05 2024-09-14T05:29:36.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365522+0000", 2024-09-14T05:29:36.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000308227 2024-09-14T05:29:36.018 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.019 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.019 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.019 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3241 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.019 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:29:36.019 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6598052220000001, 2024-09-14T05:29:36.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061041892, 2024-09-14T05:29:36.019 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.019 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.019 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.020 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.020 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3241 2024-09-14T05:29:36.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.020 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:29:36.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:29:36.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329496+0000", 2024-09-14T05:29:36.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013421799999999999 2024-09-14T05:29:36.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329499+0000", 2024-09-14T05:29:36.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8459999999999999e-06 2024-09-14T05:29:36.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329505+0000", 2024-09-14T05:29:36.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4320000000000004e-06 2024-09-14T05:29:36.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329509+0000", 2024-09-14T05:29:36.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0339999999999998e-06 2024-09-14T05:29:36.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.389947+0000", 2024-09-14T05:29:36.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060438093999999998 2024-09-14T05:29:36.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390038+0000", 2024-09-14T05:29:36.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0253999999999993e-05 2024-09-14T05:29:36.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390404+0000", 2024-09-14T05:29:36.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036601400000000001 2024-09-14T05:29:36.023 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.023 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.023 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.024 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3242 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.024 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:29:36.024 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6596503140000003, 2024-09-14T05:29:36.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061282426000000001, 2024-09-14T05:29:36.024 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.024 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.024 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.024 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.024 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.024 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.024 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3242 2024-09-14T05:29:36.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:29:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:29:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:29:36.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9900000000000001e-07 2024-09-14T05:29:36.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329518+0000", 2024-09-14T05:29:36.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.328e-06 2024-09-14T05:29:36.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329522+0000", 2024-09-14T05:29:36.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.563e-06 2024-09-14T05:29:36.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329525+0000", 2024-09-14T05:29:36.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7379999999999999e-06 2024-09-14T05:29:36.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390437+0000", 2024-09-14T05:29:36.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060912463 2024-09-14T05:29:36.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390498+0000", 2024-09-14T05:29:36.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0615999999999997e-05 2024-09-14T05:29:36.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390799+0000", 2024-09-14T05:29:36.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030131899999999998 2024-09-14T05:29:36.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.028 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.029 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3243 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.029 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:29:36.029 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6596385280000003, 2024-09-14T05:29:36.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061728980000000003, 2024-09-14T05:29:36.029 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.029 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.030 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.030 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.030 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.030 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.030 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3243 2024-09-14T05:29:36.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.030 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:29:36.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:29:36.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329692+0000", 2024-09-14T05:29:36.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000163948 2024-09-14T05:29:36.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329696+0000", 2024-09-14T05:29:36.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5540000000000002e-06 2024-09-14T05:29:36.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329706+0000", 2024-09-14T05:29:36.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7960000000000007e-06 2024-09-14T05:29:36.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329712+0000", 2024-09-14T05:29:36.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6109999999999999e-06 2024-09-14T05:29:36.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390861+0000", 2024-09-14T05:29:36.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061148767 2024-09-14T05:29:36.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390942+0000", 2024-09-14T05:29:36.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1266000000000004e-05 2024-09-14T05:29:36.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391258+0000", 2024-09-14T05:29:36.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031503800000000002 2024-09-14T05:29:36.033 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.034 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.034 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.034 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3244 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14778368~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.034 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:29:36.034 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6594434800000002, 2024-09-14T05:29:36.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062050594000000001, 2024-09-14T05:29:36.034 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.035 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.035 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.035 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.035 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.035 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.035 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3244 2024-09-14T05:29:36.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.035 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:29:36.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:29:36.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329916+0000", 2024-09-14T05:29:36.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019290799999999999 2024-09-14T05:29:36.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329919+0000", 2024-09-14T05:29:36.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9019999999999999e-06 2024-09-14T05:29:36.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329926+0000", 2024-09-14T05:29:36.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9e-06 2024-09-14T05:29:36.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329932+0000", 2024-09-14T05:29:36.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.088e-06 2024-09-14T05:29:36.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391299+0000", 2024-09-14T05:29:36.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061366136000000002 2024-09-14T05:29:36.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391391+0000", 2024-09-14T05:29:36.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2204e-05 2024-09-14T05:29:36.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391774+0000", 2024-09-14T05:29:36.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038345600000000001 2024-09-14T05:29:36.039 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.039 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.039 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.039 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3921 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 5308416~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.039 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:29:36.039 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5878996699999997, 2024-09-14T05:29:36.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060842694000000003, 2024-09-14T05:29:36.039 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.039 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.039 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.040 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3921 2024-09-14T05:29:36.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.040 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:29:36.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:29:36.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401705+0000", 2024-09-14T05:29:36.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768499999999999 2024-09-14T05:29:36.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401709+0000", 2024-09-14T05:29:36.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0319999999999997e-06 2024-09-14T05:29:36.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401720+0000", 2024-09-14T05:29:36.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1005000000000001e-05 2024-09-14T05:29:36.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401727+0000", 2024-09-14T05:29:36.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1879999999999996e-06 2024-09-14T05:29:36.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461748+0000", 2024-09-14T05:29:36.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060021199999999997 2024-09-14T05:29:36.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461793+0000", 2024-09-14T05:29:36.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4530999999999998e-05 2024-09-14T05:29:36.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462110+0000", 2024-09-14T05:29:36.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031705299999999997 2024-09-14T05:29:36.044 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.044 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.044 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.044 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3923 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 6291456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.044 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:29:36.044 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5872536420000003, 2024-09-14T05:29:36.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060891273000000003, 2024-09-14T05:29:36.044 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.045 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.045 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.045 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3923 2024-09-14T05:29:36.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.045 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:29:36.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:29:36.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407054+0000", 2024-09-14T05:29:36.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005140787 2024-09-14T05:29:36.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407059+0000", 2024-09-14T05:29:36.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.04e-06 2024-09-14T05:29:36.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407069+0000", 2024-09-14T05:29:36.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4429999999999992e-06 2024-09-14T05:29:36.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407076+0000", 2024-09-14T05:29:36.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1679999999999998e-06 2024-09-14T05:29:36.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462514+0000", 2024-09-14T05:29:36.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055438551000000003 2024-09-14T05:29:36.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462562+0000", 2024-09-14T05:29:36.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7649999999999999e-05 2024-09-14T05:29:36.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462805+0000", 2024-09-14T05:29:36.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024263400000000001 2024-09-14T05:29:36.049 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.049 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.049 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.049 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4052 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 12550144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.049 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:29:36.049 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5716816329999999, 2024-09-14T05:29:36.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060872904999999998, 2024-09-14T05:29:36.049 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.049 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.050 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.050 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4052 2024-09-14T05:29:36.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.050 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:29:36.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:29:36.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417486+0000", 2024-09-14T05:29:36.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-14T05:29:36.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417487+0000", 2024-09-14T05:29:36.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4499999999999995e-07 2024-09-14T05:29:36.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417489+0000", 2024-09-14T05:29:36.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8899999999999999e-06 2024-09-14T05:29:36.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417492+0000", 2024-09-14T05:29:36.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4389999999999999e-06 2024-09-14T05:29:36.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478027+0000", 2024-09-14T05:29:36.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060534959999999999 2024-09-14T05:29:36.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478079+0000", 2024-09-14T05:29:36.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2104999999999997e-05 2024-09-14T05:29:36.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478358+0000", 2024-09-14T05:29:36.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000279351 2024-09-14T05:29:36.054 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.054 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.054 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.054 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4053 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13041664~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.054 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:29:36.054 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5716722489999997, 2024-09-14T05:29:36.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061258866000000002, 2024-09-14T05:29:36.054 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.054 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.054 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.055 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.055 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.055 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.055 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4053 2024-09-14T05:29:36.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.055 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:29:36.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:29:36.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:29:36.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.17e-07 2024-09-14T05:29:36.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417496+0000", 2024-09-14T05:29:36.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0890000000000001e-06 2024-09-14T05:29:36.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417499+0000", 2024-09-14T05:29:36.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8629999999999999e-06 2024-09-14T05:29:36.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417502+0000", 2024-09-14T05:29:36.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6290000000000002e-06 2024-09-14T05:29:36.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478387+0000", 2024-09-14T05:29:36.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060885433000000003 2024-09-14T05:29:36.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478435+0000", 2024-09-14T05:29:36.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8167999999999997e-05 2024-09-14T05:29:36.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478754+0000", 2024-09-14T05:29:36.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031846700000000001 2024-09-14T05:29:36.059 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.059 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.059 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.059 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4054 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13533184~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.059 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:29:36.059 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5716623649999999, 2024-09-14T05:29:36.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061604694000000002, 2024-09-14T05:29:36.059 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.060 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.060 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.060 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.060 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.060 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.060 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4054 2024-09-14T05:29:36.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.060 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:29:36.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:29:36.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:29:36.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.04e-07 2024-09-14T05:29:36.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417506+0000", 2024-09-14T05:29:36.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8299999999999995e-07 2024-09-14T05:29:36.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417509+0000", 2024-09-14T05:29:36.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.88e-06 2024-09-14T05:29:36.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417511+0000", 2024-09-14T05:29:36.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5950000000000001e-06 2024-09-14T05:29:36.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478784+0000", 2024-09-14T05:29:36.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061272971000000002 2024-09-14T05:29:36.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478841+0000", 2024-09-14T05:29:36.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-14T05:29:36.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479109+0000", 2024-09-14T05:29:36.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026794600000000003 2024-09-14T05:29:36.064 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.064 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.064 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.064 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4055 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14024704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.064 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:29:36.064 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5716526630000001, 2024-09-14T05:29:36.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061932299000000003, 2024-09-14T05:29:36.064 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.064 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.064 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.065 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.065 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.065 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.065 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4055 2024-09-14T05:29:36.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.065 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:29:36.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:29:36.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417515+0000", 2024-09-14T05:29:36.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-14T05:29:36.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417516+0000", 2024-09-14T05:29:36.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7199999999999997e-07 2024-09-14T05:29:36.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417519+0000", 2024-09-14T05:29:36.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.937e-06 2024-09-14T05:29:36.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417522+0000", 2024-09-14T05:29:36.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1420000000000002e-06 2024-09-14T05:29:36.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479140+0000", 2024-09-14T05:29:36.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061617933999999999 2024-09-14T05:29:36.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479188+0000", 2024-09-14T05:29:36.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8717000000000001e-05 2024-09-14T05:29:36.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479447+0000", 2024-09-14T05:29:36.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025838800000000002 2024-09-14T05:29:36.068 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.069 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.069 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.069 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4056 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14516224~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.069 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:29:36.069 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5716424900000003, 2024-09-14T05:29:36.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062263185999999998, 2024-09-14T05:29:36.069 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.069 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.069 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.069 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.069 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.069 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.070 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4056 2024-09-14T05:29:36.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.070 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:29:36.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:29:36.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:29:36.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8700000000000002e-07 2024-09-14T05:29:36.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417528+0000", 2024-09-14T05:29:36.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.579e-06 2024-09-14T05:29:36.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417533+0000", 2024-09-14T05:29:36.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2400000000000001e-06 2024-09-14T05:29:36.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417536+0000", 2024-09-14T05:29:36.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0309999999999999e-06 2024-09-14T05:29:36.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479477+0000", 2024-09-14T05:29:36.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061940967 2024-09-14T05:29:36.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479523+0000", 2024-09-14T05:29:36.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6335000000000003e-05 2024-09-14T05:29:36.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479788+0000", 2024-09-14T05:29:36.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026474700000000002 2024-09-14T05:29:36.073 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.073 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.073 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.073 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4057 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15007744~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.073 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:29:36.073 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5716280290000002, 2024-09-14T05:29:36.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062589928000000003, 2024-09-14T05:29:36.074 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.074 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.074 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.074 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.074 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.074 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.074 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4057 2024-09-14T05:29:36.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.074 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:29:36.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:29:36.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422838+0000", 2024-09-14T05:29:36.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052990349999999997 2024-09-14T05:29:36.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422854+0000", 2024-09-14T05:29:36.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5523999999999999e-05 2024-09-14T05:29:36.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422864+0000", 2024-09-14T05:29:36.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0084e-05 2024-09-14T05:29:36.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422872+0000", 2024-09-14T05:29:36.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-14T05:29:36.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479813+0000", 2024-09-14T05:29:36.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056941389000000002 2024-09-14T05:29:36.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479868+0000", 2024-09-14T05:29:36.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4973999999999999e-05 2024-09-14T05:29:36.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.480129+0000", 2024-09-14T05:29:36.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026084200000000001 2024-09-14T05:29:36.078 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.078 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.078 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.079 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4171 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14057472~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.079 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:29:36.079 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.559630469, 2024-09-14T05:29:36.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061184929999999998, 2024-09-14T05:29:36.079 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.079 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.079 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.079 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.079 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.079 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.079 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4171 2024-09-14T05:29:36.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.079 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:29:36.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:29:36.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:29:36.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.39e-07 2024-09-14T05:29:36.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429541+0000", 2024-09-14T05:29:36.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:29:36.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429547+0000", 2024-09-14T05:29:36.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3280000000000002e-06 2024-09-14T05:29:36.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429552+0000", 2024-09-14T05:29:36.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4859999999999997e-06 2024-09-14T05:29:36.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490379+0000", 2024-09-14T05:29:36.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060826681 2024-09-14T05:29:36.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490434+0000", 2024-09-14T05:29:36.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4577999999999998e-05 2024-09-14T05:29:36.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490722+0000", 2024-09-14T05:29:36.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028781400000000001 2024-09-14T05:29:36.083 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.083 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.083 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.083 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4172 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14548992~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.083 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:29:36.083 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5596084609999998, 2024-09-14T05:29:36.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061498256000000001, 2024-09-14T05:29:36.084 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.084 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.084 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.084 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.084 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.084 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.084 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4172 2024-09-14T05:29:36.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.084 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:29:36.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:29:36.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:29:36.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.72e-07 2024-09-14T05:29:36.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429561+0000", 2024-09-14T05:29:36.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.717e-06 2024-09-14T05:29:36.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429566+0000", 2024-09-14T05:29:36.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6529999999999999e-06 2024-09-14T05:29:36.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429571+0000", 2024-09-14T05:29:36.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5979999999999999e-06 2024-09-14T05:29:36.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490751+0000", 2024-09-14T05:29:36.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061180336000000002 2024-09-14T05:29:36.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490798+0000", 2024-09-14T05:29:36.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6875000000000001e-05 2024-09-14T05:29:36.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491057+0000", 2024-09-14T05:29:36.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025870500000000002 2024-09-14T05:29:36.088 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.088 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.089 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.089 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4173 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15040512~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.089 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:29:36.089 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5595908349999998, 2024-09-14T05:29:36.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061894820000000003, 2024-09-14T05:29:36.089 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.089 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.089 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.089 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.089 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.089 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.090 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4173 2024-09-14T05:29:36.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.090 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:29:36.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:29:36.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429577+0000", 2024-09-14T05:29:36.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-14T05:29:36.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429578+0000", 2024-09-14T05:29:36.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6980000000000001e-06 2024-09-14T05:29:36.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429585+0000", 2024-09-14T05:29:36.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.725e-06 2024-09-14T05:29:36.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429590+0000", 2024-09-14T05:29:36.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.938e-06 2024-09-14T05:29:36.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491091+0000", 2024-09-14T05:29:36.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061500773000000002 2024-09-14T05:29:36.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491136+0000", 2024-09-14T05:29:36.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5284000000000002e-05 2024-09-14T05:29:36.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491471+0000", 2024-09-14T05:29:36.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033485699999999998 2024-09-14T05:29:36.093 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.093 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.093 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.093 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.094 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.094 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:36.095 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:29:36.096 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:36.094+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:36.143 INFO:teuthology.orchestra.run.smithi071.stdout:leaving image image_client.0-clone7 intact 2024-09-14T05:29:36.196 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:36.195+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:36.240 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:36.240 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:36.240 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:29:36.240 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:36.249 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:29:36.297 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:36.296+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:36.308 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-14T05:29:36.354 INFO:teuthology.orchestra.run.smithi071.stdout:663 trunc from 0xe3a3d33 to 0x9a77436 2024-09-14T05:29:36.358 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:36.358 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:36.358 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:29:36.358 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:29:36.358 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:36.367 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:29:36.377 INFO:teuthology.orchestra.run.smithi071.stdout:664 punch from 0x7530590 to 0x7540550, (0xffc0 bytes) 2024-09-14T05:29:36.377 INFO:teuthology.orchestra.run.smithi071.stdout:666 punch from 0x8ae7a6c to 0x8aed99b, (0x5f2f bytes) 2024-09-14T05:29:36.385 INFO:teuthology.orchestra.run.smithi071.stdout:669 write 0xb470d21 thru 0xb4721df (0x14bf bytes) 2024-09-14T05:29:36.389 INFO:teuthology.orchestra.run.smithi071.stdout:670 writesame 0x1fe3b0c thru 0x1fe793f data_size 0x6(0x3e34 bytes) 2024-09-14T05:29:36.395 INFO:teuthology.orchestra.run.smithi071.stdout:671 read 0x54dfd5c thru 0x54eaf12 (0xb1b7 bytes) 2024-09-14T05:29:36.397 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:36.396+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:36.403 INFO:teuthology.orchestra.run.smithi071.stdout:674 trunc from 0xb4721e0 to 0x49c8e63 2024-09-14T05:29:36.427 INFO:teuthology.orchestra.run.smithi071.stdout:675 read 0x3cb79d5 thru 0x3cbd228 (0x5854 bytes) 2024-09-14T05:29:36.427 INFO:teuthology.orchestra.run.smithi071.stdout:676 read 0x3c2848 thru 0x3c3359 (0xb12 bytes) 2024-09-14T05:29:36.433 INFO:teuthology.orchestra.run.smithi071.stdout:677 write 0x52c2125 thru 0x52c2523 (0x3ff bytes) 2024-09-14T05:29:36.436 INFO:teuthology.orchestra.run.smithi071.stdout:678 write 0x128d00e thru 0x1296f74 (0x9f67 bytes) 2024-09-14T05:29:36.440 INFO:teuthology.orchestra.run.smithi071.stdout:679 write 0x1f4c97a thru 0x1f59168 (0xc7ef bytes) 2024-09-14T05:29:36.443 INFO:teuthology.orchestra.run.smithi071.stdout:680 read 0x2f58ff5 thru 0x2f59ce6 (0xcf2 bytes) 2024-09-14T05:29:36.452 INFO:teuthology.orchestra.run.smithi071.stdout:681 writesame 0x74c4024 thru 0x74cfd6e data_size 0x1d(0xbd4b bytes) 2024-09-14T05:29:36.452 INFO:teuthology.orchestra.run.smithi071.stdout:683 punch from 0x359441f to 0x359d56a, (0x914b bytes) 2024-09-14T05:29:36.452 INFO:teuthology.orchestra.run.smithi071.stdout:684 write 0x4e2c754 thru 0x4e3c10c (0xf9b9 bytes) 2024-09-14T05:29:36.454 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:36.460 INFO:teuthology.orchestra.run.smithi071.stdout:687 write 0x84e1729 thru 0x84ec7e5 (0xb0bd bytes) 2024-09-14T05:29:36.477 INFO:teuthology.orchestra.run.smithi071.stdout:688 writesame 0xea2a2a5 thru 0xea36239 data_size 0x11(0xbf95 bytes) 2024-09-14T05:29:36.477 INFO:teuthology.orchestra.run.smithi071.stdout:691 read 0x1af3068 thru 0x1afe59d (0xb536 bytes) 2024-09-14T05:29:36.477 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:36.477 INFO:teuthology.orchestra.run.smithi071.stdout:693 read 0xab8200d thru 0xab90caa (0xec9e bytes) 2024-09-14T05:29:36.477 INFO:teuthology.orchestra.run.smithi071.stdout:695 read 0x2300dd6 thru 0x23053ed (0x4618 bytes) 2024-09-14T05:29:36.477 INFO:teuthology.orchestra.run.smithi071.stdout:696 read 0x876f0d4 thru 0x8771862 (0x278f bytes) 2024-09-14T05:29:36.479 INFO:teuthology.orchestra.run.smithi071.stdout:698 punch from 0xc20b9ea to 0xc210934, (0x4f4a bytes) 2024-09-14T05:29:36.479 INFO:teuthology.orchestra.run.smithi071.stdout:699 write 0x5ee4dcd thru 0x5eed546 (0x877a bytes) 2024-09-14T05:29:36.479 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:36.479 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:29:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:29:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:29:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:1134 3.7 e30/24 3:e1ef7f87:::rbd_data.112dd25fb4c.000000000000005f:head v 30'40, mlcod=30'36)", 2024-09-14T05:29:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.166405+0000", 2024-09-14T05:29:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.3101827039999998, 2024-09-14T05:29:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061353650000000003, 2024-09-14T05:29:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166405+0000", 2024-09-14T05:29:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166405+0000", 2024-09-14T05:29:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166406+0000", 2024-09-14T05:29:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.85e-07 2024-09-14T05:29:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166413+0000", 2024-09-14T05:29:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1029999999999996e-06 2024-09-14T05:29:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166425+0000", 2024-09-14T05:29:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.258e-05 2024-09-14T05:29:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166433+0000", 2024-09-14T05:29:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2910000000000002e-06 2024-09-14T05:29:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166492+0000", 2024-09-14T05:29:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9107999999999997e-05 2024-09-14T05:29:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166513+0000", 2024-09-14T05:29:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.128e-05 2024-09-14T05:29:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.172500+0000", 2024-09-14T05:29:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059870519999999997 2024-09-14T05:29:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.172540+0000", 2024-09-14T05:29:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0166000000000001e-05 2024-09-14T05:29:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1205 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:29:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.2786978360000001, 2024-09-14T05:29:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0086364279999999998, 2024-09-14T05:29:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1205 2024-09-14T05:29:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:29:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:29:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197891+0000", 2024-09-14T05:29:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.364e-06 2024-09-14T05:29:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197896+0000", 2024-09-14T05:29:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3000000000000001e-06 2024-09-14T05:29:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197910+0000", 2024-09-14T05:29:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3339000000000001e-05 2024-09-14T05:29:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197919+0000", 2024-09-14T05:29:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8869999999999998e-06 2024-09-14T05:29:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197974+0000", 2024-09-14T05:29:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5720000000000002e-05 2024-09-14T05:29:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.198094+0000", 2024-09-14T05:29:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119739 2024-09-14T05:29:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206526+0000", 2024-09-14T05:29:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084320790000000003 2024-09-14T05:29:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1220 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:29:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.2699414720000002, 2024-09-14T05:29:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012822337, 2024-09-14T05:29:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1220 2024-09-14T05:29:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:29:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:29:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206647+0000", 2024-09-14T05:29:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4600000000000003e-07 2024-09-14T05:29:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206652+0000", 2024-09-14T05:29:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6619999999999996e-06 2024-09-14T05:29:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206662+0000", 2024-09-14T05:29:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.995e-06 2024-09-14T05:29:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206667+0000", 2024-09-14T05:29:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-14T05:29:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206706+0000", 2024-09-14T05:29:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9523000000000001e-05 2024-09-14T05:29:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206755+0000", 2024-09-14T05:29:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000002e-05 2024-09-14T05:29:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206877+0000", 2024-09-14T05:29:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121938 2024-09-14T05:29:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.207310+0000", 2024-09-14T05:29:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000432452 2024-09-14T05:29:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219364+0000", 2024-09-14T05:29:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012054481000000001 2024-09-14T05:29:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219425+0000", 2024-09-14T05:29:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0618000000000001e-05 2024-09-14T05:29:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219468+0000", 2024-09-14T05:29:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3739000000000003e-05 2024-09-14T05:29:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1243 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:29:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.2569762449999997, 2024-09-14T05:29:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078660580000000004, 2024-09-14T05:29:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1243 2024-09-14T05:29:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:29:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:29:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219612+0000", 2024-09-14T05:29:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.09e-07 2024-09-14T05:29:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219616+0000", 2024-09-14T05:29:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.216e-06 2024-09-14T05:29:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219630+0000", 2024-09-14T05:29:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3064999999999999e-05 2024-09-14T05:29:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219637+0000", 2024-09-14T05:29:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7409999999999992e-06 2024-09-14T05:29:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219697+0000", 2024-09-14T05:29:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.02e-05 2024-09-14T05:29:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219833+0000", 2024-09-14T05:29:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013595600000000001 2024-09-14T05:29:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227477+0000", 2024-09-14T05:29:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076439710000000003 2024-09-14T05:29:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1264 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:29:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.2489654520000002, 2024-09-14T05:29:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080867990000000004, 2024-09-14T05:29:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1264 2024-09-14T05:29:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:29:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:29:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227623+0000", 2024-09-14T05:29:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:29:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227627+0000", 2024-09-14T05:29:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.194e-06 2024-09-14T05:29:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227639+0000", 2024-09-14T05:29:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1296e-05 2024-09-14T05:29:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227648+0000", 2024-09-14T05:29:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2280000000000004e-06 2024-09-14T05:29:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227689+0000", 2024-09-14T05:29:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1329999999999999e-05 2024-09-14T05:29:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227741+0000", 2024-09-14T05:29:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1641000000000002e-05 2024-09-14T05:29:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:29:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227914+0000", 2024-09-14T05:29:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000173215 2024-09-14T05:29:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228474+0000", 2024-09-14T05:29:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055980199999999998 2024-09-14T05:29:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235608+0000", 2024-09-14T05:29:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0071343830000000002 2024-09-14T05:29:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235669+0000", 2024-09-14T05:29:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1068999999999994e-05 2024-09-14T05:29:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235709+0000", 2024-09-14T05:29:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9671e-05 2024-09-14T05:29:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1520 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:29:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.1539578329999998, 2024-09-14T05:29:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061963629999999999, 2024-09-14T05:29:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1520 2024-09-14T05:29:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:29:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:29:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322631+0000", 2024-09-14T05:29:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.54e-07 2024-09-14T05:29:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322635+0000", 2024-09-14T05:29:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0450000000000002e-06 2024-09-14T05:29:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322646+0000", 2024-09-14T05:29:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1919999999999999e-05 2024-09-14T05:29:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322654+0000", 2024-09-14T05:29:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3139999999999998e-06 2024-09-14T05:29:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322704+0000", 2024-09-14T05:29:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0420000000000002e-05 2024-09-14T05:29:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322794+0000", 2024-09-14T05:29:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0037000000000005e-05 2024-09-14T05:29:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328826+0000", 2024-09-14T05:29:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060318730000000001 2024-09-14T05:29:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1526 3.4 3:2695cbe4:::rbd_data.112dd25fb4c.0000000000000119:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:29:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.150424889, 2024-09-14T05:29:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0079537359999999994, 2024-09-14T05:29:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1526 2024-09-14T05:29:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:29:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:29:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:29:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3600000000000003e-07 2024-09-14T05:29:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326168+0000", 2024-09-14T05:29:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3460000000000004e-06 2024-09-14T05:29:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326178+0000", 2024-09-14T05:29:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0291e-05 2024-09-14T05:29:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326183+0000", 2024-09-14T05:29:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-14T05:29:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326230+0000", 2024-09-14T05:29:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6650999999999998e-05 2024-09-14T05:29:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326295+0000", 2024-09-14T05:29:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5158000000000002e-05 2024-09-14T05:29:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326408+0000", 2024-09-14T05:29:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113039 2024-09-14T05:29:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.331648+0000", 2024-09-14T05:29:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052404929999999997 2024-09-14T05:29:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334011+0000", 2024-09-14T05:29:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002362669 2024-09-14T05:29:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334064+0000", 2024-09-14T05:29:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3198999999999997e-05 2024-09-14T05:29:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334116+0000", 2024-09-14T05:29:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2133999999999999e-05 2024-09-14T05:29:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1529 3.1 3:809746f7:::rbd_data.112dd25fb4c.0000000000000118:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:29:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.1486603390000001, 2024-09-14T05:29:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061981060000000001, 2024-09-14T05:29:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1529 2024-09-14T05:29:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:29:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:29:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327928+0000", 2024-09-14T05:29:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0100000000000004e-07 2024-09-14T05:29:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327933+0000", 2024-09-14T05:29:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4689999999999999e-06 2024-09-14T05:29:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327944+0000", 2024-09-14T05:29:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1623999999999999e-05 2024-09-14T05:29:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327951+0000", 2024-09-14T05:29:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9759999999999998e-06 2024-09-14T05:29:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328003+0000", 2024-09-14T05:29:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1521999999999999e-05 2024-09-14T05:29:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328087+0000", 2024-09-14T05:29:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3988999999999998e-05 2024-09-14T05:29:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328304+0000", 2024-09-14T05:29:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000217254 2024-09-14T05:29:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.333363+0000", 2024-09-14T05:29:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050586499999999996 2024-09-14T05:29:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334026+0000", 2024-09-14T05:29:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066303499999999997 2024-09-14T05:29:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334080+0000", 2024-09-14T05:29:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4261000000000001e-05 2024-09-14T05:29:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334125+0000", 2024-09-14T05:29:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5525000000000002e-05 2024-09-14T05:29:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1531 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:29:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.147606723, 2024-09-14T05:29:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082114309999999999, 2024-09-14T05:29:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1531 2024-09-14T05:29:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:29:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:29:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328982+0000", 2024-09-14T05:29:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:29:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328985+0000", 2024-09-14T05:29:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7900000000000001e-06 2024-09-14T05:29:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328996+0000", 2024-09-14T05:29:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.028e-05 2024-09-14T05:29:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329001+0000", 2024-09-14T05:29:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0610000000000003e-06 2024-09-14T05:29:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329035+0000", 2024-09-14T05:29:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4366999999999998e-05 2024-09-14T05:29:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329082+0000", 2024-09-14T05:29:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6646e-05 2024-09-14T05:29:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329195+0000", 2024-09-14T05:29:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113279 2024-09-14T05:29:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334293+0000", 2024-09-14T05:29:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050980399999999999 2024-09-14T05:29:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337101+0000", 2024-09-14T05:29:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0028076170000000001 2024-09-14T05:29:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337154+0000", 2024-09-14T05:29:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3641999999999999e-05 2024-09-14T05:29:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337192+0000", 2024-09-14T05:29:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7932000000000002e-05 2024-09-14T05:29:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.540 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.540 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:2131 3.b e30/24 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head v 30'36, mlcod=30'34)", 2024-09-14T05:29:36.540 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:29:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.6842593680000002, 2024-09-14T05:29:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031871709999999998, 2024-09-14T05:29:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:29:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:29:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792331+0000", 2024-09-14T05:29:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6900000000000001e-06 2024-09-14T05:29:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797319+0000", 2024-09-14T05:29:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049876349999999998 2024-09-14T05:29:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797349+0000", 2024-09-14T05:29:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0046e-05 2024-09-14T05:29:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797359+0000", 2024-09-14T05:29:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0037999999999999e-05 2024-09-14T05:29:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797445+0000", 2024-09-14T05:29:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6132999999999997e-05 2024-09-14T05:29:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797472+0000", 2024-09-14T05:29:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7478999999999999e-05 2024-09-14T05:29:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824161+0000", 2024-09-14T05:29:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026688714999999998 2024-09-14T05:29:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824200+0000", 2024-09-14T05:29:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8974000000000003e-05 2024-09-14T05:29:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:2259 3.b e31/24 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head v 31'38, mlcod=30'35)", 2024-09-14T05:29:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.120881+0000", 2024-09-14T05:29:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.3557067400000005, 2024-09-14T05:29:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006003815, 2024-09-14T05:29:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.120881+0000", 2024-09-14T05:29:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.120881+0000", 2024-09-14T05:29:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.120881+0000", 2024-09-14T05:29:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.65e-07 2024-09-14T05:29:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.120887+0000", 2024-09-14T05:29:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3679999999999998e-06 2024-09-14T05:29:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.120905+0000", 2024-09-14T05:29:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7448999999999998e-05 2024-09-14T05:29:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.120911+0000", 2024-09-14T05:29:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1449999999999996e-06 2024-09-14T05:29:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.120997+0000", 2024-09-14T05:29:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5641000000000002e-05 2024-09-14T05:29:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.121020+0000", 2024-09-14T05:29:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3714999999999999e-05 2024-09-14T05:29:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.126839+0000", 2024-09-14T05:29:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058184109999999999 2024-09-14T05:29:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.126885+0000", 2024-09-14T05:29:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5821000000000001e-05 2024-09-14T05:29:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4259 3.4 3:25f21dc9:::rbd_data.112d3d611e54.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 294912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:29:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.0850867539999998, 2024-09-14T05:29:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010043553, 2024-09-14T05:29:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4259 2024-09-14T05:29:36.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.554 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:29:36.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:29:36.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391503+0000", 2024-09-14T05:29:36.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9099999999999999e-06 2024-09-14T05:29:36.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391821+0000", 2024-09-14T05:29:36.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031871800000000001 2024-09-14T05:29:36.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391844+0000", 2024-09-14T05:29:36.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2045e-05 2024-09-14T05:29:36.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391853+0000", 2024-09-14T05:29:36.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7249999999999992e-06 2024-09-14T05:29:36.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391917+0000", 2024-09-14T05:29:36.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3458000000000001e-05 2024-09-14T05:29:36.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392037+0000", 2024-09-14T05:29:36.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120449 2024-09-14T05:29:36.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:36.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392305+0000", 2024-09-14T05:29:36.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026739299999999999 2024-09-14T05:29:36.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:36.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.393609+0000", 2024-09-14T05:29:36.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013042450000000001 2024-09-14T05:29:36.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:36.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401430+0000", 2024-09-14T05:29:36.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078211670000000004 2024-09-14T05:29:36.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401499+0000", 2024-09-14T05:29:36.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9374000000000004e-05 2024-09-14T05:29:36.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401544+0000", 2024-09-14T05:29:36.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5068999999999998e-05 2024-09-14T05:29:36.559 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.559 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.560 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.560 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4452 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.560 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:29:36.560 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5942741460000001, 2024-09-14T05:29:36.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066061380000000001, 2024-09-14T05:29:36.560 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.560 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.560 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:36.560 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.561 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.561 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.561 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4452 2024-09-14T05:29:36.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.561 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:29:36.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:29:36.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882314+0000", 2024-09-14T05:29:36.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5899999999999998e-07 2024-09-14T05:29:36.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882315+0000", 2024-09-14T05:29:36.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6169999999999999e-06 2024-09-14T05:29:36.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882319+0000", 2024-09-14T05:29:36.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3629999999999998e-06 2024-09-14T05:29:36.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882324+0000", 2024-09-14T05:29:36.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1370000000000001e-06 2024-09-14T05:29:36.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883369+0000", 2024-09-14T05:29:36.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010448320000000001 2024-09-14T05:29:36.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883421+0000", 2024-09-14T05:29:36.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2401000000000002e-05 2024-09-14T05:29:36.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:36.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883638+0000", 2024-09-14T05:29:36.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021725600000000001 2024-09-14T05:29:36.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:36.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.884464+0000", 2024-09-14T05:29:36.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00082569800000000004 2024-09-14T05:29:36.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:36.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888860+0000", 2024-09-14T05:29:36.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0043963370000000002 2024-09-14T05:29:36.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888896+0000", 2024-09-14T05:29:36.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6022999999999997e-05 2024-09-14T05:29:36.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888920+0000", 2024-09-14T05:29:36.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3215000000000001e-05 2024-09-14T05:29:36.566 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.567 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.567 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.567 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4467 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.567 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:29:36.567 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5759056959999995, 2024-09-14T05:29:36.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0065700189999999999, 2024-09-14T05:29:36.567 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.567 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.567 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:36.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.568 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.568 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.568 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4467 2024-09-14T05:29:36.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.568 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:29:36.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:29:36.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:29:36.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-14T05:29:36.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900684+0000", 2024-09-14T05:29:36.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.629e-06 2024-09-14T05:29:36.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900688+0000", 2024-09-14T05:29:36.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8469999999999998e-06 2024-09-14T05:29:36.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900691+0000", 2024-09-14T05:29:36.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-14T05:29:36.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.901775+0000", 2024-09-14T05:29:36.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001084177 2024-09-14T05:29:36.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.901862+0000", 2024-09-14T05:29:36.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7167000000000001e-05 2024-09-14T05:29:36.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:36.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.902223+0000", 2024-09-14T05:29:36.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036068800000000001 2024-09-14T05:29:36.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:36.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.902843+0000", 2024-09-14T05:29:36.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000620489 2024-09-14T05:29:36.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:36.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907167+0000", 2024-09-14T05:29:36.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0043242649999999999 2024-09-14T05:29:36.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907217+0000", 2024-09-14T05:29:36.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9648000000000003e-05 2024-09-14T05:29:36.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907252+0000", 2024-09-14T05:29:36.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4780000000000002e-05 2024-09-14T05:29:36.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.574 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.575 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.575 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4554 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.575 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:29:36.575 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.492185943, 2024-09-14T05:29:36.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069998999999999999, 2024-09-14T05:29:36.575 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.575 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.575 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:36.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.576 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.576 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.576 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4554 2024-09-14T05:29:36.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.576 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:29:36.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:29:36.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:29:36.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9299999999999999e-07 2024-09-14T05:29:36.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984404+0000", 2024-09-14T05:29:36.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.855e-06 2024-09-14T05:29:36.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984407+0000", 2024-09-14T05:29:36.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-14T05:29:36.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984410+0000", 2024-09-14T05:29:36.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8159999999999998e-06 2024-09-14T05:29:36.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985511+0000", 2024-09-14T05:29:36.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00110053 2024-09-14T05:29:36.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985561+0000", 2024-09-14T05:29:36.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0389000000000002e-05 2024-09-14T05:29:36.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:36.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985772+0000", 2024-09-14T05:29:36.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000210571 2024-09-14T05:29:36.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:36.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.987075+0000", 2024-09-14T05:29:36.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001303128 2024-09-14T05:29:36.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:36.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991303+0000", 2024-09-14T05:29:36.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042283310000000001 2024-09-14T05:29:36.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991359+0000", 2024-09-14T05:29:36.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6335999999999997e-05 2024-09-14T05:29:36.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991402+0000", 2024-09-14T05:29:36.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2074999999999999e-05 2024-09-14T05:29:36.582 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.582 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.582 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.582 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4711 3.4 3:226df663:::rbd_data.112d3d611e54.000000000000002d:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:29:36.582 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:29:36.582 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.3095147870000003, 2024-09-14T05:29:36.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011304193000000001, 2024-09-14T05:29:36.583 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.583 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.583 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:36.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.583 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4711 2024-09-14T05:29:36.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.583 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:29:36.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:29:36.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167074+0000", 2024-09-14T05:29:36.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0999999999999997e-07 2024-09-14T05:29:36.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167078+0000", 2024-09-14T05:29:36.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.878e-06 2024-09-14T05:29:36.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167087+0000", 2024-09-14T05:29:36.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6940000000000007e-06 2024-09-14T05:29:36.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167093+0000", 2024-09-14T05:29:36.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7740000000000001e-06 2024-09-14T05:29:36.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167128+0000", 2024-09-14T05:29:36.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5231999999999997e-05 2024-09-14T05:29:36.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167207+0000", 2024-09-14T05:29:36.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9036999999999996e-05 2024-09-14T05:29:36.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:36.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.169140+0000", 2024-09-14T05:29:36.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001932897 2024-09-14T05:29:36.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:36.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178258+0000", 2024-09-14T05:29:36.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091180819999999996 2024-09-14T05:29:36.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:36.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178314+0000", 2024-09-14T05:29:36.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5571e-05 2024-09-14T05:29:36.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178348+0000", 2024-09-14T05:29:36.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3747000000000001e-05 2024-09-14T05:29:36.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178377+0000", 2024-09-14T05:29:36.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9471e-05 2024-09-14T05:29:36.590 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.590 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.590 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.590 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4721 3.1 3:86302a6e:::rbd_data.112d3d611e54.0000000000000011:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:29:36.590 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:29:36.590 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.2775293010000004, 2024-09-14T05:29:36.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066831750000000004, 2024-09-14T05:29:36.590 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.590 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.590 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:36.591 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.591 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.591 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.591 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4721 2024-09-14T05:29:36.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.591 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:29:36.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:29:36.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199059+0000", 2024-09-14T05:29:36.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.06e-07 2024-09-14T05:29:36.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199062+0000", 2024-09-14T05:29:36.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2279999999999998e-06 2024-09-14T05:29:36.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199072+0000", 2024-09-14T05:29:36.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5999999999999996e-06 2024-09-14T05:29:36.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199078+0000", 2024-09-14T05:29:36.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9109999999999998e-06 2024-09-14T05:29:36.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199123+0000", 2024-09-14T05:29:36.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5278000000000003e-05 2024-09-14T05:29:36.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199202+0000", 2024-09-14T05:29:36.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8521000000000003e-05 2024-09-14T05:29:36.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:36.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199271+0000", 2024-09-14T05:29:36.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9857999999999994e-05 2024-09-14T05:29:36.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:36.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199941+0000", 2024-09-14T05:29:36.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066984599999999998 2024-09-14T05:29:36.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:36.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205653+0000", 2024-09-14T05:29:36.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057117319999999997 2024-09-14T05:29:36.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205701+0000", 2024-09-14T05:29:36.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8254000000000003e-05 2024-09-14T05:29:36.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205741+0000", 2024-09-14T05:29:36.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0241000000000003e-05 2024-09-14T05:29:36.597 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.597 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.597 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.597 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4819 3.12 3:4e8efced:::rbd_data.112d3d611e54.000000000000000e:head [stat,truncate 1114112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:29:36.597 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:29:36.597 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.1922151420000002, 2024-09-14T05:29:36.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062422770000000001, 2024-09-14T05:29:36.598 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.598 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.598 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:36.598 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.598 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.598 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.598 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4819 2024-09-14T05:29:36.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.598 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:29:36.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:29:36.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284373+0000", 2024-09-14T05:29:36.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3399999999999998e-07 2024-09-14T05:29:36.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284377+0000", 2024-09-14T05:29:36.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8539999999999999e-06 2024-09-14T05:29:36.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284388+0000", 2024-09-14T05:29:36.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1328999999999999e-05 2024-09-14T05:29:36.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284395+0000", 2024-09-14T05:29:36.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9759999999999998e-06 2024-09-14T05:29:36.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284723+0000", 2024-09-14T05:29:36.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032775799999999999 2024-09-14T05:29:36.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284801+0000", 2024-09-14T05:29:36.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7562e-05 2024-09-14T05:29:36.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:29:36.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284933+0000", 2024-09-14T05:29:36.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000131989 2024-09-14T05:29:36.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:36.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.287450+0000", 2024-09-14T05:29:36.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025177699999999999 2024-09-14T05:29:36.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:36.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290531+0000", 2024-09-14T05:29:36.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0030809959999999999 2024-09-14T05:29:36.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290588+0000", 2024-09-14T05:29:36.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6177000000000001e-05 2024-09-14T05:29:36.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290615+0000", 2024-09-14T05:29:36.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7132e-05 2024-09-14T05:29:36.604 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.604 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.604 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.604 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:7067 3.5 e35/24 3:a65d0cff:::rbd_data.112df7f2fe0a.0000000000000028:head v 35'89, mlcod=34'86)", 2024-09-14T05:29:36.604 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:29:36.604 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.333821871, 2024-09-14T05:29:36.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0064513080000000002, 2024-09-14T05:29:36.604 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.604 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.604 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:36.605 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:29:36.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:29:36.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:29:36.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:29:36.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142771+0000", 2024-09-14T05:29:36.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8949999999999997e-06 2024-09-14T05:29:36.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142779+0000", 2024-09-14T05:29:36.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8699999999999992e-06 2024-09-14T05:29:36.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142785+0000", 2024-09-14T05:29:36.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4899999999999997e-06 2024-09-14T05:29:36.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142843+0000", 2024-09-14T05:29:36.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7377000000000003e-05 2024-09-14T05:29:36.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142871+0000", 2024-09-14T05:29:36.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8305000000000001e-05 2024-09-14T05:29:36.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149177+0000", 2024-09-14T05:29:36.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063058639999999996 2024-09-14T05:29:36.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149217+0000", 2024-09-14T05:29:36.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0080999999999998e-05 2024-09-14T05:29:36.609 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.609 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.609 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.609 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:7066 3.15 e35/24 3:abadb03e:::rbd_data.112df7f2fe0a.0000000000000027:head v 35'25, mlcod=34'22)", 2024-09-14T05:29:36.609 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:35.143204+0000", 2024-09-14T05:29:36.609 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.3333840559999999, 2024-09-14T05:29:36.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060484919999999999, 2024-09-14T05:29:36.610 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.610 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.610 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:36.610 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.143204+0000", 2024-09-14T05:29:36.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.143204+0000", 2024-09-14T05:29:36.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.143204+0000", 2024-09-14T05:29:36.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0699999999999998e-07 2024-09-14T05:29:36.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.143208+0000", 2024-09-14T05:29:36.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5990000000000004e-06 2024-09-14T05:29:36.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.143247+0000", 2024-09-14T05:29:36.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8265e-05 2024-09-14T05:29:36.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.143255+0000", 2024-09-14T05:29:36.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3860000000000007e-06 2024-09-14T05:29:36.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.143455+0000", 2024-09-14T05:29:36.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019977100000000001 2024-09-14T05:29:36.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.143477+0000", 2024-09-14T05:29:36.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2104e-05 2024-09-14T05:29:36.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149219+0000", 2024-09-14T05:29:36.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057419180000000004 2024-09-14T05:29:36.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149252+0000", 2024-09-14T05:29:36.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3142000000000002e-05 2024-09-14T05:29:36.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.614 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.614 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.614 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:36.616 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:29:36.618 INFO:teuthology.orchestra.run.smithi071.stdout:700 write 0x76b6395 thru 0x76c4a16 (0xe682 bytes) 2024-09-14T05:29:36.618 INFO:teuthology.orchestra.run.smithi071.stdout:701 write 0xb9433bc thru 0xb94f2c9 (0xbf0e bytes) 2024-09-14T05:29:36.619 INFO:teuthology.orchestra.run.smithi071.stdout:702 punch from 0x62ad5a4 to 0x62bbc65, (0xe6c1 bytes) 2024-09-14T05:29:36.619 INFO:teuthology.orchestra.run.smithi071.stdout:703 trunc from 0xea3623a to 0x9941c4 2024-09-14T05:29:36.619 INFO:teuthology.orchestra.run.smithi071.stdout:704 read 0x49b3d6 thru 0x4a80a2 (0xcccd bytes) 2024-09-14T05:29:36.619 INFO:teuthology.orchestra.run.smithi071.stdout:705 read 0x6eb149 thru 0x6fa5e3 (0xf49b bytes) 2024-09-14T05:29:36.619 INFO:teuthology.orchestra.run.smithi071.stdout:706 write 0x3b2eeab thru 0x3b3802b (0x9181 bytes) 2024-09-14T05:29:36.619 INFO:teuthology.orchestra.run.smithi071.stdout:707 write 0x5a9c21d thru 0x5aa5bea (0x99ce bytes) 2024-09-14T05:29:36.619 INFO:teuthology.orchestra.run.smithi071.stdout:709 read 0x4ea43cc thru 0x4ea5a8a (0x16bf bytes) 2024-09-14T05:29:36.619 INFO:teuthology.orchestra.run.smithi071.stdout:710 read 0x57fd580 thru 0x58097e5 (0xc266 bytes) 2024-09-14T05:29:36.619 INFO:teuthology.orchestra.run.smithi071.stdout:711 punch from 0x41ba35b to 0x41c18b9, (0x755e bytes) 2024-09-14T05:29:36.619 INFO:teuthology.orchestra.run.smithi071.stdout:712 read 0x10b789a thru 0x10bc2ed (0x4a54 bytes) 2024-09-14T05:29:36.619 INFO:teuthology.orchestra.run.smithi071.stdout:713 writesame 0x664a9e1 thru 0x665147b data_size 0xb(0x6a9b bytes) 2024-09-14T05:29:36.619 INFO:teuthology.orchestra.run.smithi071.stdout:714 read 0x59d479 thru 0x5a8041 (0xabc9 bytes) 2024-09-14T05:29:36.620 INFO:teuthology.orchestra.run.smithi071.stdout:716 trunc from 0x665147c to 0x6e617f3 2024-09-14T05:29:36.620 INFO:teuthology.orchestra.run.smithi071.stdout:717 punch from 0x5a643f7 to 0x5a6a2eb, (0x5ef4 bytes) 2024-09-14T05:29:36.620 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:36.620 INFO:teuthology.orchestra.run.smithi071.stdout:719 writesame 0x7c2698d thru 0x7c30bef data_size 0x5d(0xa263 bytes) 2024-09-14T05:29:36.620 INFO:teuthology.orchestra.run.smithi071.stdout:721 trunc from 0x7c30bf0 to 0x8d35e5b 2024-09-14T05:29:36.620 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:36.620 INFO:teuthology.orchestra.run.smithi071.stdout:723 writesame 0x792c3e8 thru 0x79330db data_size 0x26(0x6cf4 bytes) 2024-09-14T05:29:36.620 INFO:teuthology.orchestra.run.smithi071.stdout:724 punch from 0x626c373 to 0x6272dd6, (0x6a63 bytes) 2024-09-14T05:29:36.620 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:36.620 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:36.620 INFO:teuthology.orchestra.run.smithi071.stdout:728 read 0x2561d53 thru 0x25712e1 (0xf58f bytes) 2024-09-14T05:29:36.620 INFO:teuthology.orchestra.run.smithi071.stdout:730 trunc from 0x8d35e5b to 0x9fa7c8d 2024-09-14T05:29:36.621 INFO:teuthology.orchestra.run.smithi071.stdout:731 read 0x44025f1 thru 0x440cc98 (0xa6a8 bytes) 2024-09-14T05:29:36.621 INFO:teuthology.orchestra.run.smithi071.stdout:732 clone 9 order 24 su 1048576 sc 13 2024-09-14T05:29:36.621 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:36.615+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:36.716 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:36.716+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:36.723 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:36.723 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:36.723 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:29:36.723 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:36.732 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:29:36.815 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:29:36.817 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:36.817+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:36.835 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:36.835 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:36.835 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:29:36.835 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:29:36.835 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:36.843 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:29:36.919 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:36.918+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:36.933 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:29:36.933 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:29:36.948 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:36.948 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:29:36.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:29:36.948 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:29:36.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.948 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2131 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:36.949 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:29:36.949 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.1721698619999996, 2024-09-14T05:29:36.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049526593000000001, 2024-09-14T05:29:36.949 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.949 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.949 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:36.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.949 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2131 2024-09-14T05:29:36.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.949 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:29:36.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:29:36.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774867+0000", 2024-09-14T05:29:36.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4430000000000002e-06 2024-09-14T05:29:36.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790357+0000", 2024-09-14T05:29:36.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015489466 2024-09-14T05:29:36.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790383+0000", 2024-09-14T05:29:36.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6247000000000001e-05 2024-09-14T05:29:36.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790394+0000", 2024-09-14T05:29:36.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0728e-05 2024-09-14T05:29:36.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790471+0000", 2024-09-14T05:29:36.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7539000000000004e-05 2024-09-14T05:29:36.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790639+0000", 2024-09-14T05:29:36.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016787700000000001 2024-09-14T05:29:36.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-14T05:29:36.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790998+0000", 2024-09-14T05:29:36.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035931299999999999 2024-09-14T05:29:36.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:36.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.823529+0000", 2024-09-14T05:29:36.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032530798 2024-09-14T05:29:36.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:36.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824324+0000", 2024-09-14T05:29:36.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00079442399999999995 2024-09-14T05:29:36.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:36.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824362+0000", 2024-09-14T05:29:36.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8398000000000001e-05 2024-09-14T05:29:36.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824391+0000", 2024-09-14T05:29:36.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.936e-05 2024-09-14T05:29:36.954 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.954 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.954 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.955 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3892 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7602176~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.955 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:29:36.955 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.5905918640000003, 2024-09-14T05:29:36.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047082951999999997, 2024-09-14T05:29:36.955 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.955 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.955 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.955 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.956 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3892 2024-09-14T05:29:36.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.956 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:29:36.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:29:36.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:29:36.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0199999999999998e-07 2024-09-14T05:29:36.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356454+0000", 2024-09-14T05:29:36.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0414999999999999e-05 2024-09-14T05:29:36.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356463+0000", 2024-09-14T05:29:36.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6739999999999992e-06 2024-09-14T05:29:36.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356468+0000", 2024-09-14T05:29:36.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5680000000000001e-06 2024-09-14T05:29:36.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403143+0000", 2024-09-14T05:29:36.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046675051000000002 2024-09-14T05:29:36.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403206+0000", 2024-09-14T05:29:36.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3127999999999996e-05 2024-09-14T05:29:36.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403526+0000", 2024-09-14T05:29:36.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031981400000000002 2024-09-14T05:29:36.959 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.959 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.959 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.960 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3893 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8093696~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.960 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:29:36.960 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.5905470269999995, 2024-09-14T05:29:36.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047516305000000002, 2024-09-14T05:29:36.960 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.960 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.960 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.960 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3893 2024-09-14T05:29:36.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.961 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:29:36.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:29:36.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:29:36.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6899999999999997e-07 2024-09-14T05:29:36.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356491+0000", 2024-09-14T05:29:36.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8109999999999999e-06 2024-09-14T05:29:36.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356498+0000", 2024-09-14T05:29:36.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5320000000000003e-06 2024-09-14T05:29:36.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356502+0000", 2024-09-14T05:29:36.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000002e-06 2024-09-14T05:29:36.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403579+0000", 2024-09-14T05:29:36.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047076706000000003 2024-09-14T05:29:36.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403664+0000", 2024-09-14T05:29:36.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5240000000000004e-05 2024-09-14T05:29:36.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404004+0000", 2024-09-14T05:29:36.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033974500000000002 2024-09-14T05:29:36.964 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.964 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.964 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.964 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3993 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 262144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.964 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:29:36.964 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.590525156, 2024-09-14T05:29:36.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048088476999999998, 2024-09-14T05:29:36.964 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.965 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.965 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.965 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.965 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.965 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.965 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3993 2024-09-14T05:29:36.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.965 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:29:36.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:29:36.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:29:36.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.46e-07 2024-09-14T05:29:36.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356512+0000", 2024-09-14T05:29:36.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.827e-06 2024-09-14T05:29:36.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356518+0000", 2024-09-14T05:29:36.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7780000000000004e-06 2024-09-14T05:29:36.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356522+0000", 2024-09-14T05:29:36.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.865e-06 2024-09-14T05:29:36.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404036+0000", 2024-09-14T05:29:36.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047513725999999999 2024-09-14T05:29:36.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404135+0000", 2024-09-14T05:29:36.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9352999999999996e-05 2024-09-14T05:29:36.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404598+0000", 2024-09-14T05:29:36.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046258200000000001 2024-09-14T05:29:36.968 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.968 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.969 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.969 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3994 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.969 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:29:36.969 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.5904831440000002, 2024-09-14T05:29:36.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048274625000000002, 2024-09-14T05:29:36.969 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.969 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.969 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.970 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.970 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3994 2024-09-14T05:29:36.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.970 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:29:36.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:29:36.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:29:36.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3799999999999997e-07 2024-09-14T05:29:36.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356554+0000", 2024-09-14T05:29:36.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2259999999999999e-06 2024-09-14T05:29:36.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356560+0000", 2024-09-14T05:29:36.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-14T05:29:36.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356566+0000", 2024-09-14T05:29:36.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3909999999999997e-06 2024-09-14T05:29:36.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404632+0000", 2024-09-14T05:29:36.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048066154 2024-09-14T05:29:36.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404719+0000", 2024-09-14T05:29:36.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7440000000000003e-05 2024-09-14T05:29:36.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404826+0000", 2024-09-14T05:29:36.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106813 2024-09-14T05:29:36.973 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.973 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.973 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.974 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3995 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.974 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:29:36.974 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.5904580050000003, 2024-09-14T05:29:36.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048828209999999997, 2024-09-14T05:29:36.974 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.974 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.974 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.974 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.974 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.974 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.974 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3995 2024-09-14T05:29:36.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.974 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:29:36.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:29:36.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:29:36.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6599999999999996e-07 2024-09-14T05:29:36.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356579+0000", 2024-09-14T05:29:36.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0719999999999998e-06 2024-09-14T05:29:36.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356585+0000", 2024-09-14T05:29:36.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.959e-06 2024-09-14T05:29:36.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356590+0000", 2024-09-14T05:29:36.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.464e-06 2024-09-14T05:29:36.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404875+0000", 2024-09-14T05:29:36.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048285649999999999 2024-09-14T05:29:36.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404985+0000", 2024-09-14T05:29:36.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109417 2024-09-14T05:29:36.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405405+0000", 2024-09-14T05:29:36.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042008200000000001 2024-09-14T05:29:36.978 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.978 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.978 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.978 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3996 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.978 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:29:36.979 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.5901491219999997, 2024-09-14T05:29:36.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049061595, 2024-09-14T05:29:36.979 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.979 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.979 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.979 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.979 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.979 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.979 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3996 2024-09-14T05:29:36.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.979 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:29:36.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:29:36.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:29:36.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.61e-07 2024-09-14T05:29:36.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356892+0000", 2024-09-14T05:29:36.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9050000000000002e-06 2024-09-14T05:29:36.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356901+0000", 2024-09-14T05:29:36.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4149999999999999e-06 2024-09-14T05:29:36.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356907+0000", 2024-09-14T05:29:36.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0140000000000004e-06 2024-09-14T05:29:36.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405448+0000", 2024-09-14T05:29:36.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048541647 2024-09-14T05:29:36.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405542+0000", 2024-09-14T05:29:36.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3639000000000003e-05 2024-09-14T05:29:36.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405947+0000", 2024-09-14T05:29:36.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040521399999999999 2024-09-14T05:29:36.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.983 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.983 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3997 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2228224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.983 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:29:36.983 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.5899351399999997, 2024-09-14T05:29:36.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049308774999999999, 2024-09-14T05:29:36.983 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.983 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.983 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.983 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.984 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3997 2024-09-14T05:29:36.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.984 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:29:36.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:29:36.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:29:36.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6499999999999999e-07 2024-09-14T05:29:36.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357104+0000", 2024-09-14T05:29:36.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4989999999999999e-06 2024-09-14T05:29:36.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357112+0000", 2024-09-14T05:29:36.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4279999999999996e-06 2024-09-14T05:29:36.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357117+0000", 2024-09-14T05:29:36.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6190000000000002e-06 2024-09-14T05:29:36.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405989+0000", 2024-09-14T05:29:36.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048872080999999998 2024-09-14T05:29:36.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406074+0000", 2024-09-14T05:29:36.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4676e-05 2024-09-14T05:29:36.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406408+0000", 2024-09-14T05:29:36.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033480700000000001 2024-09-14T05:29:36.987 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.987 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.987 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.988 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3998 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.988 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:29:36.988 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.5896137110000002, 2024-09-14T05:29:36.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049462314, 2024-09-14T05:29:36.988 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.988 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.988 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.988 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.988 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.988 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.988 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3998 2024-09-14T05:29:36.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.989 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:29:36.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:29:36.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357422+0000", 2024-09-14T05:29:36.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3099999999999996e-07 2024-09-14T05:29:36.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357429+0000", 2024-09-14T05:29:36.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2880000000000004e-06 2024-09-14T05:29:36.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357439+0000", 2024-09-14T05:29:36.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.302e-06 2024-09-14T05:29:36.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357445+0000", 2024-09-14T05:29:36.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4269999999999996e-06 2024-09-14T05:29:36.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406448+0000", 2024-09-14T05:29:36.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049003096000000003 2024-09-14T05:29:36.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406537+0000", 2024-09-14T05:29:36.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9135999999999995e-05 2024-09-14T05:29:36.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406883+0000", 2024-09-14T05:29:36.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000346134 2024-09-14T05:29:36.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.992 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.993 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3999 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.993 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:29:36.993 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.5895433539999999, 2024-09-14T05:29:36.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049797556, 2024-09-14T05:29:36.993 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.993 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.993 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.993 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3999 2024-09-14T05:29:36.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.993 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:29:36.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:29:36.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357492+0000", 2024-09-14T05:29:36.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8299999999999996e-07 2024-09-14T05:29:36.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357495+0000", 2024-09-14T05:29:36.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0450000000000001e-06 2024-09-14T05:29:36.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:36.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357503+0000", 2024-09-14T05:29:36.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9689999999999994e-06 2024-09-14T05:29:36.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:36.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357510+0000", 2024-09-14T05:29:36.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-14T05:29:36.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:36.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406914+0000", 2024-09-14T05:29:36.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049404293000000002 2024-09-14T05:29:36.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:36.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406965+0000", 2024-09-14T05:29:36.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0470999999999998e-05 2024-09-14T05:29:36.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:36.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407289+0000", 2024-09-14T05:29:36.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032434899999999998 2024-09-14T05:29:36.997 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.997 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:36.997 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:36.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.997 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4000 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3702784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:36.997 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:29:36.997 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.5891970090000003, 2024-09-14T05:29:36.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049936728999999999, 2024-09-14T05:29:36.997 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:36.997 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:36.997 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:36.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:36.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:36.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:36.998 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4000 2024-09-14T05:29:36.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.998 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:36.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:36.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:29:36.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:36.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:29:36.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:36.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:36.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357839+0000", 2024-09-14T05:29:36.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-14T05:29:36.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:36.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:36.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357842+0000", 2024-09-14T05:29:36.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4850000000000001e-06 2024-09-14T05:29:36.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:36.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:37.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357851+0000", 2024-09-14T05:29:37.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1819999999999995e-06 2024-09-14T05:29:37.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:37.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357859+0000", 2024-09-14T05:29:37.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8129999999999996e-06 2024-09-14T05:29:37.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:37.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407337+0000", 2024-09-14T05:29:37.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049478084999999998 2024-09-14T05:29:37.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:37.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407442+0000", 2024-09-14T05:29:37.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000105302 2024-09-14T05:29:37.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:37.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407774+0000", 2024-09-14T05:29:37.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033200100000000001 2024-09-14T05:29:37.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.001 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:37.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4001 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:29:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.5891016540000003, 2024-09-14T05:29:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050304205999999997, 2024-09-14T05:29:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4001 2024-09-14T05:29:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:29:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:29:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357934+0000", 2024-09-14T05:29:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000004e-07 2024-09-14T05:29:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357937+0000", 2024-09-14T05:29:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0869999999999998e-06 2024-09-14T05:29:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357946+0000", 2024-09-14T05:29:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1439999999999992e-06 2024-09-14T05:29:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357952+0000", 2024-09-14T05:29:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.609e-06 2024-09-14T05:29:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407820+0000", 2024-09-14T05:29:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049868580000000003 2024-09-14T05:29:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407909+0000", 2024-09-14T05:29:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8887000000000005e-05 2024-09-14T05:29:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408237+0000", 2024-09-14T05:29:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032824599999999997 2024-09-14T05:29:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4002 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:29:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.5888850589999999, 2024-09-14T05:29:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050539529999999999, 2024-09-14T05:29:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4002 2024-09-14T05:29:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:29:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:29:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:29:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5099999999999999e-07 2024-09-14T05:29:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358154+0000", 2024-09-14T05:29:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6799999999999999e-06 2024-09-14T05:29:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358165+0000", 2024-09-14T05:29:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0899e-05 2024-09-14T05:29:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358172+0000", 2024-09-14T05:29:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8689999999999998e-06 2024-09-14T05:29:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408281+0000", 2024-09-14T05:29:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050108699 2024-09-14T05:29:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408369+0000", 2024-09-14T05:29:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8542999999999996e-05 2024-09-14T05:29:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408689+0000", 2024-09-14T05:29:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032008900000000002 2024-09-14T05:29:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4003 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5177344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:29:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.5888452239999999, 2024-09-14T05:29:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050868759, 2024-09-14T05:29:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4003 2024-09-14T05:29:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:29:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:29:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:29:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-07 2024-09-14T05:29:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358193+0000", 2024-09-14T05:29:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.588e-06 2024-09-14T05:29:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358204+0000", 2024-09-14T05:29:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1525999999999999e-05 2024-09-14T05:29:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358213+0000", 2024-09-14T05:29:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5010000000000004e-06 2024-09-14T05:29:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408719+0000", 2024-09-14T05:29:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050506708999999997 2024-09-14T05:29:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408766+0000", 2024-09-14T05:29:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6579999999999998e-05 2024-09-14T05:29:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409058+0000", 2024-09-14T05:29:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000292365 2024-09-14T05:29:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4004 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5668864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:29:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.588422381, 2024-09-14T05:29:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050839848, 2024-09-14T05:29:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4004 2024-09-14T05:29:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:29:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:29:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358613+0000", 2024-09-14T05:29:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:29:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358618+0000", 2024-09-14T05:29:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6009999999999997e-06 2024-09-14T05:29:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358630+0000", 2024-09-14T05:29:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2122e-05 2024-09-14T05:29:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358638+0000", 2024-09-14T05:29:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3969999999999994e-06 2024-09-14T05:29:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409088+0000", 2024-09-14T05:29:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050449565000000002 2024-09-14T05:29:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409153+0000", 2024-09-14T05:29:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4728999999999996e-05 2024-09-14T05:29:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409452+0000", 2024-09-14T05:29:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029958900000000001 2024-09-14T05:29:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4005 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6160384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:29:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.588205447, 2024-09-14T05:29:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051054752000000002, 2024-09-14T05:29:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4005 2024-09-14T05:29:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:29:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:29:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358830+0000", 2024-09-14T05:29:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-14T05:29:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358833+0000", 2024-09-14T05:29:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9529999999999999e-06 2024-09-14T05:29:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358843+0000", 2024-09-14T05:29:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8069999999999994e-06 2024-09-14T05:29:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358849+0000", 2024-09-14T05:29:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1040000000000003e-06 2024-09-14T05:29:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409487+0000", 2024-09-14T05:29:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050638626999999999 2024-09-14T05:29:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409535+0000", 2024-09-14T05:29:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7525000000000003e-05 2024-09-14T05:29:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409884+0000", 2024-09-14T05:29:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034914800000000003 2024-09-14T05:29:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4006 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:29:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.5881683349999998, 2024-09-14T05:29:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051425323000000002, 2024-09-14T05:29:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4006 2024-09-14T05:29:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:29:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:29:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358867+0000", 2024-09-14T05:29:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2900000000000004e-07 2024-09-14T05:29:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358869+0000", 2024-09-14T05:29:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4210000000000002e-06 2024-09-14T05:29:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358876+0000", 2024-09-14T05:29:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4980000000000003e-06 2024-09-14T05:29:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358881+0000", 2024-09-14T05:29:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3090000000000003e-06 2024-09-14T05:29:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409915+0000", 2024-09-14T05:29:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051033348999999999 2024-09-14T05:29:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409964+0000", 2024-09-14T05:29:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9858000000000003e-05 2024-09-14T05:29:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410292+0000", 2024-09-14T05:29:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032735899999999999 2024-09-14T05:29:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4007 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:29:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.588145742, 2024-09-14T05:29:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05184076, 2024-09-14T05:29:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4007 2024-09-14T05:29:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:29:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:29:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:29:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:29:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358891+0000", 2024-09-14T05:29:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9819999999999998e-06 2024-09-14T05:29:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358897+0000", 2024-09-14T05:29:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9390000000000002e-06 2024-09-14T05:29:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358902+0000", 2024-09-14T05:29:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6770000000000003e-06 2024-09-14T05:29:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410335+0000", 2024-09-14T05:29:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051432991999999997 2024-09-14T05:29:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410416+0000", 2024-09-14T05:29:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1316999999999994e-05 2024-09-14T05:29:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410730+0000", 2024-09-14T05:29:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000313419 2024-09-14T05:29:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4008 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7634944~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:29:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.5881213809999997, 2024-09-14T05:29:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052208082000000003, 2024-09-14T05:29:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4008 2024-09-14T05:29:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:29:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:29:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358914+0000", 2024-09-14T05:29:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6100000000000001e-07 2024-09-14T05:29:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358916+0000", 2024-09-14T05:29:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.902e-06 2024-09-14T05:29:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358923+0000", 2024-09-14T05:29:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4780000000000002e-06 2024-09-14T05:29:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358929+0000", 2024-09-14T05:29:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0970000000000002e-06 2024-09-14T05:29:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410761+0000", 2024-09-14T05:29:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051831436000000002 2024-09-14T05:29:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410829+0000", 2024-09-14T05:29:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7874999999999996e-05 2024-09-14T05:29:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411121+0000", 2024-09-14T05:29:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029283299999999999 2024-09-14T05:29:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4009 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8126464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:29:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.5879741559999996, 2024-09-14T05:29:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052518169000000003, 2024-09-14T05:29:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4009 2024-09-14T05:29:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:29:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:29:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:29:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0200000000000002e-07 2024-09-14T05:29:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359066+0000", 2024-09-14T05:29:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7899999999999999e-06 2024-09-14T05:29:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359074+0000", 2024-09-14T05:29:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2579999999999995e-06 2024-09-14T05:29:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359082+0000", 2024-09-14T05:29:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.464e-06 2024-09-14T05:29:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411160+0000", 2024-09-14T05:29:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052078078 2024-09-14T05:29:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411244+0000", 2024-09-14T05:29:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4041000000000004e-05 2024-09-14T05:29:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:37.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411579+0000", 2024-09-14T05:29:37.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033493599999999998 2024-09-14T05:29:37.043 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.043 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:37.043 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.043 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:37.043 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:37.043 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:37.044 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:29:37.044 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:29:37.045 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:29:37.045 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:29:37.046 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:29:37.046 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:29:37.046 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:29:37.047 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:29:37.047 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:29:37.047 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:29:37.047 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:29:37.048 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:29:37.048 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:29:37.048 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:37.044+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:37.090 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T05:29:37.145 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:37.145+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:37.157 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone8 from 0x9fa7c8d (overlap 0x9941c4) to 0x5bf442 2024-09-14T05:29:37.164 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:37.164 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:37.176 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:29:37.246 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:37.245+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:37.249 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:29:37.250 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:29:37.258 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-14T05:29:37.301 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:37.301 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:37.316 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:29:37.358 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:37.357+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:37.447 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:37.447+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:37.465 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:37.466 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:37.475 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:29:37.509 INFO:teuthology.orchestra.run.smithi071.stdout:733 punch from 0x71509fc to 0x7154d54, (0x4358 bytes) 2024-09-14T05:29:37.513 INFO:teuthology.orchestra.run.smithi071.stdout:735 punch from 0x6185033 to 0x61897f9, (0x47c6 bytes) 2024-09-14T05:29:37.514 INFO:teuthology.orchestra.run.smithi071.stdout:736 read 0x8d7585a thru 0x8d7f9b6 (0xa15d bytes) 2024-09-14T05:29:37.514 INFO:teuthology.orchestra.run.smithi071.stdout:738 writesame 0x86a061e thru 0x86adf5e data_size 0x1(0xd941 bytes) 2024-09-14T05:29:37.544 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:37.545 INFO:teuthology.orchestra.run.smithi071.stdout:741 writesame 0x83946ef thru 0x8394bc7 data_size 0x1(0x4d9 bytes) 2024-09-14T05:29:37.549 INFO:teuthology.orchestra.run.smithi071.stdout:744 read 0x8a3c488 thru 0x8a4b530 (0xf0a9 bytes) 2024-09-14T05:29:37.549 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:37.548+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:37.559 INFO:teuthology.orchestra.run.smithi071.stdout:745 trunc from 0x9fa7c8d to 0x190f199 2024-09-14T05:29:37.593 INFO:teuthology.orchestra.run.smithi071.stdout:746 trunc from 0x190f199 to 0xe3ba508 2024-09-14T05:29:37.598 INFO:teuthology.orchestra.run.smithi071.stdout:747 read 0x9d92497 thru 0x9d94746 (0x22b0 bytes) 2024-09-14T05:29:37.598 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:37.598 INFO:teuthology.orchestra.run.smithi071.stdout:749 write 0x379b41f thru 0x37a80c7 (0xcca9 bytes) 2024-09-14T05:29:37.600 INFO:teuthology.orchestra.run.smithi071.stdout:751 read 0x177f373 thru 0x1785128 (0x5db6 bytes) 2024-09-14T05:29:37.600 INFO:teuthology.orchestra.run.smithi071.stdout:753 read 0xd118221 thru 0xd11f810 (0x75f0 bytes) 2024-09-14T05:29:37.600 INFO:teuthology.orchestra.run.smithi071.stdout:755 writesame 0x85dce52 thru 0x85ebb0c data_size 0x1(0xecbb bytes) 2024-09-14T05:29:37.624 INFO:teuthology.orchestra.run.smithi071.stdout:756 read 0x2f70f76 thru 0x2f710ff (0x18a bytes) 2024-09-14T05:29:37.627 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:37.627 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:37.635 INFO:teuthology.orchestra.run.smithi071.stdout:757 trunc from 0xe3ba508 to 0x4e2abf7 2024-09-14T05:29:37.641 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:29:37.644 INFO:teuthology.orchestra.run.smithi071.stdout:760 read 0x3e4b272 thru 0x3e5ab69 (0xf8f8 bytes) 2024-09-14T05:29:37.650 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:37.649+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:37.654 INFO:teuthology.orchestra.run.smithi071.stdout:762 writesame 0x9031634 thru 0x903293e data_size 0x19(0x130b bytes) 2024-09-14T05:29:37.655 INFO:teuthology.orchestra.run.smithi071.stdout:763 read 0x877d04c thru 0x87875fd (0xa5b2 bytes) 2024-09-14T05:29:37.655 INFO:teuthology.orchestra.run.smithi071.stdout:764 write 0x485a67c thru 0x485eb62 (0x44e7 bytes) 2024-09-14T05:29:37.657 INFO:teuthology.orchestra.run.smithi071.stdout:765 write 0x1fae967 thru 0x1fbbc25 (0xd2bf bytes) 2024-09-14T05:29:37.659 INFO:teuthology.orchestra.run.smithi071.stdout:766 write 0x3aedf4f thru 0x3af5661 (0x7713 bytes) 2024-09-14T05:29:37.672 INFO:teuthology.orchestra.run.smithi071.stdout:767 write 0xeba8985 thru 0xebaa8c0 (0x1f3c bytes) 2024-09-14T05:29:37.687 INFO:teuthology.orchestra.run.smithi071.stdout:769 trunc from 0xebaa8c1 to 0x5299d87 2024-09-14T05:29:37.701 INFO:teuthology.orchestra.run.smithi071.stdout:771 writesame 0x50fcbf4 thru 0x5108b36 data_size 0x13(0xbf43 bytes) 2024-09-14T05:29:37.714 INFO:teuthology.orchestra.run.smithi071.stdout:772 writesame 0x972b826 thru 0x9733e91 data_size 0x4(0x866c bytes) 2024-09-14T05:29:37.715 INFO:teuthology.orchestra.run.smithi071.stdout:773 punch from 0x67341e9 to 0x673b500, (0x7317 bytes) 2024-09-14T05:29:37.724 INFO:teuthology.orchestra.run.smithi071.stdout:774 trunc from 0x9733e92 to 0x227a3c5 2024-09-14T05:29:37.751 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:37.750+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:37.753 INFO:teuthology.orchestra.run.smithi071.stdout:775 writesame 0xd95197d thru 0xd95ef2c data_size 0x34(0xd5b0 bytes) 2024-09-14T05:29:37.753 INFO:teuthology.orchestra.run.smithi071.stdout:777 write 0x925dd66 thru 0x92622dd (0x4578 bytes) 2024-09-14T05:29:37.754 INFO:teuthology.orchestra.run.smithi071.stdout:778 read 0x6296535 thru 0x629e7db (0x82a7 bytes) 2024-09-14T05:29:37.754 INFO:teuthology.orchestra.run.smithi071.stdout:780 write 0x60a3bd3 thru 0x60b2d3a (0xf168 bytes) 2024-09-14T05:29:37.756 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:37.756 INFO:teuthology.orchestra.run.smithi071.stdout:782 read 0x978431a thru 0x978cd5b (0x8a42 bytes) 2024-09-14T05:29:37.756 INFO:teuthology.orchestra.run.smithi071.stdout:783 write 0x4deeb60 thru 0x4dfcd70 (0xe211 bytes) 2024-09-14T05:29:37.758 INFO:teuthology.orchestra.run.smithi071.stdout:784 writesame 0x7cb0fe7 thru 0x7cb74de data_size 0x24(0x64f8 bytes) 2024-09-14T05:29:37.759 INFO:teuthology.orchestra.run.smithi071.stdout:785 punch from 0x4bc03d8 to 0x4bcacfc, (0xa924 bytes) 2024-09-14T05:29:37.760 INFO:teuthology.orchestra.run.smithi071.stdout:786 write 0xbf4687d thru 0xbf4d979 (0x70fd bytes) 2024-09-14T05:29:37.760 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:37.763 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:37.763 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:37.765 INFO:teuthology.orchestra.run.smithi071.stdout:790 trunc from 0xd95ef2d to 0x973ac35 2024-09-14T05:29:37.775 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:29:37.784 INFO:teuthology.orchestra.run.smithi071.stdout:791 trunc from 0x973ac35 to 0x1425a02 2024-09-14T05:29:37.793 INFO:teuthology.orchestra.run.smithi071.stdout:793 trunc from 0x1425a02 to 0x1df5db0 2024-09-14T05:29:37.805 INFO:teuthology.orchestra.run.smithi071.stdout:794 writesame 0x2638e33 thru 0x263adc3 data_size 0x1(0x1f91 bytes) 2024-09-14T05:29:37.810 INFO:teuthology.orchestra.run.smithi071.stdout:796 trunc from 0x263adc4 to 0x567f4d3 2024-09-14T05:29:37.815 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:37.829 INFO:teuthology.orchestra.run.smithi071.stdout:798 write 0xc71f7d3 thru 0xc720581 (0xdaf bytes) 2024-09-14T05:29:37.830 INFO:teuthology.orchestra.run.smithi071.stdout:799 read 0x85cd523 thru 0x85d5995 (0x8473 bytes) 2024-09-14T05:29:37.830 INFO:teuthology.orchestra.run.smithi071.stdout:800 read 0x4a38ca1 thru 0x4a41c8d (0x8fed bytes) 2024-09-14T05:29:37.838 INFO:teuthology.orchestra.run.smithi071.stdout:802 write 0xd058c1c thru 0xd062b25 (0x9f0a bytes) 2024-09-14T05:29:37.848 INFO:teuthology.orchestra.run.smithi071.stdout:803 write 0xe24fe56 thru 0xe25b31f (0xb4ca bytes) 2024-09-14T05:29:37.852 INFO:teuthology.orchestra.run.smithi071.stdout:804 write 0x1675801 thru 0x167a35a (0x4b5a bytes) 2024-09-14T05:29:37.852 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:37.851+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:37.860 INFO:teuthology.orchestra.run.smithi071.stdout:805 write 0xecd26cc thru 0xece2025 (0xf95a bytes) 2024-09-14T05:29:37.864 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:37.864 INFO:teuthology.orchestra.run.smithi071.stdout:807 read 0x24ba768 thru 0x24bf976 (0x520f bytes) 2024-09-14T05:29:37.864 INFO:teuthology.orchestra.run.smithi071.stdout:811 write 0xb58d794 thru 0xb5992d8 (0xbb45 bytes) 2024-09-14T05:29:37.866 INFO:teuthology.orchestra.run.smithi071.stdout:812 write 0xb310860 thru 0xb3153d7 (0x4b78 bytes) 2024-09-14T05:29:37.867 INFO:teuthology.orchestra.run.smithi071.stdout:813 read 0x94b982b thru 0x94c67c0 (0xcf96 bytes) 2024-09-14T05:29:37.873 INFO:teuthology.orchestra.run.smithi071.stdout:815 trunc from 0xece2026 to 0xa23ee14 2024-09-14T05:29:37.889 INFO:teuthology.orchestra.run.smithi071.stdout:816 trunc from 0xa23ee14 to 0xa219ff0 2024-09-14T05:29:37.904 INFO:teuthology.orchestra.run.smithi071.stdout:817 write 0xdfb486f thru 0xdfc336d (0xeaff bytes) 2024-09-14T05:29:37.905 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:37.905 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:37.907 INFO:teuthology.orchestra.run.smithi071.stdout:818 write 0x897630a thru 0x897cc1d (0x6914 bytes) 2024-09-14T05:29:37.909 INFO:teuthology.orchestra.run.smithi071.stdout:822 write 0x6ba3c30 thru 0x6babbea (0x7fbb bytes) 2024-09-14T05:29:37.910 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:37.910 INFO:teuthology.orchestra.run.smithi071.stdout:826 write 0x97668a7 thru 0x976d59e (0x6cf8 bytes) 2024-09-14T05:29:37.911 INFO:teuthology.orchestra.run.smithi071.stdout:827 writesame 0x9724b77 thru 0x9734397 data_size 0x1(0xf821 bytes) 2024-09-14T05:29:37.915 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:29:37.953 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:37.952+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:37.953 INFO:teuthology.orchestra.run.smithi071.stdout:828 trunc from 0xdfc336e to 0x5bdccc1 2024-09-14T05:29:37.966 INFO:teuthology.orchestra.run.smithi071.stdout:830 writesame 0x260b0b5 thru 0x26185c7 data_size 0x1(0xd513 bytes) 2024-09-14T05:29:37.991 INFO:teuthology.orchestra.run.smithi071.stdout:831 read 0x31df983 thru 0x31e4124 (0x47a2 bytes) 2024-09-14T05:29:37.992 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:37.992 INFO:teuthology.orchestra.run.smithi071.stdout:833 punch from 0x27ed266 to 0x27f389a, (0x6634 bytes) 2024-09-14T05:29:38.005 INFO:teuthology.orchestra.run.smithi071.stdout:835 writesame 0xc6656b7 thru 0xc67153f data_size 0x47(0xbe89 bytes) 2024-09-14T05:29:38.009 INFO:teuthology.orchestra.run.smithi071.stdout:836 trunc from 0xc671540 to 0x9c96850 2024-09-14T05:29:38.026 INFO:teuthology.orchestra.run.smithi071.stdout:839 write 0xd2c11f7 thru 0xd2c8ca8 (0x7ab2 bytes) 2024-09-14T05:29:38.029 INFO:teuthology.orchestra.run.smithi071.stdout:840 read 0x3fbd6c3 thru 0x3fcd0aa (0xf9e8 bytes) 2024-09-14T05:29:38.029 INFO:teuthology.orchestra.run.smithi071.stdout:841 read 0xa2842d2 thru 0xa29242a (0xe159 bytes) 2024-09-14T05:29:38.029 INFO:teuthology.orchestra.run.smithi071.stdout:842 write 0xbae348b thru 0xbae8864 (0x53da bytes) 2024-09-14T05:29:38.031 INFO:teuthology.orchestra.run.smithi071.stdout:843 read 0x2d713f thru 0x2da2a8 (0x316a bytes) 2024-09-14T05:29:38.031 INFO:teuthology.orchestra.run.smithi071.stdout:844 writesame 0x8477c40 thru 0x8484704 data_size 0xd(0xcac5 bytes) 2024-09-14T05:29:38.039 INFO:teuthology.orchestra.run.smithi071.stdout:845 trunc from 0xd2c8ca9 to 0x8c2359d 2024-09-14T05:29:38.052 INFO:teuthology.orchestra.run.smithi071.stdout:846 punch from 0x78e30a8 to 0x78e810c, (0x5064 bytes) 2024-09-14T05:29:38.053 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:38.052+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:38.058 INFO:teuthology.orchestra.run.smithi071.stdout:847 trunc from 0x8c2359d to 0x4253fec 2024-09-14T05:29:38.069 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.071 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:38.071 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:38.075 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-14T05:29:38.086 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:29:38.086 INFO:teuthology.orchestra.run.smithi071.stdout:850 write 0x8daf3c4 thru 0x8db9308 (0x9f45 bytes) 2024-09-14T05:29:38.087 INFO:teuthology.orchestra.run.smithi071.stdout:851 write 0x4aaf023 thru 0x4abe4d3 (0xf4b1 bytes) 2024-09-14T05:29:38.090 INFO:teuthology.orchestra.run.smithi071.stdout:852 punch from 0x5628c3c to 0x562b8a2, (0x2c66 bytes) 2024-09-14T05:29:38.090 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.090 INFO:teuthology.orchestra.run.smithi071.stdout:856 read 0x2463139 thru 0x246f0b2 (0xbf7a bytes) 2024-09-14T05:29:38.090 INFO:teuthology.orchestra.run.smithi071.stdout:859 read 0x581c291 thru 0x5822542 (0x62b2 bytes) 2024-09-14T05:29:38.090 INFO:teuthology.orchestra.run.smithi071.stdout:860 punch from 0xb94cf3 to 0xb9fc33, (0xaf40 bytes) 2024-09-14T05:29:38.091 INFO:teuthology.orchestra.run.smithi071.stdout:861 write 0x74bf6fa thru 0x74c9632 (0x9f39 bytes) 2024-09-14T05:29:38.092 INFO:teuthology.orchestra.run.smithi071.stdout:862 write 0x79a47bd thru 0x79abcf4 (0x7538 bytes) 2024-09-14T05:29:38.094 INFO:teuthology.orchestra.run.smithi071.stdout:863 read 0xb78ef0 thru 0xb87e47 (0xef58 bytes) 2024-09-14T05:29:38.094 INFO:teuthology.orchestra.run.smithi071.stdout:865 write 0x5701c10 thru 0x570a537 (0x8928 bytes) 2024-09-14T05:29:38.096 INFO:teuthology.orchestra.run.smithi071.stdout:867 writesame 0x5cc7065 thru 0x5ccc5d8 data_size 0x6(0x5574 bytes) 2024-09-14T05:29:38.098 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.109 INFO:teuthology.orchestra.run.smithi071.stdout:869 write 0xd9abc6d thru 0xd9b0049 (0x43dd bytes) 2024-09-14T05:29:38.112 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.112 INFO:teuthology.orchestra.run.smithi071.stdout:871 write 0x53212d thru 0x53ae31 (0x8d05 bytes) 2024-09-14T05:29:38.113 INFO:teuthology.orchestra.run.smithi071.stdout:872 write 0x69b5116 thru 0x69bc7e0 (0x76cb bytes) 2024-09-14T05:29:38.115 INFO:teuthology.orchestra.run.smithi071.stdout:873 punch from 0x53ef953 to 0x53f4cfc, (0x53a9 bytes) 2024-09-14T05:29:38.115 INFO:teuthology.orchestra.run.smithi071.stdout:874 write 0x2df8089 thru 0x2dfe122 (0x609a bytes) 2024-09-14T05:29:38.127 INFO:teuthology.orchestra.run.smithi071.stdout:875 write 0xecfae0a thru 0xecfd1f8 (0x23ef bytes) 2024-09-14T05:29:38.130 INFO:teuthology.orchestra.run.smithi071.stdout:876 write 0x74ac55 thru 0x75103f (0x63eb bytes) 2024-09-14T05:29:38.131 INFO:teuthology.orchestra.run.smithi071.stdout:877 read 0xa2f582f thru 0xa2fe40d (0x8bdf bytes) 2024-09-14T05:29:38.132 INFO:teuthology.orchestra.run.smithi071.stdout:878 write 0xddef806 thru 0xddfe921 (0xf11c bytes) 2024-09-14T05:29:38.136 INFO:teuthology.orchestra.run.smithi071.stdout:879 read 0x431248a thru 0x432174f (0xf2c6 bytes) 2024-09-14T05:29:38.136 INFO:teuthology.orchestra.run.smithi071.stdout:880 write 0xcea87b6 thru 0xcea905d (0x8a8 bytes) 2024-09-14T05:29:38.138 INFO:teuthology.orchestra.run.smithi071.stdout:881 write 0xa14d9da thru 0xa15071e (0x2d45 bytes) 2024-09-14T05:29:38.139 INFO:teuthology.orchestra.run.smithi071.stdout:882 read 0x1c2b489 thru 0x1c3603b (0xabb3 bytes) 2024-09-14T05:29:38.143 INFO:teuthology.orchestra.run.smithi071.stdout:883 trunc from 0xecfd1f9 to 0xc3b871f 2024-09-14T05:29:38.153 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:38.152+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:38.159 INFO:teuthology.orchestra.run.smithi071.stdout:886 write 0x5aa673a thru 0x5ab59c1 (0xf288 bytes) 2024-09-14T05:29:38.161 INFO:teuthology.orchestra.run.smithi071.stdout:890 writesame 0x1593051 thru 0x159d960 data_size 0x28(0xa910 bytes) 2024-09-14T05:29:38.162 INFO:teuthology.orchestra.run.smithi071.stdout:891 read 0xac9e8fd thru 0xacaa516 (0xbc1a bytes) 2024-09-14T05:29:38.173 INFO:teuthology.orchestra.run.smithi071.stdout:892 trunc from 0xc3b871f to 0x3c5fb14 2024-09-14T05:29:38.197 INFO:teuthology.orchestra.run.smithi071.stdout:893 write 0x852eb42 thru 0x853d87b (0xed3a bytes) 2024-09-14T05:29:38.205 INFO:teuthology.orchestra.run.smithi071.stdout:894 trunc from 0x853d87c to 0xe2be67a 2024-09-14T05:29:38.207 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:38.207 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:38.211 INFO:teuthology.orchestra.run.smithi071.stdout:895 read 0x9d11794 thru 0x9d178dd (0x614a bytes) 2024-09-14T05:29:38.211 INFO:teuthology.orchestra.run.smithi071.stdout:897 read 0x42164 thru 0x4282d (0x6ca bytes) 2024-09-14T05:29:38.211 INFO:teuthology.orchestra.run.smithi071.stdout:898 read 0x51010cc thru 0x510fde4 (0xed19 bytes) 2024-09-14T05:29:38.212 INFO:teuthology.orchestra.run.smithi071.stdout:899 read 0x8694b6 thru 0x86b0f6 (0x1c41 bytes) 2024-09-14T05:29:38.212 INFO:teuthology.orchestra.run.smithi071.stdout:900 read 0x3777361 thru 0x37845bc (0xd25c bytes) 2024-09-14T05:29:38.213 INFO:teuthology.orchestra.run.smithi071.stdout:901 write 0x96cf6b1 thru 0x96deed2 (0xf822 bytes) 2024-09-14T05:29:38.214 INFO:teuthology.orchestra.run.smithi071.stdout:902 writesame 0x2a0df17 thru 0x2a1b75a data_size 0x2(0xd844 bytes) 2024-09-14T05:29:38.217 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:29:38.228 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.228 INFO:teuthology.orchestra.run.smithi071.stdout:906 read 0x45f77da thru 0x4601970 (0xa197 bytes) 2024-09-14T05:29:38.229 INFO:teuthology.orchestra.run.smithi071.stdout:907 punch from 0xc5701c1 to 0xc5788c7, (0x8706 bytes) 2024-09-14T05:29:38.229 INFO:teuthology.orchestra.run.smithi071.stdout:908 write 0x94ccc49 thru 0x94d1db6 (0x516e bytes) 2024-09-14T05:29:38.233 INFO:teuthology.orchestra.run.smithi071.stdout:909 trunc from 0xe2be67a to 0xb5fe812 2024-09-14T05:29:38.241 INFO:teuthology.orchestra.run.smithi071.stdout:910 punch from 0x98822b5 to 0x988b9e6, (0x9731 bytes) 2024-09-14T05:29:38.243 INFO:teuthology.orchestra.run.smithi071.stdout:911 trunc from 0xb5fe812 to 0x9fed954 2024-09-14T05:29:38.252 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.254 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:38.253+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:38.262 INFO:teuthology.orchestra.run.smithi071.stdout:916 write 0xe042960 thru 0xe04fbaa (0xd24b bytes) 2024-09-14T05:29:38.264 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.265 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.265 INFO:teuthology.orchestra.run.smithi071.stdout:919 write 0x8fb549e thru 0x8fb7d84 (0x28e7 bytes) 2024-09-14T05:29:38.266 INFO:teuthology.orchestra.run.smithi071.stdout:920 writesame 0x599f48d thru 0x59aa75e data_size 0x2f(0xb2d2 bytes) 2024-09-14T05:29:38.267 INFO:teuthology.orchestra.run.smithi071.stdout:922 writesame 0xdcc9ed7 thru 0xdcd0fec data_size 0x4b(0x7116 bytes) 2024-09-14T05:29:38.283 INFO:teuthology.orchestra.run.smithi071.stdout:923 trunc from 0xe04fbab to 0x34782c8 2024-09-14T05:29:38.298 INFO:teuthology.orchestra.run.smithi071.stdout:924 trunc from 0x34782c8 to 0xe74d7c 2024-09-14T05:29:38.324 INFO:teuthology.orchestra.run.smithi071.stdout:925 write 0x9141c2d thru 0x91437fa (0x1bce bytes) 2024-09-14T05:29:38.333 INFO:teuthology.orchestra.run.smithi071.stdout:926 write 0xaba25d2 thru 0xabb0117 (0xdb46 bytes) 2024-09-14T05:29:38.335 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.335 INFO:teuthology.orchestra.run.smithi071.stdout:929 punch from 0x8edd782 to 0x8eeab28, (0xd3a6 bytes) 2024-09-14T05:29:38.335 INFO:teuthology.orchestra.run.smithi071.stdout:930 writesame 0x4941ac2 thru 0x494e0c8 data_size 0x1(0xc607 bytes) 2024-09-14T05:29:38.346 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:38.347 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:38.355 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:38.354+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:38.356 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:29:38.365 INFO:teuthology.orchestra.run.smithi071.stdout:931 trunc from 0xabb0118 to 0x4a3956f 2024-09-14T05:29:38.388 INFO:teuthology.orchestra.run.smithi071.stdout:933 write 0xd1d9d3a thru 0xd1df633 (0x58fa bytes) 2024-09-14T05:29:38.391 INFO:teuthology.orchestra.run.smithi071.stdout:936 read 0x3c01e76 thru 0x3c0ad78 (0x8f03 bytes) 2024-09-14T05:29:38.392 INFO:teuthology.orchestra.run.smithi071.stdout:938 write 0x7d5615d thru 0x7d6035d (0xa201 bytes) 2024-09-14T05:29:38.401 INFO:teuthology.orchestra.run.smithi071.stdout:939 write 0xec45618 thru 0xec4c437 (0x6e20 bytes) 2024-09-14T05:29:38.403 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.410 INFO:teuthology.orchestra.run.smithi071.stdout:941 trunc from 0xec4c438 to 0x9af96cb 2024-09-14T05:29:38.422 INFO:teuthology.orchestra.run.smithi071.stdout:942 read 0x867ae89 thru 0x86812f8 (0x6470 bytes) 2024-09-14T05:29:38.422 INFO:teuthology.orchestra.run.smithi071.stdout:943 write 0x8e4f772 thru 0x8e57183 (0x7a12 bytes) 2024-09-14T05:29:38.429 INFO:teuthology.orchestra.run.smithi071.stdout:944 trunc from 0x9af96cb to 0x4cb944a 2024-09-14T05:29:38.437 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.437 INFO:teuthology.orchestra.run.smithi071.stdout:946 punch from 0x40a63be to 0x40a6d99, (0x9db bytes) 2024-09-14T05:29:38.437 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.437 INFO:teuthology.orchestra.run.smithi071.stdout:951 write 0x22c1cba thru 0x22c4f31 (0x3278 bytes) 2024-09-14T05:29:38.438 INFO:teuthology.orchestra.run.smithi071.stdout:954 writesame 0x3fba174 thru 0x3fc6c3f data_size 0x2(0xcacc bytes) 2024-09-14T05:29:38.453 INFO:teuthology.orchestra.run.smithi071.stdout:956 trunc from 0x4cb944a to 0x22465a2 2024-09-14T05:29:38.456 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:38.455+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:38.465 INFO:teuthology.orchestra.run.smithi071.stdout:957 trunc from 0x22465a2 to 0x5fb424e 2024-09-14T05:29:38.470 INFO:teuthology.orchestra.run.smithi071.stdout:959 read 0x5651fda thru 0x565f433 (0xd45a bytes) 2024-09-14T05:29:38.471 INFO:teuthology.orchestra.run.smithi071.stdout:961 read 0x4991ba6 thru 0x49977f8 (0x5c53 bytes) 2024-09-14T05:29:38.471 INFO:teuthology.orchestra.run.smithi071.stdout:962 read 0x4629d72 thru 0x462e067 (0x42f6 bytes) 2024-09-14T05:29:38.472 INFO:teuthology.orchestra.run.smithi071.stdout:965 write 0x55af352 thru 0x55bb5b4 (0xc263 bytes) 2024-09-14T05:29:38.473 INFO:teuthology.orchestra.run.smithi071.stdout:966 writesame 0x17bd754 thru 0x17be23a data_size 0x1(0xae7 bytes) 2024-09-14T05:29:38.487 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:38.487 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:38.490 INFO:teuthology.orchestra.run.smithi071.stdout:967 writesame 0xe05fdb6 thru 0xe06b11b data_size 0x1(0xb366 bytes) 2024-09-14T05:29:38.490 INFO:teuthology.orchestra.run.smithi071.stdout:968 writesame 0xac6156b thru 0xac68bee data_size 0x52(0x7684 bytes) 2024-09-14T05:29:38.492 INFO:teuthology.orchestra.run.smithi071.stdout:970 read 0xdfb1b8 thru 0xe0a449 (0xf292 bytes) 2024-09-14T05:29:38.498 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:29:38.505 INFO:teuthology.orchestra.run.smithi071.stdout:971 trunc from 0xe06b11c to 0x374170d 2024-09-14T05:29:38.512 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T05:29:38.512 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T05:29:38.513 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T05:29:38.513 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.016656 2024-09-14T05:29:38.513 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 0.016655863524579484 2024-09-14T05:29:38.519 INFO:teuthology.orchestra.run.smithi071.stdout:972 trunc from 0x374170d to 0x641508f 2024-09-14T05:29:38.526 INFO:teuthology.orchestra.run.smithi071.stdout:974 punch from 0x35875f4 to 0x3588fa2, (0x19ae bytes) 2024-09-14T05:29:38.526 INFO:teuthology.orchestra.run.smithi071.stdout:976 read 0x824ca5 thru 0x8305ce (0xb92a bytes) 2024-09-14T05:29:38.526 INFO:teuthology.orchestra.run.smithi071.stdout:977 read 0x173597f thru 0x17363c9 (0xa4b bytes) 2024-09-14T05:29:38.527 INFO:teuthology.orchestra.run.smithi071.stdout:978 punch from 0x1ee9598 to 0x1ef1e99, (0x8901 bytes) 2024-09-14T05:29:38.527 INFO:teuthology.orchestra.run.smithi071.stdout:979 read 0x16d389a thru 0x16e1c59 (0xe3c0 bytes) 2024-09-14T05:29:38.527 INFO:teuthology.orchestra.run.smithi071.stdout:981 punch from 0x60214d to 0x60d795, (0xb648 bytes) 2024-09-14T05:29:38.527 INFO:teuthology.orchestra.run.smithi071.stdout:982 read 0x4917f56 thru 0x4919e06 (0x1eb1 bytes) 2024-09-14T05:29:38.541 INFO:teuthology.orchestra.run.smithi071.stdout:983 writesame 0xd31d7fb thru 0xd322d5c data_size 0x3(0x5562 bytes) 2024-09-14T05:29:38.541 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.549 INFO:teuthology.orchestra.run.smithi071.stdout:985 write 0xebf945a thru 0xec087a3 (0xf34a bytes) 2024-09-14T05:29:38.552 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.552 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.553 INFO:teuthology.orchestra.run.smithi071.stdout:988 read 0x16a8ada thru 0x16ad771 (0x4c98 bytes) 2024-09-14T05:29:38.553 INFO:teuthology.orchestra.run.smithi071.stdout:989 read 0x584d42a thru 0x585348e (0x6065 bytes) 2024-09-14T05:29:38.553 INFO:teuthology.orchestra.run.smithi071.stdout:990 read 0x80da929 thru 0x80e8936 (0xe00e bytes) 2024-09-14T05:29:38.553 INFO:teuthology.orchestra.run.smithi071.stdout:991 write 0x70f5197 thru 0x70faf7b (0x5de5 bytes) 2024-09-14T05:29:38.555 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.555 INFO:teuthology.orchestra.run.smithi071.stdout:993 punch from 0x3c40431 to 0x3c46590, (0x615f bytes) 2024-09-14T05:29:38.555 INFO:teuthology.orchestra.run.smithi071.stdout:994 write 0x5c5ea9a thru 0x5c6b6d0 (0xcc37 bytes) 2024-09-14T05:29:38.557 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:38.556+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:38.557 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.558 INFO:teuthology.orchestra.run.smithi071.stdout:996 punch from 0x9e452e to 0x9f43c6, (0xfe98 bytes) 2024-09-14T05:29:38.558 INFO:teuthology.orchestra.run.smithi071.stdout:997 read 0xa1bf2bc thru 0xa1c5c78 (0x69bd bytes) 2024-09-14T05:29:38.558 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.558 INFO:teuthology.orchestra.run.smithi071.stdout:1000 write 0x625e3a8 thru 0x626149b (0x30f4 bytes) 2024-09-14T05:29:38.559 INFO:teuthology.orchestra.run.smithi071.stdout:1001 read 0xb8661a4 thru 0xb867bfe (0x1a5b bytes) 2024-09-14T05:29:38.559 INFO:teuthology.orchestra.run.smithi071.stdout:1002 read 0xc1bc29e thru 0xc1beac6 (0x2829 bytes) 2024-09-14T05:29:38.559 INFO:teuthology.orchestra.run.smithi071.stdout:1003 writesame 0xbdf3aaf thru 0xbdf6417 data_size 0x1(0x2969 bytes) 2024-09-14T05:29:38.565 INFO:teuthology.orchestra.run.smithi071.stdout:1004 read 0x94a9104 thru 0x94b4da8 (0xbca5 bytes) 2024-09-14T05:29:38.565 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.565 INFO:teuthology.orchestra.run.smithi071.stdout:1006 write 0xd772211 thru 0xd774af2 (0x28e2 bytes) 2024-09-14T05:29:38.569 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.569 INFO:teuthology.orchestra.run.smithi071.stdout:1010 write 0x471a3f1 thru 0x471d6c6 (0x32d6 bytes) 2024-09-14T05:29:38.575 INFO:teuthology.orchestra.run.smithi071.stdout:1011 write 0xec45ee5 thru 0xec54865 (0xe981 bytes) 2024-09-14T05:29:38.577 INFO:teuthology.orchestra.run.smithi071.stdout:1015 write 0xa182 thru 0x181b4 (0xe033 bytes) 2024-09-14T05:29:38.578 INFO:teuthology.orchestra.run.smithi071.stdout:1016 punch from 0x2a93592 to 0x2a9b030, (0x7a9e bytes) 2024-09-14T05:29:38.578 INFO:teuthology.orchestra.run.smithi071.stdout:1017 punch from 0xa9aa7aa to 0xa9b4966, (0xa1bc bytes) 2024-09-14T05:29:38.595 INFO:teuthology.orchestra.run.smithi071.stdout:1018 trunc from 0xec54866 to 0xbdc015 2024-09-14T05:29:38.625 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:38.626 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:38.627 INFO:teuthology.orchestra.run.smithi071.stdout:1019 trunc from 0xbdc015 to 0xc6eee0e 2024-09-14T05:29:38.634 INFO:teuthology.orchestra.run.smithi071.stdout:1020 read 0x29363d4 thru 0x293700a (0xc37 bytes) 2024-09-14T05:29:38.634 INFO:teuthology.orchestra.run.smithi071.stdout:1021 write 0x30199e1 thru 0x3026057 (0xc677 bytes) 2024-09-14T05:29:38.636 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:29:38.636 INFO:teuthology.orchestra.run.smithi071.stdout:1023 write 0x2b9bcb3 thru 0x2b9fd4c (0x409a bytes) 2024-09-14T05:29:38.637 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.637 INFO:teuthology.orchestra.run.smithi071.stdout:1029 write 0x46b0f33 thru 0x46b3bff (0x2ccd bytes) 2024-09-14T05:29:38.639 INFO:teuthology.orchestra.run.smithi071.stdout:1030 punch from 0xe51a35 to 0xe60e4a, (0xf415 bytes) 2024-09-14T05:29:38.640 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.640 INFO:teuthology.orchestra.run.smithi071.stdout:1035 write 0xad2044d thru 0xad2a362 (0x9f16 bytes) 2024-09-14T05:29:38.640 INFO:teuthology.orchestra.run.smithi071.stdout:1038 read 0x65d2155 thru 0x65d4ce0 (0x2b8c bytes) 2024-09-14T05:29:38.641 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.641 INFO:teuthology.orchestra.run.smithi071.stdout:1042 punch from 0x56d656c to 0x56dc5de, (0x6072 bytes) 2024-09-14T05:29:38.641 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.641 INFO:teuthology.orchestra.run.smithi071.stdout:1044 writesame 0x8d15dbe thru 0x8d1679c data_size 0x1(0x9df bytes) 2024-09-14T05:29:38.642 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:38.642 INFO:teuthology.orchestra.run.smithi071.stdout:1047 write 0x79a637e thru 0x79b062f (0xa2b2 bytes) 2024-09-14T05:29:38.644 INFO:teuthology.orchestra.run.smithi071.stdout:1049 read 0xc28b0f2 thru 0xc29378b (0x869a bytes) 2024-09-14T05:29:38.644 INFO:teuthology.orchestra.run.smithi071.stdout:1051 read 0x4fa6387 thru 0x4fa982f (0x34a9 bytes) 2024-09-14T05:29:38.655 INFO:teuthology.orchestra.run.smithi071.stdout:1052 trunc from 0xc6eee0e to 0x43dc3a2 2024-09-14T05:29:38.657 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:38.657+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:38.676 INFO:teuthology.orchestra.run.smithi071.stdout:1054 write 0x97e8a44 thru 0x97f698b (0xdf48 bytes) 2024-09-14T05:29:38.678 INFO:teuthology.orchestra.run.smithi071.stdout:1055 read 0x20e6263 thru 0x20e6d2e (0xacc bytes) 2024-09-14T05:29:38.678 INFO:teuthology.orchestra.run.smithi071.stdout:1057 punch from 0x7b37eff to 0x7b43240, (0xb341 bytes) 2024-09-14T05:29:38.678 INFO:teuthology.orchestra.run.smithi071.stdout:1058 punch from 0x1529125 to 0x1529d8c, (0xc67 bytes) 2024-09-14T05:29:38.679 INFO:teuthology.orchestra.run.smithi071.stdout:1059 read 0x4fbd838 thru 0x4fbe1a3 (0x96c bytes) 2024-09-14T05:29:38.687 INFO:teuthology.orchestra.run.smithi071.stdout:1062 write 0xc052a58 thru 0xc05600c (0x35b5 bytes) 2024-09-14T05:29:38.690 INFO:teuthology.orchestra.run.smithi071.stdout:1064 trunc from 0xc05600d to 0xd8079da 2024-09-14T05:29:38.696 INFO:teuthology.orchestra.run.smithi071.stdout:1067 read 0xc4a486c thru 0xc4b0488 (0xbc1d bytes) 2024-09-14T05:29:38.697 INFO:teuthology.orchestra.run.smithi071.stdout:1068 punch from 0xcb6f1ae to 0xcb7ba29, (0xc87b bytes) 2024-09-14T05:29:38.697 INFO:teuthology.orchestra.run.smithi071.stdout:1070 writesame 0xc83f371 thru 0xc847d11 data_size 0x1(0x89a1 bytes) 2024-09-14T05:29:38.717 INFO:teuthology.orchestra.run.smithi071.stdout:1071 read 0x94a52d8 thru 0x94ad728 (0x8451 bytes) 2024-09-14T05:29:38.717 INFO:teuthology.orchestra.run.smithi071.stdout:1073 write 0x5c5305f thru 0x5c55024 (0x1fc6 bytes) 2024-09-14T05:29:38.719 INFO:teuthology.orchestra.run.smithi071.stdout:1075 writesame 0x38ef6ce thru 0x38f3cd7 data_size 0x16(0x460a bytes) 2024-09-14T05:29:38.721 INFO:teuthology.orchestra.run.smithi071.stdout:1078 write 0x695fe1d thru 0x696cdc5 (0xcfa9 bytes) 2024-09-14T05:29:38.722 INFO:teuthology.orchestra.run.smithi071.stdout:1081 read 0x9c9cdcb thru 0x9cab556 (0xe78c bytes) 2024-09-14T05:29:38.722 INFO:teuthology.orchestra.run.smithi071.stdout:1082 read 0x1fa550 thru 0x1fae2b (0x8dc bytes) 2024-09-14T05:29:38.723 INFO:teuthology.orchestra.run.smithi071.stdout:1083 read 0x52bb54f thru 0x52bc8df (0x1391 bytes) 2024-09-14T05:29:38.723 INFO:teuthology.orchestra.run.smithi071.stdout:1085 read 0x33f62f5 thru 0x33f6c5d (0x969 bytes) 2024-09-14T05:29:38.723 INFO:teuthology.orchestra.run.smithi071.stdout:1086 read 0x23ca0c9 thru 0x23d1aab (0x79e3 bytes) 2024-09-14T05:29:38.723 INFO:teuthology.orchestra.run.smithi071.stdout:1087 punch from 0x5f2bcc3 to 0x5f3805b, (0xc398 bytes) 2024-09-14T05:29:38.731 INFO:teuthology.orchestra.run.smithi071.stdout:1088 trunc from 0xd8079da to 0x6d2f10a 2024-09-14T05:29:38.741 INFO:teuthology.orchestra.run.smithi071.stdout:1090 trunc from 0x6d2f10a to 0x6d0f32a 2024-09-14T05:29:38.747 INFO:teuthology.orchestra.run.smithi071.stdout:1091 read 0x2c850d8 thru 0x2c8eea4 (0x9dcd bytes) 2024-09-14T05:29:38.759 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:38.758+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:38.759 INFO:teuthology.orchestra.run.smithi071.stdout:1094 write 0xbea5380 thru 0xbeb4b81 (0xf802 bytes) 2024-09-14T05:29:38.761 INFO:teuthology.orchestra.run.smithi071.stdout:1096 write 0x339bbe0 thru 0x339d0c0 (0x14e1 bytes) 2024-09-14T05:29:38.765 INFO:teuthology.orchestra.run.smithi071.stdout:1098 clone 10 order 21 su 131072 sc 6 2024-09-14T05:29:38.770 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:38.771 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:38.780 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:29:38.859 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:38.859+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:38.883 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:38.883 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:38.892 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:29:38.961 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:38.960+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:38.995 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:38.995 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:39.003 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:29:39.061 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:39.060+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:39.089 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.6 primary-affinity to 0.0166559 (810912) 2024-09-14T05:29:39.111 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:39.111 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:39.119 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:29:39.163 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:39.162+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:39.262 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:39.262+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:39.264 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:29:39.264 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:39.265 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:39.273 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:29:39.363 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:39.362+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:39.385 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:29:39.386 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:29:39.402 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:29:39.402 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:29:39.416 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:39.416 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:39.418 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:29:39.418 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:29:39.425 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:29:39.434 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:29:39.434 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:29:39.450 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:29:39.450 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:29:39.463 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:39.463+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:39.466 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:29:39.466 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:29:39.483 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:29:39.484 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:29:39.499 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:29:39.500 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:29:39.564 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:39.564+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:39.570 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:39.571 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:29:39.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:29:39.571 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:29:39.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.571 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1989 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 16326080~54496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:39.571 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:29:39.571 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.0079330679999998, 2024-09-14T05:29:39.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096664322999999996, 2024-09-14T05:29:39.571 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.571 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.571 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:39.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.572 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.572 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.572 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1989 2024-09-14T05:29:39.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.572 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:29:39.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:29:39.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561455+0000", 2024-09-14T05:29:39.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1699999999999997e-07 2024-09-14T05:29:39.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590703+0000", 2024-09-14T05:29:39.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029248225999999999 2024-09-14T05:29:39.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590732+0000", 2024-09-14T05:29:39.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8679000000000001e-05 2024-09-14T05:29:39.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590759+0000", 2024-09-14T05:29:39.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7319999999999999e-05 2024-09-14T05:29:39.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590835+0000", 2024-09-14T05:29:39.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5543999999999994e-05 2024-09-14T05:29:39.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590956+0000", 2024-09-14T05:29:39.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012168 2024-09-14T05:29:39.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 1", 2024-09-14T05:29:39.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.591261+0000", 2024-09-14T05:29:39.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030484799999999998 2024-09-14T05:29:39.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:39.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.637682+0000", 2024-09-14T05:29:39.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046420550999999997 2024-09-14T05:29:39.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:39.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.657912+0000", 2024-09-14T05:29:39.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020230011999999999 2024-09-14T05:29:39.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:39.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658007+0000", 2024-09-14T05:29:39.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5814000000000001e-05 2024-09-14T05:29:39.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658118+0000", 2024-09-14T05:29:39.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110832 2024-09-14T05:29:39.577 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.577 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.577 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.577 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2033 3.1e 3:7e47f6eb:::rbd_data.112d1f36d097.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30853661~63531] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:39.577 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:29:39.577 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.8966584019999999, 2024-09-14T05:29:39.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065167630000000004, 2024-09-14T05:29:39.577 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.578 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.578 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:39.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.578 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2033 2024-09-14T05:29:39.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.578 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:29:39.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:29:39.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672730+0000", 2024-09-14T05:29:39.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.393e-06 2024-09-14T05:29:39.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691289+0000", 2024-09-14T05:29:39.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018558991 2024-09-14T05:29:39.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691316+0000", 2024-09-14T05:29:39.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7498999999999999e-05 2024-09-14T05:29:39.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691331+0000", 2024-09-14T05:29:39.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4652e-05 2024-09-14T05:29:39.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691397+0000", 2024-09-14T05:29:39.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6112000000000007e-05 2024-09-14T05:29:39.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691509+0000", 2024-09-14T05:29:39.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111599 2024-09-14T05:29:39.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:39.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.692253+0000", 2024-09-14T05:29:39.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074456200000000004 2024-09-14T05:29:39.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:39.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.719140+0000", 2024-09-14T05:29:39.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026886542999999999 2024-09-14T05:29:39.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:39.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737789+0000", 2024-09-14T05:29:39.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018649203999999999 2024-09-14T05:29:39.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:39.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737846+0000", 2024-09-14T05:29:39.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6501e-05 2024-09-14T05:29:39.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737896+0000", 2024-09-14T05:29:39.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0574000000000001e-05 2024-09-14T05:29:39.582 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.582 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.583 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.583 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3008 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13729792~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:39.583 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:29:39.583 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.2657345319999997, 2024-09-14T05:29:39.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061007651000000003, 2024-09-14T05:29:39.583 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.583 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.583 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:39.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.583 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3008 2024-09-14T05:29:39.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.584 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:29:39.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:29:39.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303653+0000", 2024-09-14T05:29:39.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0699999999999999e-07 2024-09-14T05:29:39.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303654+0000", 2024-09-14T05:29:39.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-07 2024-09-14T05:29:39.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303657+0000", 2024-09-14T05:29:39.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2890000000000002e-06 2024-09-14T05:29:39.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303661+0000", 2024-09-14T05:29:39.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3290000000000004e-06 2024-09-14T05:29:39.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364233+0000", 2024-09-14T05:29:39.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060572134 2024-09-14T05:29:39.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364315+0000", 2024-09-14T05:29:39.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1549000000000001e-05 2024-09-14T05:29:39.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364660+0000", 2024-09-14T05:29:39.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034525300000000001 2024-09-14T05:29:39.587 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.587 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.587 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.587 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3009 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14221312~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:39.587 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:29:39.587 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.2657204419999992, 2024-09-14T05:29:39.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061445063000000001, 2024-09-14T05:29:39.587 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.587 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.588 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:39.588 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.588 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.588 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.588 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3009 2024-09-14T05:29:39.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.588 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:29:39.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:29:39.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:29:39.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:29:39.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303669+0000", 2024-09-14T05:29:39.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7150000000000001e-06 2024-09-14T05:29:39.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303675+0000", 2024-09-14T05:29:39.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5339999999999996e-06 2024-09-14T05:29:39.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303681+0000", 2024-09-14T05:29:39.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3639999999999998e-06 2024-09-14T05:29:39.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364705+0000", 2024-09-14T05:29:39.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061024123 2024-09-14T05:29:39.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364786+0000", 2024-09-14T05:29:39.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1378999999999995e-05 2024-09-14T05:29:39.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365112+0000", 2024-09-14T05:29:39.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032558700000000001 2024-09-14T05:29:39.591 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.591 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.591 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.592 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3010 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14712832~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:39.592 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:29:39.592 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.2656995779999995, 2024-09-14T05:29:39.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061834630000000002, 2024-09-14T05:29:39.592 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.592 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.592 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:39.592 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.592 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.592 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.592 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3010 2024-09-14T05:29:39.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.592 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:29:39.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:29:39.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303688+0000", 2024-09-14T05:29:39.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:29:39.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303690+0000", 2024-09-14T05:29:39.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9460000000000001e-06 2024-09-14T05:29:39.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303696+0000", 2024-09-14T05:29:39.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2230000000000004e-06 2024-09-14T05:29:39.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303700+0000", 2024-09-14T05:29:39.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7290000000000002e-06 2024-09-14T05:29:39.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365163+0000", 2024-09-14T05:29:39.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061463093000000003 2024-09-14T05:29:39.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365214+0000", 2024-09-14T05:29:39.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0977999999999999e-05 2024-09-14T05:29:39.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365522+0000", 2024-09-14T05:29:39.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000308227 2024-09-14T05:29:39.595 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.596 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.596 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.596 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3241 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:39.596 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:29:39.596 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.2400251559999997, 2024-09-14T05:29:39.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061041892, 2024-09-14T05:29:39.596 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.596 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.596 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:39.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.597 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3241 2024-09-14T05:29:39.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.597 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:29:39.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:29:39.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329496+0000", 2024-09-14T05:29:39.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013421799999999999 2024-09-14T05:29:39.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329499+0000", 2024-09-14T05:29:39.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8459999999999999e-06 2024-09-14T05:29:39.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329505+0000", 2024-09-14T05:29:39.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4320000000000004e-06 2024-09-14T05:29:39.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329509+0000", 2024-09-14T05:29:39.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0339999999999998e-06 2024-09-14T05:29:39.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.389947+0000", 2024-09-14T05:29:39.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060438093999999998 2024-09-14T05:29:39.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390038+0000", 2024-09-14T05:29:39.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0253999999999993e-05 2024-09-14T05:29:39.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390404+0000", 2024-09-14T05:29:39.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036601400000000001 2024-09-14T05:29:39.600 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.600 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.600 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.600 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3242 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:39.600 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:29:39.600 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.2398702480000008, 2024-09-14T05:29:39.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061282426000000001, 2024-09-14T05:29:39.600 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.600 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.600 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:39.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.601 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3242 2024-09-14T05:29:39.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.601 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:29:39.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:29:39.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:29:39.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9900000000000001e-07 2024-09-14T05:29:39.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329518+0000", 2024-09-14T05:29:39.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.328e-06 2024-09-14T05:29:39.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329522+0000", 2024-09-14T05:29:39.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.563e-06 2024-09-14T05:29:39.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329525+0000", 2024-09-14T05:29:39.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7379999999999999e-06 2024-09-14T05:29:39.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390437+0000", 2024-09-14T05:29:39.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060912463 2024-09-14T05:29:39.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390498+0000", 2024-09-14T05:29:39.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0615999999999997e-05 2024-09-14T05:29:39.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390799+0000", 2024-09-14T05:29:39.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030131899999999998 2024-09-14T05:29:39.604 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.604 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.604 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.604 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3243 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:39.604 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:29:39.604 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.2398584620000008, 2024-09-14T05:29:39.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061728980000000003, 2024-09-14T05:29:39.604 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.604 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.605 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:39.605 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.605 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.605 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.605 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3243 2024-09-14T05:29:39.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.605 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:29:39.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:29:39.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329692+0000", 2024-09-14T05:29:39.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000163948 2024-09-14T05:29:39.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329696+0000", 2024-09-14T05:29:39.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5540000000000002e-06 2024-09-14T05:29:39.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329706+0000", 2024-09-14T05:29:39.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7960000000000007e-06 2024-09-14T05:29:39.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329712+0000", 2024-09-14T05:29:39.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6109999999999999e-06 2024-09-14T05:29:39.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390861+0000", 2024-09-14T05:29:39.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061148767 2024-09-14T05:29:39.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390942+0000", 2024-09-14T05:29:39.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1266000000000004e-05 2024-09-14T05:29:39.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391258+0000", 2024-09-14T05:29:39.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031503800000000002 2024-09-14T05:29:39.608 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.608 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.608 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.608 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3244 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14778368~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:39.608 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:29:39.608 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.2396634140000007, 2024-09-14T05:29:39.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062050594000000001, 2024-09-14T05:29:39.609 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.609 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.609 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:39.609 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.609 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.609 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.609 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3244 2024-09-14T05:29:39.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.609 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:29:39.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:29:39.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329916+0000", 2024-09-14T05:29:39.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019290799999999999 2024-09-14T05:29:39.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329919+0000", 2024-09-14T05:29:39.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9019999999999999e-06 2024-09-14T05:29:39.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329926+0000", 2024-09-14T05:29:39.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9e-06 2024-09-14T05:29:39.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329932+0000", 2024-09-14T05:29:39.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.088e-06 2024-09-14T05:29:39.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391299+0000", 2024-09-14T05:29:39.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061366136000000002 2024-09-14T05:29:39.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391391+0000", 2024-09-14T05:29:39.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2204e-05 2024-09-14T05:29:39.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391774+0000", 2024-09-14T05:29:39.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038345600000000001 2024-09-14T05:29:39.612 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.612 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.613 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.613 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3921 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 5308416~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:39.613 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:29:39.613 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.1681196039999993, 2024-09-14T05:29:39.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060842694000000003, 2024-09-14T05:29:39.613 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.613 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.613 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:39.613 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.613 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.613 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.614 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3921 2024-09-14T05:29:39.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.614 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:29:39.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:29:39.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401705+0000", 2024-09-14T05:29:39.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768499999999999 2024-09-14T05:29:39.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401709+0000", 2024-09-14T05:29:39.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0319999999999997e-06 2024-09-14T05:29:39.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401720+0000", 2024-09-14T05:29:39.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1005000000000001e-05 2024-09-14T05:29:39.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401727+0000", 2024-09-14T05:29:39.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1879999999999996e-06 2024-09-14T05:29:39.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461748+0000", 2024-09-14T05:29:39.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060021199999999997 2024-09-14T05:29:39.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461793+0000", 2024-09-14T05:29:39.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4530999999999998e-05 2024-09-14T05:29:39.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462110+0000", 2024-09-14T05:29:39.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031705299999999997 2024-09-14T05:29:39.617 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.617 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.617 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.617 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3923 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 6291456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:39.617 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:29:39.617 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.1674735760000008, 2024-09-14T05:29:39.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060891273000000003, 2024-09-14T05:29:39.618 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.618 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.618 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:39.618 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.618 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.618 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.618 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3923 2024-09-14T05:29:39.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.618 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:29:39.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:29:39.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407054+0000", 2024-09-14T05:29:39.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005140787 2024-09-14T05:29:39.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407059+0000", 2024-09-14T05:29:39.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.04e-06 2024-09-14T05:29:39.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407069+0000", 2024-09-14T05:29:39.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4429999999999992e-06 2024-09-14T05:29:39.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407076+0000", 2024-09-14T05:29:39.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1679999999999998e-06 2024-09-14T05:29:39.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462514+0000", 2024-09-14T05:29:39.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055438551000000003 2024-09-14T05:29:39.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462562+0000", 2024-09-14T05:29:39.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7649999999999999e-05 2024-09-14T05:29:39.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462805+0000", 2024-09-14T05:29:39.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024263400000000001 2024-09-14T05:29:39.621 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.621 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.621 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.622 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4052 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 12550144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:39.622 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:29:39.622 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.1519015669999995, 2024-09-14T05:29:39.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060872904999999998, 2024-09-14T05:29:39.622 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.622 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.622 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:39.622 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.622 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.622 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.622 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4052 2024-09-14T05:29:39.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.622 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:29:39.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:29:39.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417486+0000", 2024-09-14T05:29:39.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-14T05:29:39.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417487+0000", 2024-09-14T05:29:39.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4499999999999995e-07 2024-09-14T05:29:39.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417489+0000", 2024-09-14T05:29:39.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8899999999999999e-06 2024-09-14T05:29:39.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417492+0000", 2024-09-14T05:29:39.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4389999999999999e-06 2024-09-14T05:29:39.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478027+0000", 2024-09-14T05:29:39.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060534959999999999 2024-09-14T05:29:39.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478079+0000", 2024-09-14T05:29:39.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2104999999999997e-05 2024-09-14T05:29:39.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478358+0000", 2024-09-14T05:29:39.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000279351 2024-09-14T05:29:39.625 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.625 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.625 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.626 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4053 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13041664~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:39.626 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:29:39.626 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.1518921829999993, 2024-09-14T05:29:39.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061258866000000002, 2024-09-14T05:29:39.626 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.626 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.626 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:39.626 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.626 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.626 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.626 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4053 2024-09-14T05:29:39.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.626 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:29:39.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:29:39.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:29:39.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.17e-07 2024-09-14T05:29:39.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417496+0000", 2024-09-14T05:29:39.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0890000000000001e-06 2024-09-14T05:29:39.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417499+0000", 2024-09-14T05:29:39.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8629999999999999e-06 2024-09-14T05:29:39.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417502+0000", 2024-09-14T05:29:39.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6290000000000002e-06 2024-09-14T05:29:39.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478387+0000", 2024-09-14T05:29:39.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060885433000000003 2024-09-14T05:29:39.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478435+0000", 2024-09-14T05:29:39.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8167999999999997e-05 2024-09-14T05:29:39.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478754+0000", 2024-09-14T05:29:39.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031846700000000001 2024-09-14T05:29:39.629 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.629 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.630 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.630 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4054 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13533184~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:39.630 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:29:39.630 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.1518822990000004, 2024-09-14T05:29:39.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061604694000000002, 2024-09-14T05:29:39.630 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.630 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.630 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:39.630 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.630 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.630 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.630 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4054 2024-09-14T05:29:39.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.631 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:29:39.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:29:39.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:29:39.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.04e-07 2024-09-14T05:29:39.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417506+0000", 2024-09-14T05:29:39.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8299999999999995e-07 2024-09-14T05:29:39.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417509+0000", 2024-09-14T05:29:39.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.88e-06 2024-09-14T05:29:39.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417511+0000", 2024-09-14T05:29:39.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5950000000000001e-06 2024-09-14T05:29:39.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478784+0000", 2024-09-14T05:29:39.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061272971000000002 2024-09-14T05:29:39.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478841+0000", 2024-09-14T05:29:39.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-14T05:29:39.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479109+0000", 2024-09-14T05:29:39.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026794600000000003 2024-09-14T05:29:39.634 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.634 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.634 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.634 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4055 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14024704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:39.634 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:29:39.634 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.1518725970000006, 2024-09-14T05:29:39.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061932299000000003, 2024-09-14T05:29:39.634 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.634 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.635 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:39.635 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.635 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.635 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.635 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4055 2024-09-14T05:29:39.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.635 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:29:39.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:29:39.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417515+0000", 2024-09-14T05:29:39.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-14T05:29:39.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417516+0000", 2024-09-14T05:29:39.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7199999999999997e-07 2024-09-14T05:29:39.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417519+0000", 2024-09-14T05:29:39.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.937e-06 2024-09-14T05:29:39.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417522+0000", 2024-09-14T05:29:39.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1420000000000002e-06 2024-09-14T05:29:39.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479140+0000", 2024-09-14T05:29:39.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061617933999999999 2024-09-14T05:29:39.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479188+0000", 2024-09-14T05:29:39.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8717000000000001e-05 2024-09-14T05:29:39.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479447+0000", 2024-09-14T05:29:39.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025838800000000002 2024-09-14T05:29:39.638 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.638 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.639 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.639 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4056 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14516224~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:39.639 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:29:39.639 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.1518624240000008, 2024-09-14T05:29:39.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062263185999999998, 2024-09-14T05:29:39.639 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.639 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.639 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:39.639 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.639 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.639 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.639 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4056 2024-09-14T05:29:39.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.640 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:29:39.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:29:39.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:29:39.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8700000000000002e-07 2024-09-14T05:29:39.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417528+0000", 2024-09-14T05:29:39.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.579e-06 2024-09-14T05:29:39.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417533+0000", 2024-09-14T05:29:39.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2400000000000001e-06 2024-09-14T05:29:39.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417536+0000", 2024-09-14T05:29:39.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0309999999999999e-06 2024-09-14T05:29:39.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479477+0000", 2024-09-14T05:29:39.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061940967 2024-09-14T05:29:39.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479523+0000", 2024-09-14T05:29:39.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6335000000000003e-05 2024-09-14T05:29:39.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479788+0000", 2024-09-14T05:29:39.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026474700000000002 2024-09-14T05:29:39.643 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.643 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.643 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.643 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4057 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15007744~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:39.643 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:29:39.643 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.1518479629999998, 2024-09-14T05:29:39.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062589928000000003, 2024-09-14T05:29:39.643 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.643 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.643 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:39.644 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.644 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.644 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.644 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4057 2024-09-14T05:29:39.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.644 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:29:39.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:29:39.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422838+0000", 2024-09-14T05:29:39.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052990349999999997 2024-09-14T05:29:39.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422854+0000", 2024-09-14T05:29:39.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5523999999999999e-05 2024-09-14T05:29:39.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422864+0000", 2024-09-14T05:29:39.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0084e-05 2024-09-14T05:29:39.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422872+0000", 2024-09-14T05:29:39.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-14T05:29:39.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479813+0000", 2024-09-14T05:29:39.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056941389000000002 2024-09-14T05:29:39.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479868+0000", 2024-09-14T05:29:39.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4973999999999999e-05 2024-09-14T05:29:39.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.480129+0000", 2024-09-14T05:29:39.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026084200000000001 2024-09-14T05:29:39.647 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.647 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.647 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.647 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4171 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14057472~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:39.647 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:29:39.647 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.1398504030000005, 2024-09-14T05:29:39.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061184929999999998, 2024-09-14T05:29:39.648 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.648 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.648 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:39.648 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.648 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.648 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.648 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4171 2024-09-14T05:29:39.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.648 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:29:39.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:29:39.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:29:39.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.39e-07 2024-09-14T05:29:39.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429541+0000", 2024-09-14T05:29:39.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:29:39.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429547+0000", 2024-09-14T05:29:39.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3280000000000002e-06 2024-09-14T05:29:39.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429552+0000", 2024-09-14T05:29:39.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4859999999999997e-06 2024-09-14T05:29:39.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490379+0000", 2024-09-14T05:29:39.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060826681 2024-09-14T05:29:39.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490434+0000", 2024-09-14T05:29:39.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4577999999999998e-05 2024-09-14T05:29:39.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490722+0000", 2024-09-14T05:29:39.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028781400000000001 2024-09-14T05:29:39.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.651 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.651 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4172 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14548992~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:39.652 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:29:39.652 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.1398283950000003, 2024-09-14T05:29:39.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061498256000000001, 2024-09-14T05:29:39.652 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.652 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.652 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:39.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.652 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4172 2024-09-14T05:29:39.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.652 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:29:39.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:29:39.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:29:39.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.72e-07 2024-09-14T05:29:39.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429561+0000", 2024-09-14T05:29:39.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.717e-06 2024-09-14T05:29:39.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429566+0000", 2024-09-14T05:29:39.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6529999999999999e-06 2024-09-14T05:29:39.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429571+0000", 2024-09-14T05:29:39.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5979999999999999e-06 2024-09-14T05:29:39.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490751+0000", 2024-09-14T05:29:39.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061180336000000002 2024-09-14T05:29:39.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490798+0000", 2024-09-14T05:29:39.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6875000000000001e-05 2024-09-14T05:29:39.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491057+0000", 2024-09-14T05:29:39.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025870500000000002 2024-09-14T05:29:39.656 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.656 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.656 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.656 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4173 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15040512~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:39.656 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:29:39.656 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.1398107690000003, 2024-09-14T05:29:39.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061894820000000003, 2024-09-14T05:29:39.656 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:39.656 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:39.656 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:39.657 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:39.657 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:39.657 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:39.657 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4173 2024-09-14T05:29:39.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.657 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:39.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:39.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:29:39.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:39.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:29:39.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:39.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:39.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429577+0000", 2024-09-14T05:29:39.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-14T05:29:39.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:39.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429578+0000", 2024-09-14T05:29:39.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6980000000000001e-06 2024-09-14T05:29:39.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:39.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429585+0000", 2024-09-14T05:29:39.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.725e-06 2024-09-14T05:29:39.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:39.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429590+0000", 2024-09-14T05:29:39.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.938e-06 2024-09-14T05:29:39.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:39.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491091+0000", 2024-09-14T05:29:39.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061500773000000002 2024-09-14T05:29:39.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:39.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491136+0000", 2024-09-14T05:29:39.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5284000000000002e-05 2024-09-14T05:29:39.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:39.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:39.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:39.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491471+0000", 2024-09-14T05:29:39.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033485699999999998 2024-09-14T05:29:39.660 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.660 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.660 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.660 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:39.660 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:39.660 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:39.661 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:29:39.705 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:39.704+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:39.765 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:39.765 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:39.765 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:29:39.765 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:39.765 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:39.765+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:39.773 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:29:39.866 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:39.865+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:39.917 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:39.917 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:39.917 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:29:39.917 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:29:39.917 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:39.926 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:29:39.967 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:39.966+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:40.068 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:40.068+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:40.071 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:40.071 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:29:40.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:29:40.072 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:29:40.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.072 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:1134 3.7 e30/24 3:e1ef7f87:::rbd_data.112dd25fb4c.000000000000005f:head v 30'40, mlcod=30'36)", 2024-09-14T05:29:40.072 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.166405+0000", 2024-09-14T05:29:40.072 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.9034706969999995, 2024-09-14T05:29:40.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061353650000000003, 2024-09-14T05:29:40.072 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.072 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.072 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:40.072 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166405+0000", 2024-09-14T05:29:40.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166405+0000", 2024-09-14T05:29:40.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166406+0000", 2024-09-14T05:29:40.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.85e-07 2024-09-14T05:29:40.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166413+0000", 2024-09-14T05:29:40.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1029999999999996e-06 2024-09-14T05:29:40.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166425+0000", 2024-09-14T05:29:40.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.258e-05 2024-09-14T05:29:40.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166433+0000", 2024-09-14T05:29:40.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2910000000000002e-06 2024-09-14T05:29:40.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166492+0000", 2024-09-14T05:29:40.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9107999999999997e-05 2024-09-14T05:29:40.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166513+0000", 2024-09-14T05:29:40.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.128e-05 2024-09-14T05:29:40.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:40.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.172500+0000", 2024-09-14T05:29:40.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059870519999999997 2024-09-14T05:29:40.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.172540+0000", 2024-09-14T05:29:40.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0166000000000001e-05 2024-09-14T05:29:40.076 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.076 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.076 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.077 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1205 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:40.077 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:29:40.077 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.8719858289999998, 2024-09-14T05:29:40.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0086364279999999998, 2024-09-14T05:29:40.077 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.077 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.077 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.077 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1205 2024-09-14T05:29:40.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.077 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:29:40.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:29:40.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197891+0000", 2024-09-14T05:29:40.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.364e-06 2024-09-14T05:29:40.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197896+0000", 2024-09-14T05:29:40.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3000000000000001e-06 2024-09-14T05:29:40.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197910+0000", 2024-09-14T05:29:40.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3339000000000001e-05 2024-09-14T05:29:40.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197919+0000", 2024-09-14T05:29:40.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8869999999999998e-06 2024-09-14T05:29:40.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197974+0000", 2024-09-14T05:29:40.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5720000000000002e-05 2024-09-14T05:29:40.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.198094+0000", 2024-09-14T05:29:40.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119739 2024-09-14T05:29:40.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206526+0000", 2024-09-14T05:29:40.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084320790000000003 2024-09-14T05:29:40.081 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.081 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.081 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.081 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1220 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:40.082 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:29:40.082 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.8632294649999999, 2024-09-14T05:29:40.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012822337, 2024-09-14T05:29:40.082 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.082 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.082 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:40.082 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.082 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.082 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.082 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1220 2024-09-14T05:29:40.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.082 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:29:40.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:29:40.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206647+0000", 2024-09-14T05:29:40.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4600000000000003e-07 2024-09-14T05:29:40.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206652+0000", 2024-09-14T05:29:40.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6619999999999996e-06 2024-09-14T05:29:40.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206662+0000", 2024-09-14T05:29:40.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.995e-06 2024-09-14T05:29:40.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206667+0000", 2024-09-14T05:29:40.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-14T05:29:40.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206706+0000", 2024-09-14T05:29:40.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9523000000000001e-05 2024-09-14T05:29:40.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206755+0000", 2024-09-14T05:29:40.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000002e-05 2024-09-14T05:29:40.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:40.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206877+0000", 2024-09-14T05:29:40.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121938 2024-09-14T05:29:40.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:40.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.207310+0000", 2024-09-14T05:29:40.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000432452 2024-09-14T05:29:40.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:40.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219364+0000", 2024-09-14T05:29:40.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012054481000000001 2024-09-14T05:29:40.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:40.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219425+0000", 2024-09-14T05:29:40.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0618000000000001e-05 2024-09-14T05:29:40.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219468+0000", 2024-09-14T05:29:40.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3739000000000003e-05 2024-09-14T05:29:40.087 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.087 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.087 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.087 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1243 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:40.087 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:29:40.087 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.8502642379999994, 2024-09-14T05:29:40.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078660580000000004, 2024-09-14T05:29:40.087 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.088 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.088 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.088 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.088 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.088 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.088 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1243 2024-09-14T05:29:40.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.088 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:29:40.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:29:40.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219612+0000", 2024-09-14T05:29:40.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.09e-07 2024-09-14T05:29:40.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219616+0000", 2024-09-14T05:29:40.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.216e-06 2024-09-14T05:29:40.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219630+0000", 2024-09-14T05:29:40.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3064999999999999e-05 2024-09-14T05:29:40.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219637+0000", 2024-09-14T05:29:40.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7409999999999992e-06 2024-09-14T05:29:40.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219697+0000", 2024-09-14T05:29:40.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.02e-05 2024-09-14T05:29:40.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219833+0000", 2024-09-14T05:29:40.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013595600000000001 2024-09-14T05:29:40.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227477+0000", 2024-09-14T05:29:40.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076439710000000003 2024-09-14T05:29:40.091 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.092 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.092 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.092 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1264 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:40.092 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:29:40.092 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.8422534450000008, 2024-09-14T05:29:40.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080867990000000004, 2024-09-14T05:29:40.092 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.092 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.092 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:40.092 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.093 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.093 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.093 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1264 2024-09-14T05:29:40.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.093 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:29:40.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:29:40.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227623+0000", 2024-09-14T05:29:40.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:29:40.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227627+0000", 2024-09-14T05:29:40.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.194e-06 2024-09-14T05:29:40.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227639+0000", 2024-09-14T05:29:40.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1296e-05 2024-09-14T05:29:40.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227648+0000", 2024-09-14T05:29:40.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2280000000000004e-06 2024-09-14T05:29:40.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227689+0000", 2024-09-14T05:29:40.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1329999999999999e-05 2024-09-14T05:29:40.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227741+0000", 2024-09-14T05:29:40.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1641000000000002e-05 2024-09-14T05:29:40.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:29:40.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227914+0000", 2024-09-14T05:29:40.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000173215 2024-09-14T05:29:40.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:40.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228474+0000", 2024-09-14T05:29:40.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055980199999999998 2024-09-14T05:29:40.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:40.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235608+0000", 2024-09-14T05:29:40.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0071343830000000002 2024-09-14T05:29:40.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:40.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235669+0000", 2024-09-14T05:29:40.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1068999999999994e-05 2024-09-14T05:29:40.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235709+0000", 2024-09-14T05:29:40.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9671e-05 2024-09-14T05:29:40.098 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.098 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.098 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.098 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1520 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:40.098 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:29:40.098 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.7472458260000003, 2024-09-14T05:29:40.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061963629999999999, 2024-09-14T05:29:40.099 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.099 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.099 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.099 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.099 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.099 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.099 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1520 2024-09-14T05:29:40.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.099 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:29:40.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:29:40.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322631+0000", 2024-09-14T05:29:40.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.54e-07 2024-09-14T05:29:40.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322635+0000", 2024-09-14T05:29:40.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0450000000000002e-06 2024-09-14T05:29:40.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322646+0000", 2024-09-14T05:29:40.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1919999999999999e-05 2024-09-14T05:29:40.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322654+0000", 2024-09-14T05:29:40.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3139999999999998e-06 2024-09-14T05:29:40.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322704+0000", 2024-09-14T05:29:40.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0420000000000002e-05 2024-09-14T05:29:40.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322794+0000", 2024-09-14T05:29:40.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0037000000000005e-05 2024-09-14T05:29:40.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328826+0000", 2024-09-14T05:29:40.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060318730000000001 2024-09-14T05:29:40.104 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.104 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.104 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.104 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1526 3.4 3:2695cbe4:::rbd_data.112dd25fb4c.0000000000000119:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:40.104 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:29:40.104 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.7437128820000005, 2024-09-14T05:29:40.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0079537359999999994, 2024-09-14T05:29:40.104 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.104 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.104 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:40.105 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.105 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.105 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.105 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1526 2024-09-14T05:29:40.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.105 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:29:40.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:29:40.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:29:40.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3600000000000003e-07 2024-09-14T05:29:40.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326168+0000", 2024-09-14T05:29:40.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3460000000000004e-06 2024-09-14T05:29:40.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326178+0000", 2024-09-14T05:29:40.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0291e-05 2024-09-14T05:29:40.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326183+0000", 2024-09-14T05:29:40.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-14T05:29:40.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326230+0000", 2024-09-14T05:29:40.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6650999999999998e-05 2024-09-14T05:29:40.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326295+0000", 2024-09-14T05:29:40.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5158000000000002e-05 2024-09-14T05:29:40.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:40.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326408+0000", 2024-09-14T05:29:40.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113039 2024-09-14T05:29:40.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:40.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.331648+0000", 2024-09-14T05:29:40.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052404929999999997 2024-09-14T05:29:40.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:40.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334011+0000", 2024-09-14T05:29:40.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002362669 2024-09-14T05:29:40.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:40.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334064+0000", 2024-09-14T05:29:40.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3198999999999997e-05 2024-09-14T05:29:40.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334116+0000", 2024-09-14T05:29:40.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2133999999999999e-05 2024-09-14T05:29:40.110 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.110 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.110 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.110 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1529 3.1 3:809746f7:::rbd_data.112dd25fb4c.0000000000000118:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:40.110 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:29:40.110 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.7419483319999998, 2024-09-14T05:29:40.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061981060000000001, 2024-09-14T05:29:40.111 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.111 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.111 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:40.111 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.111 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.111 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.111 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1529 2024-09-14T05:29:40.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.111 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:29:40.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:29:40.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327928+0000", 2024-09-14T05:29:40.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0100000000000004e-07 2024-09-14T05:29:40.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327933+0000", 2024-09-14T05:29:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4689999999999999e-06 2024-09-14T05:29:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327944+0000", 2024-09-14T05:29:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1623999999999999e-05 2024-09-14T05:29:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327951+0000", 2024-09-14T05:29:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9759999999999998e-06 2024-09-14T05:29:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328003+0000", 2024-09-14T05:29:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1521999999999999e-05 2024-09-14T05:29:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328087+0000", 2024-09-14T05:29:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3988999999999998e-05 2024-09-14T05:29:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:40.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328304+0000", 2024-09-14T05:29:40.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000217254 2024-09-14T05:29:40.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:40.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.333363+0000", 2024-09-14T05:29:40.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050586499999999996 2024-09-14T05:29:40.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:40.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334026+0000", 2024-09-14T05:29:40.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066303499999999997 2024-09-14T05:29:40.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:40.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334080+0000", 2024-09-14T05:29:40.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4261000000000001e-05 2024-09-14T05:29:40.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334125+0000", 2024-09-14T05:29:40.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5525000000000002e-05 2024-09-14T05:29:40.117 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.117 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.117 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.117 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1531 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:40.117 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:29:40.117 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.7408947159999997, 2024-09-14T05:29:40.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082114309999999999, 2024-09-14T05:29:40.117 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.117 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.117 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:40.117 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.118 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.118 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.118 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1531 2024-09-14T05:29:40.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.118 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:29:40.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:29:40.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328982+0000", 2024-09-14T05:29:40.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:29:40.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328985+0000", 2024-09-14T05:29:40.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7900000000000001e-06 2024-09-14T05:29:40.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328996+0000", 2024-09-14T05:29:40.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.028e-05 2024-09-14T05:29:40.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329001+0000", 2024-09-14T05:29:40.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0610000000000003e-06 2024-09-14T05:29:40.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329035+0000", 2024-09-14T05:29:40.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4366999999999998e-05 2024-09-14T05:29:40.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329082+0000", 2024-09-14T05:29:40.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6646e-05 2024-09-14T05:29:40.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:40.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329195+0000", 2024-09-14T05:29:40.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113279 2024-09-14T05:29:40.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:40.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334293+0000", 2024-09-14T05:29:40.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050980399999999999 2024-09-14T05:29:40.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:40.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337101+0000", 2024-09-14T05:29:40.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0028076170000000001 2024-09-14T05:29:40.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:40.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337154+0000", 2024-09-14T05:29:40.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3641999999999999e-05 2024-09-14T05:29:40.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337192+0000", 2024-09-14T05:29:40.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7932000000000002e-05 2024-09-14T05:29:40.123 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.123 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.123 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.123 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:2131 3.b e30/24 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head v 30'36, mlcod=30'34)", 2024-09-14T05:29:40.123 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:29:40.123 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2775473609999999, 2024-09-14T05:29:40.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031871709999999998, 2024-09-14T05:29:40.123 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.124 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.124 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:40.124 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:29:40.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:29:40.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792331+0000", 2024-09-14T05:29:40.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6900000000000001e-06 2024-09-14T05:29:40.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797319+0000", 2024-09-14T05:29:40.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049876349999999998 2024-09-14T05:29:40.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797349+0000", 2024-09-14T05:29:40.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0046e-05 2024-09-14T05:29:40.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797359+0000", 2024-09-14T05:29:40.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0037999999999999e-05 2024-09-14T05:29:40.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797445+0000", 2024-09-14T05:29:40.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6132999999999997e-05 2024-09-14T05:29:40.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797472+0000", 2024-09-14T05:29:40.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7478999999999999e-05 2024-09-14T05:29:40.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:40.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824161+0000", 2024-09-14T05:29:40.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026688714999999998 2024-09-14T05:29:40.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824200+0000", 2024-09-14T05:29:40.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8974000000000003e-05 2024-09-14T05:29:40.127 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.127 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.127 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.128 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:2259 3.b e31/24 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head v 31'38, mlcod=30'35)", 2024-09-14T05:29:40.128 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.120881+0000", 2024-09-14T05:29:40.128 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9489947329999993, 2024-09-14T05:29:40.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006003815, 2024-09-14T05:29:40.128 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.128 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.128 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:40.128 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.120881+0000", 2024-09-14T05:29:40.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.120881+0000", 2024-09-14T05:29:40.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.120881+0000", 2024-09-14T05:29:40.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.65e-07 2024-09-14T05:29:40.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.120887+0000", 2024-09-14T05:29:40.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3679999999999998e-06 2024-09-14T05:29:40.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.120905+0000", 2024-09-14T05:29:40.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7448999999999998e-05 2024-09-14T05:29:40.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.120911+0000", 2024-09-14T05:29:40.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1449999999999996e-06 2024-09-14T05:29:40.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.120997+0000", 2024-09-14T05:29:40.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5641000000000002e-05 2024-09-14T05:29:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.121020+0000", 2024-09-14T05:29:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3714999999999999e-05 2024-09-14T05:29:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.126839+0000", 2024-09-14T05:29:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058184109999999999 2024-09-14T05:29:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.126885+0000", 2024-09-14T05:29:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5821000000000001e-05 2024-09-14T05:29:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4259 3.4 3:25f21dc9:::rbd_data.112d3d611e54.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 294912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:29:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.6783747469999994, 2024-09-14T05:29:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010043553, 2024-09-14T05:29:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4259 2024-09-14T05:29:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:29:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:29:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391503+0000", 2024-09-14T05:29:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9099999999999999e-06 2024-09-14T05:29:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391821+0000", 2024-09-14T05:29:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031871800000000001 2024-09-14T05:29:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391844+0000", 2024-09-14T05:29:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2045e-05 2024-09-14T05:29:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391853+0000", 2024-09-14T05:29:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7249999999999992e-06 2024-09-14T05:29:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391917+0000", 2024-09-14T05:29:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3458000000000001e-05 2024-09-14T05:29:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392037+0000", 2024-09-14T05:29:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120449 2024-09-14T05:29:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392305+0000", 2024-09-14T05:29:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026739299999999999 2024-09-14T05:29:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.393609+0000", 2024-09-14T05:29:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013042450000000001 2024-09-14T05:29:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401430+0000", 2024-09-14T05:29:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078211670000000004 2024-09-14T05:29:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401499+0000", 2024-09-14T05:29:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9374000000000004e-05 2024-09-14T05:29:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401544+0000", 2024-09-14T05:29:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5068999999999998e-05 2024-09-14T05:29:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4452 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:29:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.1875621390000006, 2024-09-14T05:29:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066061380000000001, 2024-09-14T05:29:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4452 2024-09-14T05:29:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:29:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:29:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882314+0000", 2024-09-14T05:29:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5899999999999998e-07 2024-09-14T05:29:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882315+0000", 2024-09-14T05:29:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6169999999999999e-06 2024-09-14T05:29:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882319+0000", 2024-09-14T05:29:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3629999999999998e-06 2024-09-14T05:29:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882324+0000", 2024-09-14T05:29:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1370000000000001e-06 2024-09-14T05:29:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883369+0000", 2024-09-14T05:29:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010448320000000001 2024-09-14T05:29:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883421+0000", 2024-09-14T05:29:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2401000000000002e-05 2024-09-14T05:29:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883638+0000", 2024-09-14T05:29:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021725600000000001 2024-09-14T05:29:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.884464+0000", 2024-09-14T05:29:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00082569800000000004 2024-09-14T05:29:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888860+0000", 2024-09-14T05:29:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0043963370000000002 2024-09-14T05:29:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888896+0000", 2024-09-14T05:29:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6022999999999997e-05 2024-09-14T05:29:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888920+0000", 2024-09-14T05:29:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3215000000000001e-05 2024-09-14T05:29:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4467 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:29:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.1691936890000001, 2024-09-14T05:29:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0065700189999999999, 2024-09-14T05:29:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4467 2024-09-14T05:29:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:29:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:29:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:29:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-14T05:29:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900684+0000", 2024-09-14T05:29:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.629e-06 2024-09-14T05:29:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900688+0000", 2024-09-14T05:29:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8469999999999998e-06 2024-09-14T05:29:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900691+0000", 2024-09-14T05:29:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-14T05:29:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.901775+0000", 2024-09-14T05:29:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001084177 2024-09-14T05:29:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.901862+0000", 2024-09-14T05:29:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7167000000000001e-05 2024-09-14T05:29:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.902223+0000", 2024-09-14T05:29:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036068800000000001 2024-09-14T05:29:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.902843+0000", 2024-09-14T05:29:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000620489 2024-09-14T05:29:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907167+0000", 2024-09-14T05:29:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0043242649999999999 2024-09-14T05:29:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907217+0000", 2024-09-14T05:29:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9648000000000003e-05 2024-09-14T05:29:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907252+0000", 2024-09-14T05:29:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4780000000000002e-05 2024-09-14T05:29:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4554 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:29:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0854739359999996, 2024-09-14T05:29:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069998999999999999, 2024-09-14T05:29:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4554 2024-09-14T05:29:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:29:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:29:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:29:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9299999999999999e-07 2024-09-14T05:29:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984404+0000", 2024-09-14T05:29:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.855e-06 2024-09-14T05:29:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984407+0000", 2024-09-14T05:29:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-14T05:29:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984410+0000", 2024-09-14T05:29:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8159999999999998e-06 2024-09-14T05:29:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985511+0000", 2024-09-14T05:29:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00110053 2024-09-14T05:29:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985561+0000", 2024-09-14T05:29:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0389000000000002e-05 2024-09-14T05:29:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985772+0000", 2024-09-14T05:29:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000210571 2024-09-14T05:29:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.987075+0000", 2024-09-14T05:29:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001303128 2024-09-14T05:29:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991303+0000", 2024-09-14T05:29:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042283310000000001 2024-09-14T05:29:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991359+0000", 2024-09-14T05:29:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6335999999999997e-05 2024-09-14T05:29:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991402+0000", 2024-09-14T05:29:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2074999999999999e-05 2024-09-14T05:29:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4711 3.4 3:226df663:::rbd_data.112d3d611e54.000000000000002d:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:29:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:29:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.90280278, 2024-09-14T05:29:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011304193000000001, 2024-09-14T05:29:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4711 2024-09-14T05:29:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:29:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:29:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167074+0000", 2024-09-14T05:29:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0999999999999997e-07 2024-09-14T05:29:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167078+0000", 2024-09-14T05:29:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.878e-06 2024-09-14T05:29:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167087+0000", 2024-09-14T05:29:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6940000000000007e-06 2024-09-14T05:29:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167093+0000", 2024-09-14T05:29:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7740000000000001e-06 2024-09-14T05:29:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167128+0000", 2024-09-14T05:29:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5231999999999997e-05 2024-09-14T05:29:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167207+0000", 2024-09-14T05:29:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9036999999999996e-05 2024-09-14T05:29:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.169140+0000", 2024-09-14T05:29:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001932897 2024-09-14T05:29:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178258+0000", 2024-09-14T05:29:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091180819999999996 2024-09-14T05:29:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178314+0000", 2024-09-14T05:29:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5571e-05 2024-09-14T05:29:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178348+0000", 2024-09-14T05:29:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3747000000000001e-05 2024-09-14T05:29:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178377+0000", 2024-09-14T05:29:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9471e-05 2024-09-14T05:29:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4721 3.1 3:86302a6e:::rbd_data.112d3d611e54.0000000000000011:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:29:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:29:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.8708172940000001, 2024-09-14T05:29:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066831750000000004, 2024-09-14T05:29:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4721 2024-09-14T05:29:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:29:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:29:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199059+0000", 2024-09-14T05:29:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.06e-07 2024-09-14T05:29:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199062+0000", 2024-09-14T05:29:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2279999999999998e-06 2024-09-14T05:29:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199072+0000", 2024-09-14T05:29:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5999999999999996e-06 2024-09-14T05:29:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199078+0000", 2024-09-14T05:29:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9109999999999998e-06 2024-09-14T05:29:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199123+0000", 2024-09-14T05:29:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5278000000000003e-05 2024-09-14T05:29:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199202+0000", 2024-09-14T05:29:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8521000000000003e-05 2024-09-14T05:29:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199271+0000", 2024-09-14T05:29:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9857999999999994e-05 2024-09-14T05:29:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199941+0000", 2024-09-14T05:29:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066984599999999998 2024-09-14T05:29:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205653+0000", 2024-09-14T05:29:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057117319999999997 2024-09-14T05:29:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205701+0000", 2024-09-14T05:29:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8254000000000003e-05 2024-09-14T05:29:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205741+0000", 2024-09-14T05:29:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0241000000000003e-05 2024-09-14T05:29:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4819 3.12 3:4e8efced:::rbd_data.112d3d611e54.000000000000000e:head [stat,truncate 1114112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:29:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:29:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7855031349999999, 2024-09-14T05:29:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062422770000000001, 2024-09-14T05:29:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4819 2024-09-14T05:29:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:29:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:29:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284373+0000", 2024-09-14T05:29:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3399999999999998e-07 2024-09-14T05:29:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284377+0000", 2024-09-14T05:29:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8539999999999999e-06 2024-09-14T05:29:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284388+0000", 2024-09-14T05:29:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1328999999999999e-05 2024-09-14T05:29:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284395+0000", 2024-09-14T05:29:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9759999999999998e-06 2024-09-14T05:29:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284723+0000", 2024-09-14T05:29:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032775799999999999 2024-09-14T05:29:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284801+0000", 2024-09-14T05:29:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7562e-05 2024-09-14T05:29:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:29:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284933+0000", 2024-09-14T05:29:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000131989 2024-09-14T05:29:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.287450+0000", 2024-09-14T05:29:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025177699999999999 2024-09-14T05:29:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290531+0000", 2024-09-14T05:29:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0030809959999999999 2024-09-14T05:29:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290588+0000", 2024-09-14T05:29:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6177000000000001e-05 2024-09-14T05:29:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290615+0000", 2024-09-14T05:29:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7132e-05 2024-09-14T05:29:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:7067 3.5 e35/24 3:a65d0cff:::rbd_data.112df7f2fe0a.0000000000000028:head v 35'89, mlcod=34'86)", 2024-09-14T05:29:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:29:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.9271098640000002, 2024-09-14T05:29:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0064513080000000002, 2024-09-14T05:29:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:29:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:29:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:29:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:29:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142771+0000", 2024-09-14T05:29:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8949999999999997e-06 2024-09-14T05:29:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142779+0000", 2024-09-14T05:29:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8699999999999992e-06 2024-09-14T05:29:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142785+0000", 2024-09-14T05:29:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4899999999999997e-06 2024-09-14T05:29:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142843+0000", 2024-09-14T05:29:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7377000000000003e-05 2024-09-14T05:29:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142871+0000", 2024-09-14T05:29:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8305000000000001e-05 2024-09-14T05:29:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149177+0000", 2024-09-14T05:29:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063058639999999996 2024-09-14T05:29:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149217+0000", 2024-09-14T05:29:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0080999999999998e-05 2024-09-14T05:29:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:7066 3.15 e35/24 3:abadb03e:::rbd_data.112df7f2fe0a.0000000000000027:head v 35'25, mlcod=34'22)", 2024-09-14T05:29:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:35.143204+0000", 2024-09-14T05:29:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.9266720490000004, 2024-09-14T05:29:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060484919999999999, 2024-09-14T05:29:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.143204+0000", 2024-09-14T05:29:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.143204+0000", 2024-09-14T05:29:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.143204+0000", 2024-09-14T05:29:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0699999999999998e-07 2024-09-14T05:29:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.143208+0000", 2024-09-14T05:29:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5990000000000004e-06 2024-09-14T05:29:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.143247+0000", 2024-09-14T05:29:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8265e-05 2024-09-14T05:29:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.143255+0000", 2024-09-14T05:29:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3860000000000007e-06 2024-09-14T05:29:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.143455+0000", 2024-09-14T05:29:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019977100000000001 2024-09-14T05:29:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.143477+0000", 2024-09-14T05:29:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2104e-05 2024-09-14T05:29:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149219+0000", 2024-09-14T05:29:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057419180000000004 2024-09-14T05:29:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149252+0000", 2024-09-14T05:29:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3142000000000002e-05 2024-09-14T05:29:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.183 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:40.183 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T05:29:40.184 INFO:teuthology.orchestra.run.smithi071.stdout:leaving image image_client.0-clone9 intact 2024-09-14T05:29:40.185 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:29:40.185 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:40.184+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:40.285 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:40.284+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:40.321 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-14T05:29:40.329 INFO:teuthology.orchestra.run.smithi071.stdout:1099 write 0x6bf4747 thru 0x6bfc6e8 (0x7fa2 bytes) 2024-09-14T05:29:40.337 INFO:teuthology.orchestra.run.smithi071.stdout:1100 write 0x7252147 thru 0x725451f (0x23d9 bytes) 2024-09-14T05:29:40.337 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:40.338 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:40.338 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:29:40.338 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:40.341 INFO:teuthology.orchestra.run.smithi071.stdout:1101 punch from 0x488d1a1 to 0x4899681, (0xc4e0 bytes) 2024-09-14T05:29:40.341 INFO:teuthology.orchestra.run.smithi071.stdout:1102 read 0x2946652 thru 0x2948ae7 (0x2496 bytes) 2024-09-14T05:29:40.342 INFO:teuthology.orchestra.run.smithi071.stdout:1103 write 0xab7157c thru 0xab75ba0 (0x4625 bytes) 2024-09-14T05:29:40.345 INFO:teuthology.orchestra.run.smithi071.stdout:1104 read 0x1bdcb0 thru 0x1c71cd (0x951e bytes) 2024-09-14T05:29:40.348 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:29:40.354 INFO:teuthology.orchestra.run.smithi071.stdout:1108 write 0xe3ce2cb thru 0xe3d73a9 (0x90df bytes) 2024-09-14T05:29:40.357 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:40.358 INFO:teuthology.orchestra.run.smithi071.stdout:1110 read 0x94306f thru 0x947659 (0x45eb bytes) 2024-09-14T05:29:40.358 INFO:teuthology.orchestra.run.smithi071.stdout:1112 read 0xa1ed614 thru 0xa1fa648 (0xd035 bytes) 2024-09-14T05:29:40.358 INFO:teuthology.orchestra.run.smithi071.stdout:1113 write 0x339223a thru 0x339b7f2 (0x95b9 bytes) 2024-09-14T05:29:40.377 INFO:teuthology.orchestra.run.smithi071.stdout:1114 trunc from 0xe3d73aa to 0x2a4927e 2024-09-14T05:29:40.386 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:40.385+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:40.398 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:40.404 INFO:teuthology.orchestra.run.smithi071.stdout:1116 writesame 0x3a89e65 thru 0x3a99529 data_size 0x1(0xf6c5 bytes) 2024-09-14T05:29:40.416 INFO:teuthology.orchestra.run.smithi071.stdout:1117 writesame 0x91e2987 thru 0x91e4cf0 data_size 0x3(0x236a bytes) 2024-09-14T05:29:40.416 INFO:teuthology.orchestra.run.smithi071.stdout:1118 read 0x590bb25 thru 0x5910c5b (0x5137 bytes) 2024-09-14T05:29:40.417 INFO:teuthology.orchestra.run.smithi071.stdout:1119 flatten 2024-09-14T05:29:40.454 INFO:teuthology.orchestra.run.smithi071.stdout:1120 write 0xe94637b thru 0xe951fec (0xbc72 bytes) 2024-09-14T05:29:40.457 INFO:teuthology.orchestra.run.smithi071.stdout:1122 writesame 0x37a2d2a thru 0x37b0d44 data_size 0x1(0xe01b bytes) 2024-09-14T05:29:40.463 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:40.463 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:40.463 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:29:40.463 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:29:40.463 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:40.473 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:29:40.486 INFO:teuthology.orchestra.run.smithi071.stdout:1123 writesame 0x56c02f2 thru 0x56c089a data_size 0x15(0x5a9 bytes) 2024-09-14T05:29:40.489 INFO:teuthology.orchestra.run.smithi071.stdout:1124 write 0xb305572 thru 0xb313cb5 (0xe744 bytes) 2024-09-14T05:29:40.492 INFO:teuthology.orchestra.run.smithi071.stdout:1125 writesame 0xcac2796 thru 0xcac2875 data_size 0x4(0xe0 bytes) 2024-09-14T05:29:40.494 INFO:teuthology.orchestra.run.smithi071.stdout:1126 writesame 0xe54ba76 thru 0xe55a205 data_size 0x5f(0xe790 bytes) 2024-09-14T05:29:40.504 INFO:teuthology.orchestra.run.smithi071.stdout:1127 trunc from 0xe951fed to 0x9345ad1 2024-09-14T05:29:40.514 INFO:teuthology.orchestra.run.smithi071.stdout:1128 writesame 0x55e20cd thru 0x55e7076 data_size 0x21(0x4faa bytes) 2024-09-14T05:29:40.517 INFO:teuthology.orchestra.run.smithi071.stdout:1129 read 0x4ed202e thru 0x4ed8256 (0x6229 bytes) 2024-09-14T05:29:40.517 INFO:teuthology.orchestra.run.smithi071.stdout:1130 read 0x6577192 thru 0x65819ee (0xa85d bytes) 2024-09-14T05:29:40.518 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:40.517+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:40.522 INFO:teuthology.orchestra.run.smithi071.stdout:1131 trunc from 0x9345ad1 to 0xd53fb7e 2024-09-14T05:29:40.540 INFO:teuthology.orchestra.run.smithi071.stdout:1132 trunc from 0xd53fb7e to 0x3bb59e1 2024-09-14T05:29:40.565 INFO:teuthology.orchestra.run.smithi071.stdout:1133 write 0xb6a2bb6 thru 0xb6ab00b (0x8456 bytes) 2024-09-14T05:29:40.568 INFO:teuthology.orchestra.run.smithi071.stdout:1134 writesame 0x5cf0039 thru 0x5cfdfc4 data_size 0x72(0xdf8c bytes) 2024-09-14T05:29:40.571 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:40.571 INFO:teuthology.orchestra.run.smithi071.stdout:1136 read 0x57f403e thru 0x57f6ffe (0x2fc1 bytes) 2024-09-14T05:29:40.572 INFO:teuthology.orchestra.run.smithi071.stdout:1137 writesame 0xb28c422 thru 0xb29b651 data_size 0xc8(0xf230 bytes) 2024-09-14T05:29:40.575 INFO:teuthology.orchestra.run.smithi071.stdout:1138 read 0x9d19e6a thru 0x9d2429f (0xa436 bytes) 2024-09-14T05:29:40.575 INFO:teuthology.orchestra.run.smithi071.stdout:1139 read 0x29ac4f8 thru 0x29b1d2a (0x5833 bytes) 2024-09-14T05:29:40.575 INFO:teuthology.orchestra.run.smithi071.stdout:1140 writesame 0x787d0a7 thru 0x788c308 data_size 0x92(0xf262 bytes) 2024-09-14T05:29:40.579 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:40.587 INFO:teuthology.orchestra.run.smithi071.stdout:1142 write 0xd60d719 thru 0xd61154a (0x3e32 bytes) 2024-09-14T05:29:40.588 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:40.587+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:40.589 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:40.589 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:29:40.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:29:40.589 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:29:40.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.589 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2131 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:40.589 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:29:40.590 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.8127158360000006, 2024-09-14T05:29:40.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049526593000000001, 2024-09-14T05:29:40.590 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.590 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.590 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:40.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.590 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2131 2024-09-14T05:29:40.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.590 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:29:40.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:29:40.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774867+0000", 2024-09-14T05:29:40.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4430000000000002e-06 2024-09-14T05:29:40.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790357+0000", 2024-09-14T05:29:40.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015489466 2024-09-14T05:29:40.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790383+0000", 2024-09-14T05:29:40.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6247000000000001e-05 2024-09-14T05:29:40.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790394+0000", 2024-09-14T05:29:40.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0728e-05 2024-09-14T05:29:40.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790471+0000", 2024-09-14T05:29:40.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7539000000000004e-05 2024-09-14T05:29:40.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790639+0000", 2024-09-14T05:29:40.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016787700000000001 2024-09-14T05:29:40.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-14T05:29:40.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790998+0000", 2024-09-14T05:29:40.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035931299999999999 2024-09-14T05:29:40.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:40.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.823529+0000", 2024-09-14T05:29:40.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032530798 2024-09-14T05:29:40.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:40.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824324+0000", 2024-09-14T05:29:40.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00079442399999999995 2024-09-14T05:29:40.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:40.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824362+0000", 2024-09-14T05:29:40.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8398000000000001e-05 2024-09-14T05:29:40.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824391+0000", 2024-09-14T05:29:40.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.936e-05 2024-09-14T05:29:40.595 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.595 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.595 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.595 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3892 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7602176~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.595 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:29:40.595 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2311378380000004, 2024-09-14T05:29:40.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047082951999999997, 2024-09-14T05:29:40.596 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.596 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.596 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.596 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3892 2024-09-14T05:29:40.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.596 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:29:40.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:29:40.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:29:40.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0199999999999998e-07 2024-09-14T05:29:40.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356454+0000", 2024-09-14T05:29:40.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0414999999999999e-05 2024-09-14T05:29:40.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356463+0000", 2024-09-14T05:29:40.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6739999999999992e-06 2024-09-14T05:29:40.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356468+0000", 2024-09-14T05:29:40.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5680000000000001e-06 2024-09-14T05:29:40.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403143+0000", 2024-09-14T05:29:40.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046675051000000002 2024-09-14T05:29:40.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403206+0000", 2024-09-14T05:29:40.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3127999999999996e-05 2024-09-14T05:29:40.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403526+0000", 2024-09-14T05:29:40.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031981400000000002 2024-09-14T05:29:40.599 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.599 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.599 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.599 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3893 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8093696~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.599 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:29:40.600 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2310930009999996, 2024-09-14T05:29:40.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047516305000000002, 2024-09-14T05:29:40.600 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.600 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.600 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.600 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.600 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.600 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.600 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3893 2024-09-14T05:29:40.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.600 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:29:40.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:29:40.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:29:40.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6899999999999997e-07 2024-09-14T05:29:40.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356491+0000", 2024-09-14T05:29:40.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8109999999999999e-06 2024-09-14T05:29:40.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356498+0000", 2024-09-14T05:29:40.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5320000000000003e-06 2024-09-14T05:29:40.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356502+0000", 2024-09-14T05:29:40.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000002e-06 2024-09-14T05:29:40.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403579+0000", 2024-09-14T05:29:40.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047076706000000003 2024-09-14T05:29:40.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403664+0000", 2024-09-14T05:29:40.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5240000000000004e-05 2024-09-14T05:29:40.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404004+0000", 2024-09-14T05:29:40.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033974500000000002 2024-09-14T05:29:40.605 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.605 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.605 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.605 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3993 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 262144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.605 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:29:40.605 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2310711300000001, 2024-09-14T05:29:40.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048088476999999998, 2024-09-14T05:29:40.605 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.605 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.605 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.606 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3993 2024-09-14T05:29:40.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.606 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:29:40.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:29:40.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:29:40.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.46e-07 2024-09-14T05:29:40.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356512+0000", 2024-09-14T05:29:40.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.827e-06 2024-09-14T05:29:40.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356518+0000", 2024-09-14T05:29:40.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7780000000000004e-06 2024-09-14T05:29:40.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356522+0000", 2024-09-14T05:29:40.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.865e-06 2024-09-14T05:29:40.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404036+0000", 2024-09-14T05:29:40.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047513725999999999 2024-09-14T05:29:40.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404135+0000", 2024-09-14T05:29:40.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9352999999999996e-05 2024-09-14T05:29:40.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404598+0000", 2024-09-14T05:29:40.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046258200000000001 2024-09-14T05:29:40.609 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.609 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.609 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.609 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3994 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.609 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:29:40.609 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2310291180000004, 2024-09-14T05:29:40.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048274625000000002, 2024-09-14T05:29:40.610 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.610 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.610 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.610 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3994 2024-09-14T05:29:40.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.610 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:29:40.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:29:40.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:29:40.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3799999999999997e-07 2024-09-14T05:29:40.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356554+0000", 2024-09-14T05:29:40.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2259999999999999e-06 2024-09-14T05:29:40.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356560+0000", 2024-09-14T05:29:40.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-14T05:29:40.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356566+0000", 2024-09-14T05:29:40.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3909999999999997e-06 2024-09-14T05:29:40.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404632+0000", 2024-09-14T05:29:40.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048066154 2024-09-14T05:29:40.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404719+0000", 2024-09-14T05:29:40.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7440000000000003e-05 2024-09-14T05:29:40.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404826+0000", 2024-09-14T05:29:40.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106813 2024-09-14T05:29:40.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.614 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.614 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3995 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.614 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:29:40.614 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2310039790000005, 2024-09-14T05:29:40.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048828209999999997, 2024-09-14T05:29:40.614 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.615 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.615 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.615 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3995 2024-09-14T05:29:40.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.615 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:29:40.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:29:40.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:29:40.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6599999999999996e-07 2024-09-14T05:29:40.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356579+0000", 2024-09-14T05:29:40.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0719999999999998e-06 2024-09-14T05:29:40.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356585+0000", 2024-09-14T05:29:40.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.959e-06 2024-09-14T05:29:40.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356590+0000", 2024-09-14T05:29:40.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.464e-06 2024-09-14T05:29:40.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404875+0000", 2024-09-14T05:29:40.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048285649999999999 2024-09-14T05:29:40.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404985+0000", 2024-09-14T05:29:40.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109417 2024-09-14T05:29:40.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405405+0000", 2024-09-14T05:29:40.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042008200000000001 2024-09-14T05:29:40.619 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.619 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.620 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.620 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3996 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.620 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:29:40.620 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2306950959999998, 2024-09-14T05:29:40.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049061595, 2024-09-14T05:29:40.621 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.621 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.621 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.621 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.621 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.621 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.622 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3996 2024-09-14T05:29:40.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.622 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:29:40.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:29:40.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:29:40.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.61e-07 2024-09-14T05:29:40.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356892+0000", 2024-09-14T05:29:40.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9050000000000002e-06 2024-09-14T05:29:40.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356901+0000", 2024-09-14T05:29:40.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4149999999999999e-06 2024-09-14T05:29:40.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356907+0000", 2024-09-14T05:29:40.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0140000000000004e-06 2024-09-14T05:29:40.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405448+0000", 2024-09-14T05:29:40.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048541647 2024-09-14T05:29:40.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405542+0000", 2024-09-14T05:29:40.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3639000000000003e-05 2024-09-14T05:29:40.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405947+0000", 2024-09-14T05:29:40.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040521399999999999 2024-09-14T05:29:40.629 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.629 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.630 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.630 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3997 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2228224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.630 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:29:40.630 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2304811139999998, 2024-09-14T05:29:40.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049308774999999999, 2024-09-14T05:29:40.631 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.631 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.631 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.631 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3997 2024-09-14T05:29:40.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.632 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:29:40.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:29:40.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:29:40.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6499999999999999e-07 2024-09-14T05:29:40.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357104+0000", 2024-09-14T05:29:40.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4989999999999999e-06 2024-09-14T05:29:40.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357112+0000", 2024-09-14T05:29:40.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4279999999999996e-06 2024-09-14T05:29:40.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357117+0000", 2024-09-14T05:29:40.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6190000000000002e-06 2024-09-14T05:29:40.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405989+0000", 2024-09-14T05:29:40.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048872080999999998 2024-09-14T05:29:40.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406074+0000", 2024-09-14T05:29:40.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4676e-05 2024-09-14T05:29:40.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406408+0000", 2024-09-14T05:29:40.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033480700000000001 2024-09-14T05:29:40.636 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.636 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.636 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.636 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3998 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.637 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:29:40.637 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2301596850000003, 2024-09-14T05:29:40.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049462314, 2024-09-14T05:29:40.637 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.637 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.637 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.637 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.637 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.637 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.637 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3998 2024-09-14T05:29:40.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.637 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:29:40.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:29:40.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357422+0000", 2024-09-14T05:29:40.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3099999999999996e-07 2024-09-14T05:29:40.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357429+0000", 2024-09-14T05:29:40.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2880000000000004e-06 2024-09-14T05:29:40.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357439+0000", 2024-09-14T05:29:40.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.302e-06 2024-09-14T05:29:40.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357445+0000", 2024-09-14T05:29:40.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4269999999999996e-06 2024-09-14T05:29:40.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406448+0000", 2024-09-14T05:29:40.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049003096000000003 2024-09-14T05:29:40.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406537+0000", 2024-09-14T05:29:40.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9135999999999995e-05 2024-09-14T05:29:40.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406883+0000", 2024-09-14T05:29:40.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000346134 2024-09-14T05:29:40.641 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.641 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.641 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.641 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3999 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.641 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:29:40.641 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.230089328, 2024-09-14T05:29:40.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049797556, 2024-09-14T05:29:40.642 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.642 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.642 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.642 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.642 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.642 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.642 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3999 2024-09-14T05:29:40.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.642 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:29:40.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:29:40.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357492+0000", 2024-09-14T05:29:40.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8299999999999996e-07 2024-09-14T05:29:40.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357495+0000", 2024-09-14T05:29:40.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0450000000000001e-06 2024-09-14T05:29:40.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357503+0000", 2024-09-14T05:29:40.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9689999999999994e-06 2024-09-14T05:29:40.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357510+0000", 2024-09-14T05:29:40.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-14T05:29:40.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406914+0000", 2024-09-14T05:29:40.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049404293000000002 2024-09-14T05:29:40.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406965+0000", 2024-09-14T05:29:40.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0470999999999998e-05 2024-09-14T05:29:40.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407289+0000", 2024-09-14T05:29:40.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032434899999999998 2024-09-14T05:29:40.645 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.645 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.646 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.646 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4000 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3702784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.646 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:29:40.646 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2297429829999995, 2024-09-14T05:29:40.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049936728999999999, 2024-09-14T05:29:40.646 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.646 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.646 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.646 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.646 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.647 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.647 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4000 2024-09-14T05:29:40.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.647 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:29:40.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:29:40.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357839+0000", 2024-09-14T05:29:40.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-14T05:29:40.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357842+0000", 2024-09-14T05:29:40.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4850000000000001e-06 2024-09-14T05:29:40.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357851+0000", 2024-09-14T05:29:40.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1819999999999995e-06 2024-09-14T05:29:40.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357859+0000", 2024-09-14T05:29:40.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8129999999999996e-06 2024-09-14T05:29:40.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407337+0000", 2024-09-14T05:29:40.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049478084999999998 2024-09-14T05:29:40.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407442+0000", 2024-09-14T05:29:40.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000105302 2024-09-14T05:29:40.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407774+0000", 2024-09-14T05:29:40.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033200100000000001 2024-09-14T05:29:40.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.651 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.651 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4001 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.651 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:29:40.651 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2296476280000004, 2024-09-14T05:29:40.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050304205999999997, 2024-09-14T05:29:40.651 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.651 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.652 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.652 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4001 2024-09-14T05:29:40.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.652 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:29:40.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:29:40.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357934+0000", 2024-09-14T05:29:40.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000004e-07 2024-09-14T05:29:40.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357937+0000", 2024-09-14T05:29:40.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0869999999999998e-06 2024-09-14T05:29:40.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357946+0000", 2024-09-14T05:29:40.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1439999999999992e-06 2024-09-14T05:29:40.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357952+0000", 2024-09-14T05:29:40.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.609e-06 2024-09-14T05:29:40.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407820+0000", 2024-09-14T05:29:40.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049868580000000003 2024-09-14T05:29:40.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407909+0000", 2024-09-14T05:29:40.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8887000000000005e-05 2024-09-14T05:29:40.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408237+0000", 2024-09-14T05:29:40.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032824599999999997 2024-09-14T05:29:40.656 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.656 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.656 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.656 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4002 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.656 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:29:40.657 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2294310330000009, 2024-09-14T05:29:40.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050539529999999999, 2024-09-14T05:29:40.657 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.657 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.657 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.657 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.657 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.657 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.657 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4002 2024-09-14T05:29:40.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.657 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:29:40.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:29:40.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:29:40.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5099999999999999e-07 2024-09-14T05:29:40.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358154+0000", 2024-09-14T05:29:40.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6799999999999999e-06 2024-09-14T05:29:40.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358165+0000", 2024-09-14T05:29:40.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0899e-05 2024-09-14T05:29:40.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358172+0000", 2024-09-14T05:29:40.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8689999999999998e-06 2024-09-14T05:29:40.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408281+0000", 2024-09-14T05:29:40.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050108699 2024-09-14T05:29:40.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408369+0000", 2024-09-14T05:29:40.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8542999999999996e-05 2024-09-14T05:29:40.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408689+0000", 2024-09-14T05:29:40.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032008900000000002 2024-09-14T05:29:40.661 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.661 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.661 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.661 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4003 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5177344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.661 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:29:40.661 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2293911980000001, 2024-09-14T05:29:40.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050868759, 2024-09-14T05:29:40.662 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.662 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.662 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.662 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.662 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.662 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.662 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4003 2024-09-14T05:29:40.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.662 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:29:40.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:29:40.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:29:40.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-07 2024-09-14T05:29:40.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358193+0000", 2024-09-14T05:29:40.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.588e-06 2024-09-14T05:29:40.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358204+0000", 2024-09-14T05:29:40.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1525999999999999e-05 2024-09-14T05:29:40.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358213+0000", 2024-09-14T05:29:40.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5010000000000004e-06 2024-09-14T05:29:40.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408719+0000", 2024-09-14T05:29:40.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050506708999999997 2024-09-14T05:29:40.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408766+0000", 2024-09-14T05:29:40.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6579999999999998e-05 2024-09-14T05:29:40.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409058+0000", 2024-09-14T05:29:40.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000292365 2024-09-14T05:29:40.666 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.666 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.666 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.666 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4004 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5668864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.666 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:29:40.666 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2289683549999992, 2024-09-14T05:29:40.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050839848, 2024-09-14T05:29:40.667 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.667 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.667 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.667 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.667 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.667 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.667 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4004 2024-09-14T05:29:40.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.667 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:29:40.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:29:40.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358613+0000", 2024-09-14T05:29:40.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:29:40.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358618+0000", 2024-09-14T05:29:40.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6009999999999997e-06 2024-09-14T05:29:40.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358630+0000", 2024-09-14T05:29:40.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2122e-05 2024-09-14T05:29:40.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358638+0000", 2024-09-14T05:29:40.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3969999999999994e-06 2024-09-14T05:29:40.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409088+0000", 2024-09-14T05:29:40.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050449565000000002 2024-09-14T05:29:40.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409153+0000", 2024-09-14T05:29:40.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4728999999999996e-05 2024-09-14T05:29:40.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409452+0000", 2024-09-14T05:29:40.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029958900000000001 2024-09-14T05:29:40.670 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.671 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.671 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.671 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4005 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6160384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.671 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:29:40.671 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2287514210000001, 2024-09-14T05:29:40.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051054752000000002, 2024-09-14T05:29:40.671 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.671 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.671 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.671 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.671 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.672 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.672 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4005 2024-09-14T05:29:40.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.672 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:29:40.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:29:40.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358830+0000", 2024-09-14T05:29:40.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-14T05:29:40.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358833+0000", 2024-09-14T05:29:40.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9529999999999999e-06 2024-09-14T05:29:40.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358843+0000", 2024-09-14T05:29:40.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8069999999999994e-06 2024-09-14T05:29:40.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358849+0000", 2024-09-14T05:29:40.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1040000000000003e-06 2024-09-14T05:29:40.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409487+0000", 2024-09-14T05:29:40.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050638626999999999 2024-09-14T05:29:40.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409535+0000", 2024-09-14T05:29:40.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7525000000000003e-05 2024-09-14T05:29:40.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409884+0000", 2024-09-14T05:29:40.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034914800000000003 2024-09-14T05:29:40.675 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.675 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.675 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.675 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4006 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.675 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:29:40.676 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2287143090000008, 2024-09-14T05:29:40.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051425323000000002, 2024-09-14T05:29:40.676 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.676 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.676 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.676 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.676 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.676 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.676 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4006 2024-09-14T05:29:40.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.676 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:29:40.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:29:40.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358867+0000", 2024-09-14T05:29:40.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2900000000000004e-07 2024-09-14T05:29:40.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358869+0000", 2024-09-14T05:29:40.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4210000000000002e-06 2024-09-14T05:29:40.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358876+0000", 2024-09-14T05:29:40.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4980000000000003e-06 2024-09-14T05:29:40.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358881+0000", 2024-09-14T05:29:40.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3090000000000003e-06 2024-09-14T05:29:40.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409915+0000", 2024-09-14T05:29:40.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051033348999999999 2024-09-14T05:29:40.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409964+0000", 2024-09-14T05:29:40.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9858000000000003e-05 2024-09-14T05:29:40.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410292+0000", 2024-09-14T05:29:40.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032735899999999999 2024-09-14T05:29:40.680 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.680 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.680 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.680 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4007 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.680 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:29:40.680 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2286917160000002, 2024-09-14T05:29:40.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05184076, 2024-09-14T05:29:40.681 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.681 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.681 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.681 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.681 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.681 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.681 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4007 2024-09-14T05:29:40.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.681 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:29:40.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:29:40.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:29:40.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:29:40.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358891+0000", 2024-09-14T05:29:40.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9819999999999998e-06 2024-09-14T05:29:40.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358897+0000", 2024-09-14T05:29:40.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9390000000000002e-06 2024-09-14T05:29:40.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358902+0000", 2024-09-14T05:29:40.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6770000000000003e-06 2024-09-14T05:29:40.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410335+0000", 2024-09-14T05:29:40.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051432991999999997 2024-09-14T05:29:40.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410416+0000", 2024-09-14T05:29:40.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1316999999999994e-05 2024-09-14T05:29:40.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410730+0000", 2024-09-14T05:29:40.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000313419 2024-09-14T05:29:40.685 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.685 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.685 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.685 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4008 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7634944~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.685 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:29:40.685 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2286673550000007, 2024-09-14T05:29:40.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052208082000000003, 2024-09-14T05:29:40.686 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.686 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.686 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.686 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.686 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.686 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.686 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4008 2024-09-14T05:29:40.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.686 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:29:40.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:29:40.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358914+0000", 2024-09-14T05:29:40.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6100000000000001e-07 2024-09-14T05:29:40.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358916+0000", 2024-09-14T05:29:40.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.902e-06 2024-09-14T05:29:40.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358923+0000", 2024-09-14T05:29:40.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4780000000000002e-06 2024-09-14T05:29:40.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358929+0000", 2024-09-14T05:29:40.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0970000000000002e-06 2024-09-14T05:29:40.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410761+0000", 2024-09-14T05:29:40.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051831436000000002 2024-09-14T05:29:40.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410829+0000", 2024-09-14T05:29:40.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7874999999999996e-05 2024-09-14T05:29:40.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411121+0000", 2024-09-14T05:29:40.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029283299999999999 2024-09-14T05:29:40.689 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.689 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.690 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.690 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4009 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8126464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:40.690 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:29:40.690 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2285201299999997, 2024-09-14T05:29:40.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052518169000000003, 2024-09-14T05:29:40.690 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:40.690 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:40.690 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:40.690 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:40.690 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:40.690 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:40.690 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4009 2024-09-14T05:29:40.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.691 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:40.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:40.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:29:40.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:40.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:29:40.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:40.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:40.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:29:40.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0200000000000002e-07 2024-09-14T05:29:40.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:40.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359066+0000", 2024-09-14T05:29:40.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7899999999999999e-06 2024-09-14T05:29:40.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:40.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359074+0000", 2024-09-14T05:29:40.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2579999999999995e-06 2024-09-14T05:29:40.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:40.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359082+0000", 2024-09-14T05:29:40.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.464e-06 2024-09-14T05:29:40.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:40.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411160+0000", 2024-09-14T05:29:40.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052078078 2024-09-14T05:29:40.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:40.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411244+0000", 2024-09-14T05:29:40.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4041000000000004e-05 2024-09-14T05:29:40.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:40.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:40.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:40.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411579+0000", 2024-09-14T05:29:40.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033493599999999998 2024-09-14T05:29:40.694 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.694 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.694 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.694 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:40.694 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:40.694 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:40.696 INFO:teuthology.orchestra.run.smithi071.stdout:1143 read 0x89fe9fc thru 0x8a0da04 (0xf009 bytes) 2024-09-14T05:29:40.696 INFO:teuthology.orchestra.run.smithi071.stdout:1144 writesame 0x4dbf886 thru 0x4dc3cb4 data_size 0x1(0x442f bytes) 2024-09-14T05:29:40.696 INFO:teuthology.orchestra.run.smithi071.stdout:1145 write 0x4383556 thru 0x438d61f (0xa0ca bytes) 2024-09-14T05:29:40.696 INFO:teuthology.orchestra.run.smithi071.stdout:1147 punch from 0x4b8b4dd to 0x4b98166, (0xcc89 bytes) 2024-09-14T05:29:40.696 INFO:teuthology.orchestra.run.smithi071.stdout:1148 clone 11 order 24 su 8388608 sc 4 2024-09-14T05:29:40.697 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:40.695+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:40.697 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:29:40.797 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:40.796+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:40.801 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:40.801 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:40.801 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:29:40.801 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:40.810 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:29:40.898 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:40.897+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:40.951 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:40.951 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:40.951 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:40.951 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:40.951 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:40.960 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:29:40.998 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:40.998+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:41.064 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:41.065 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:41.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:41.065 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:41.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.065 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1163 3.f 3:f5d44725:::rbd_data.112dd25fb4c.0000000000000076:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:41.065 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.177281+0000", 2024-09-14T05:29:41.065 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 10.885832436999999, 2024-09-14T05:29:41.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0080900699999999996, 2024-09-14T05:29:41.065 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.065 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.066 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.066 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.066 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.066 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.066 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1163 2024-09-14T05:29:41.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.066 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177281+0000", 2024-09-14T05:29:41.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.067 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.067 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177281+0000", 2024-09-14T05:29:41.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.067 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.067 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177283+0000", 2024-09-14T05:29:41.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.429e-06 2024-09-14T05:29:41.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.067 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.067 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177288+0000", 2024-09-14T05:29:41.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5759999999999996e-06 2024-09-14T05:29:41.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177301+0000", 2024-09-14T05:29:41.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3149999999999999e-05 2024-09-14T05:29:41.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177307+0000", 2024-09-14T05:29:41.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.643e-06 2024-09-14T05:29:41.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177358+0000", 2024-09-14T05:29:41.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1523000000000001e-05 2024-09-14T05:29:41.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.069 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.069 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.177440+0000", 2024-09-14T05:29:41.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2037000000000001e-05 2024-09-14T05:29:41.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.069 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.069 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.185371+0000", 2024-09-14T05:29:41.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0079307119999999995 2024-09-14T05:29:41.069 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.069 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.069 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.070 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1197 3.f 3:f1ddeb6e:::rbd_data.112dd25fb4c.0000000000000082:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:41.070 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.194432+0000", 2024-09-14T05:29:41.070 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 10.868681188, 2024-09-14T05:29:41.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0081070949999999999, 2024-09-14T05:29:41.070 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.070 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.070 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:41.070 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.070 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.070 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.070 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1197 2024-09-14T05:29:41.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.070 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194432+0000", 2024-09-14T05:29:41.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194432+0000", 2024-09-14T05:29:41.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194433+0000", 2024-09-14T05:29:41.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.099e-06 2024-09-14T05:29:41.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194440+0000", 2024-09-14T05:29:41.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7950000000000002e-06 2024-09-14T05:29:41.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194453+0000", 2024-09-14T05:29:41.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2529e-05 2024-09-14T05:29:41.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194460+0000", 2024-09-14T05:29:41.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6690000000000006e-06 2024-09-14T05:29:41.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.073 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.073 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194508+0000", 2024-09-14T05:29:41.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7182000000000003e-05 2024-09-14T05:29:41.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.073 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.073 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194584+0000", 2024-09-14T05:29:41.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6588999999999995e-05 2024-09-14T05:29:41.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.073 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-14T05:29:41.074 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.194795+0000", 2024-09-14T05:29:41.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021061499999999999 2024-09-14T05:29:41.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.074 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.074 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:41.074 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202114+0000", 2024-09-14T05:29:41.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0073192869999999998 2024-09-14T05:29:41.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.074 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.074 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:41.074 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202443+0000", 2024-09-14T05:29:41.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00032850000000000002 2024-09-14T05:29:41.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.074 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.074 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:41.074 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202496+0000", 2024-09-14T05:29:41.075 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3560000000000002e-05 2024-09-14T05:29:41.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.075 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.075 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202539+0000", 2024-09-14T05:29:41.075 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3269999999999997e-05 2024-09-14T05:29:41.075 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.075 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.075 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.075 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1206 3.3 3:c4477d03:::rbd_data.112dd25fb4c.0000000000000089:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:41.075 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.198115+0000", 2024-09-14T05:29:41.075 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 10.864998944, 2024-09-14T05:29:41.075 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0083903599999999995, 2024-09-14T05:29:41.076 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.076 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.076 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.076 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.076 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.076 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.076 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1206 2024-09-14T05:29:41.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.076 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.076 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.076 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.198115+0000", 2024-09-14T05:29:41.076 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.077 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.198115+0000", 2024-09-14T05:29:41.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.077 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.077 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.198116+0000", 2024-09-14T05:29:41.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.031e-06 2024-09-14T05:29:41.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.077 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.077 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.198120+0000", 2024-09-14T05:29:41.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7389999999999999e-06 2024-09-14T05:29:41.078 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.078 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.078 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.078 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.198141+0000", 2024-09-14T05:29:41.078 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0922000000000001e-05 2024-09-14T05:29:41.078 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.078 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.078 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.078 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.198148+0000", 2024-09-14T05:29:41.078 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1570000000000003e-06 2024-09-14T05:29:41.078 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.078 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.078 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.078 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.198209+0000", 2024-09-14T05:29:41.079 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0103999999999999e-05 2024-09-14T05:29:41.079 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.079 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.079 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.079 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.198319+0000", 2024-09-14T05:29:41.079 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000110566 2024-09-14T05:29:41.079 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.079 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.079 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.079 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.206505+0000", 2024-09-14T05:29:41.079 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0081858409999999993 2024-09-14T05:29:41.079 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.079 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.079 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.079 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.080 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.080 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1209 3.3 3:c1bb0aea:::rbd_data.112dd25fb4c.000000000000008b:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:41.080 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.202565+0000", 2024-09-14T05:29:41.080 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 10.860548101999999, 2024-09-14T05:29:41.080 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.012188134, 2024-09-14T05:29:41.080 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.080 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.080 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.080 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.080 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.080 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.080 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1209 2024-09-14T05:29:41.080 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.081 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.081 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.081 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.081 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202565+0000", 2024-09-14T05:29:41.081 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.081 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.081 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.081 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.081 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202565+0000", 2024-09-14T05:29:41.081 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.081 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.081 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.082 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.082 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202566+0000", 2024-09-14T05:29:41.082 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.051e-06 2024-09-14T05:29:41.082 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.082 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.082 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.082 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202571+0000", 2024-09-14T05:29:41.082 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.369e-06 2024-09-14T05:29:41.082 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.082 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.082 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.082 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202580+0000", 2024-09-14T05:29:41.082 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6969999999999999e-06 2024-09-14T05:29:41.082 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.082 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.083 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.083 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202586+0000", 2024-09-14T05:29:41.083 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.224e-06 2024-09-14T05:29:41.083 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.083 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.083 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.083 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202635+0000", 2024-09-14T05:29:41.083 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9126999999999999e-05 2024-09-14T05:29:41.083 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.083 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.083 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.083 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202755+0000", 2024-09-14T05:29:41.083 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012050899999999999 2024-09-14T05:29:41.083 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.083 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.083 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.084 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.214754+0000", 2024-09-14T05:29:41.084 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.011998157000000001 2024-09-14T05:29:41.084 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.084 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.084 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.084 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.084 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.084 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1211 3.c 3:305f1e23:::rbd_data.112dd25fb4c.000000000000008c:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:41.084 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.202765+0000", 2024-09-14T05:29:41.084 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 10.860348525999999, 2024-09-14T05:29:41.084 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0080686799999999999, 2024-09-14T05:29:41.084 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.084 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.084 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.084 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.085 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.085 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.085 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1211 2024-09-14T05:29:41.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.085 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.085 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.085 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.085 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202765+0000", 2024-09-14T05:29:41.085 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.085 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.085 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.085 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202765+0000", 2024-09-14T05:29:41.085 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.086 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.086 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.086 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.086 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202766+0000", 2024-09-14T05:29:41.086 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5000000000000001e-07 2024-09-14T05:29:41.086 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.086 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.086 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.086 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202773+0000", 2024-09-14T05:29:41.086 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7519999999999999e-06 2024-09-14T05:29:41.086 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.086 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.086 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.087 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202782+0000", 2024-09-14T05:29:41.087 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3370000000000006e-06 2024-09-14T05:29:41.087 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.087 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.087 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.087 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202792+0000", 2024-09-14T05:29:41.087 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9669999999999996e-06 2024-09-14T05:29:41.087 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.087 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.087 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.087 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202853+0000", 2024-09-14T05:29:41.087 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1310999999999996e-05 2024-09-14T05:29:41.087 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.087 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.087 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.088 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.202978+0000", 2024-09-14T05:29:41.088 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000124323 2024-09-14T05:29:41.088 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.088 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.088 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.088 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.210834+0000", 2024-09-14T05:29:41.088 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0078561399999999993 2024-09-14T05:29:41.088 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.088 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.088 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.088 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.088 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.088 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:1214 3.13 e30/24 3:ce6c3324:::rbd_data.112dd25fb4c.0000000000000085:head v 30'14, mlcod=30'12)", 2024-09-14T05:29:41.088 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.203370+0000", 2024-09-14T05:29:41.089 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 10.859743713, 2024-09-14T05:29:41.089 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0074113449999999997, 2024-09-14T05:29:41.089 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.089 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.089 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:41.089 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.089 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.089 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.089 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203370+0000", 2024-09-14T05:29:41.089 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.089 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.089 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.089 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.089 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203370+0000", 2024-09-14T05:29:41.090 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.090 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.090 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.090 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.090 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203370+0000", 2024-09-14T05:29:41.090 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4499999999999997e-07 2024-09-14T05:29:41.090 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.090 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.090 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.090 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203379+0000", 2024-09-14T05:29:41.090 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3410000000000003e-06 2024-09-14T05:29:41.090 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.090 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.090 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.090 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203390+0000", 2024-09-14T05:29:41.091 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1267000000000001e-05 2024-09-14T05:29:41.091 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.091 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.091 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.091 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203397+0000", 2024-09-14T05:29:41.091 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5270000000000004e-06 2024-09-14T05:29:41.091 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.091 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.091 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.091 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203443+0000", 2024-09-14T05:29:41.091 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6434999999999998e-05 2024-09-14T05:29:41.091 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.091 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.091 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.091 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.203466+0000", 2024-09-14T05:29:41.092 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2540000000000001e-05 2024-09-14T05:29:41.092 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.092 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.092 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:41.092 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.210738+0000", 2024-09-14T05:29:41.092 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0072729140000000001 2024-09-14T05:29:41.092 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.092 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.092 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.092 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.210781+0000", 2024-09-14T05:29:41.092 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2676000000000003e-05 2024-09-14T05:29:41.092 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.092 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.093 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.093 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.093 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.093 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1219 3.3 3:c4477d03:::rbd_data.112dd25fb4c.0000000000000089:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:41.093 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.206611+0000", 2024-09-14T05:29:41.093 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 10.856502835000001, 2024-09-14T05:29:41.093 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.012919409999999999, 2024-09-14T05:29:41.093 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.093 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.093 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:41.093 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.093 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.093 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.093 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1219 2024-09-14T05:29:41.094 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.094 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.094 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.094 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.094 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.206611+0000", 2024-09-14T05:29:41.094 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.094 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.094 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.094 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.094 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.206611+0000", 2024-09-14T05:29:41.094 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.094 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.094 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.094 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.094 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.206611+0000", 2024-09-14T05:29:41.094 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3499999999999995e-07 2024-09-14T05:29:41.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.095 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.095 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.095 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.206616+0000", 2024-09-14T05:29:41.095 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2220000000000004e-06 2024-09-14T05:29:41.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.095 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.095 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.095 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.206625+0000", 2024-09-14T05:29:41.095 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8780000000000004e-06 2024-09-14T05:29:41.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.095 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.095 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.095 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.206630+0000", 2024-09-14T05:29:41.095 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1479999999999997e-06 2024-09-14T05:29:41.096 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.096 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.096 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.096 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.206678+0000", 2024-09-14T05:29:41.096 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8699999999999998e-05 2024-09-14T05:29:41.096 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.096 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.096 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.096 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.206736+0000", 2024-09-14T05:29:41.096 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7553000000000003e-05 2024-09-14T05:29:41.096 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.096 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.096 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:41.096 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.206857+0000", 2024-09-14T05:29:41.096 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012155300000000001 2024-09-14T05:29:41.097 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.097 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.097 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:41.097 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.214763+0000", 2024-09-14T05:29:41.097 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0079053950000000008 2024-09-14T05:29:41.097 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.097 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.097 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:41.097 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.219444+0000", 2024-09-14T05:29:41.097 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0046810280000000003 2024-09-14T05:29:41.097 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.097 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.097 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:41.097 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.219487+0000", 2024-09-14T05:29:41.097 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2815999999999998e-05 2024-09-14T05:29:41.097 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.219530+0000", 2024-09-14T05:29:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3381999999999999e-05 2024-09-14T05:29:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1222 3.c 3:305f1e23:::rbd_data.112dd25fb4c.000000000000008c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.211001+0000", 2024-09-14T05:29:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 10.852112626, 2024-09-14T05:29:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0084031929999999998, 2024-09-14T05:29:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1222 2024-09-14T05:29:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211001+0000", 2024-09-14T05:29:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211001+0000", 2024-09-14T05:29:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211002+0000", 2024-09-14T05:29:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6500000000000008e-07 2024-09-14T05:29:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211007+0000", 2024-09-14T05:29:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2320000000000001e-06 2024-09-14T05:29:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211017+0000", 2024-09-14T05:29:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3889999999999994e-06 2024-09-14T05:29:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211024+0000", 2024-09-14T05:29:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.272e-06 2024-09-14T05:29:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211071+0000", 2024-09-14T05:29:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7465e-05 2024-09-14T05:29:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211129+0000", 2024-09-14T05:29:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7865000000000002e-05 2024-09-14T05:29:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 7", 2024-09-14T05:29:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211249+0000", 2024-09-14T05:29:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000120035 2024-09-14T05:29:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.212226+0000", 2024-09-14T05:29:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00097679800000000008 2024-09-14T05:29:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.219302+0000", 2024-09-14T05:29:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0070764249999999999 2024-09-14T05:29:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.219362+0000", 2024-09-14T05:29:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9558999999999997e-05 2024-09-14T05:29:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.219404+0000", 2024-09-14T05:29:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2188000000000003e-05 2024-09-14T05:29:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1223 3.c 3:34659a14:::rbd_data.112dd25fb4c.000000000000008e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:41.104 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.211051+0000", 2024-09-14T05:29:41.104 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 10.852063017999999, 2024-09-14T05:29:41.104 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0084285249999999992, 2024-09-14T05:29:41.104 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.104 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.104 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:41.104 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.104 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.104 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.104 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1223 2024-09-14T05:29:41.104 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.105 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.105 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.105 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.105 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211051+0000", 2024-09-14T05:29:41.105 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.105 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.105 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.105 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.105 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211051+0000", 2024-09-14T05:29:41.105 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.106 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.106 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.106 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.106 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211051+0000", 2024-09-14T05:29:41.106 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4600000000000005e-07 2024-09-14T05:29:41.106 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.106 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.106 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.106 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211054+0000", 2024-09-14T05:29:41.106 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.145e-06 2024-09-14T05:29:41.106 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.106 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.106 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.106 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211062+0000", 2024-09-14T05:29:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8599999999999993e-06 2024-09-14T05:29:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211068+0000", 2024-09-14T05:29:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.285e-06 2024-09-14T05:29:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211475+0000", 2024-09-14T05:29:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00040667600000000002 2024-09-14T05:29:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211562+0000", 2024-09-14T05:29:41.108 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6550000000000003e-05 2024-09-14T05:29:41.108 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.108 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.108 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 7", 2024-09-14T05:29:41.108 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.211764+0000", 2024-09-14T05:29:41.108 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00020240199999999999 2024-09-14T05:29:41.108 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.108 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.108 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:41.108 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.212441+0000", 2024-09-14T05:29:41.108 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00067703000000000001 2024-09-14T05:29:41.108 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.108 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.108 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:41.108 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.219411+0000", 2024-09-14T05:29:41.109 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0069699310000000004 2024-09-14T05:29:41.109 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.109 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.109 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:41.109 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.219443+0000", 2024-09-14T05:29:41.109 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1687999999999999e-05 2024-09-14T05:29:41.109 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.109 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.109 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.109 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.219479+0000", 2024-09-14T05:29:41.109 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6412000000000003e-05 2024-09-14T05:29:41.109 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.109 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.109 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.109 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.110 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.110 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:1229 3.3 3:c1bb0aea:::rbd_data.112dd25fb4c.000000000000008b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:41.110 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.215352+0000", 2024-09-14T05:29:41.110 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 10.847761617, 2024-09-14T05:29:41.110 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0078661779999999997, 2024-09-14T05:29:41.110 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.110 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.110 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:41.110 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.110 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.110 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.110 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1229 2024-09-14T05:29:41.110 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.215352+0000", 2024-09-14T05:29:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.215352+0000", 2024-09-14T05:29:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.215353+0000", 2024-09-14T05:29:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6799999999999999e-07 2024-09-14T05:29:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.215357+0000", 2024-09-14T05:29:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2509999999999996e-06 2024-09-14T05:29:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.215368+0000", 2024-09-14T05:29:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0988999999999999e-05 2024-09-14T05:29:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.215373+0000", 2024-09-14T05:29:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9690000000000002e-06 2024-09-14T05:29:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.215413+0000", 2024-09-14T05:29:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0028999999999999e-05 2024-09-14T05:29:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.215474+0000", 2024-09-14T05:29:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0683000000000002e-05 2024-09-14T05:29:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.215589+0000", 2024-09-14T05:29:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000114795 2024-09-14T05:29:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.222979+0000", 2024-09-14T05:29:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0073905840000000004 2024-09-14T05:29:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.223124+0000", 2024-09-14T05:29:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000144974 2024-09-14T05:29:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.223175+0000", 2024-09-14T05:29:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1381000000000002e-05 2024-09-14T05:29:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.223218+0000", 2024-09-14T05:29:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2654999999999997e-05 2024-09-14T05:29:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:1272 3.1b e30/24 3:dd2c2221:::rbd_data.112dd25fb4c.00000000000000a0:head v 30'6, mlcod=30'4)", 2024-09-14T05:29:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.232502+0000", 2024-09-14T05:29:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 10.830611887, 2024-09-14T05:29:41.116 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0070299239999999999, 2024-09-14T05:29:41.116 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.116 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.116 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:41.116 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.116 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.232502+0000", 2024-09-14T05:29:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.232502+0000", 2024-09-14T05:29:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.232502+0000", 2024-09-14T05:29:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7999999999999995e-07 2024-09-14T05:29:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.232508+0000", 2024-09-14T05:29:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3140000000000003e-06 2024-09-14T05:29:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.232517+0000", 2024-09-14T05:29:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0189999999999995e-06 2024-09-14T05:29:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.232521+0000", 2024-09-14T05:29:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.301e-06 2024-09-14T05:29:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.232555+0000", 2024-09-14T05:29:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3908999999999997e-05 2024-09-14T05:29:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.232567+0000", 2024-09-14T05:29:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2184000000000001e-05 2024-09-14T05:29:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.239489+0000", 2024-09-14T05:29:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0069224509999999996 2024-09-14T05:29:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.239532+0000", 2024-09-14T05:29:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2166000000000003e-05 2024-09-14T05:29:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:2130 3.1b e30/24 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:head v 30'21, mlcod=30'19)", 2024-09-14T05:29:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.782900+0000", 2024-09-14T05:29:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 10.28021324, 2024-09-14T05:29:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.034589779000000001, 2024-09-14T05:29:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.782900+0000", 2024-09-14T05:29:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.782900+0000", 2024-09-14T05:29:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.782903+0000", 2024-09-14T05:29:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6800000000000002e-06 2024-09-14T05:29:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.123 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.123 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.802899+0000", 2024-09-14T05:29:41.123 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.019996435999999999 2024-09-14T05:29:41.123 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.123 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.123 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.123 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.802924+0000", 2024-09-14T05:29:41.123 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.463e-05 2024-09-14T05:29:41.123 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.123 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.123 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.123 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.802934+0000", 2024-09-14T05:29:41.123 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8800000000000003e-06 2024-09-14T05:29:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.803003+0000", 2024-09-14T05:29:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9158000000000004e-05 2024-09-14T05:29:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.803029+0000", 2024-09-14T05:29:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6395999999999999e-05 2024-09-14T05:29:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.817445+0000", 2024-09-14T05:29:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.014415258 2024-09-14T05:29:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.817490+0000", 2024-09-14T05:29:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5340999999999998e-05 2024-09-14T05:29:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:2132 3.1b e30/24 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:head v 30'22, mlcod=30'19)", 2024-09-14T05:29:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.802961+0000", 2024-09-14T05:29:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 10.260152314999999, 2024-09-14T05:29:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026740369, 2024-09-14T05:29:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.802961+0000", 2024-09-14T05:29:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.802961+0000", 2024-09-14T05:29:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.802962+0000", 2024-09-14T05:29:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7999999999999993e-07 2024-09-14T05:29:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.818597+0000", 2024-09-14T05:29:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.015634577 2024-09-14T05:29:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.818609+0000", 2024-09-14T05:29:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2669e-05 2024-09-14T05:29:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.818617+0000", 2024-09-14T05:29:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1620000000000002e-06 2024-09-14T05:29:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.818663+0000", 2024-09-14T05:29:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6137999999999998e-05 2024-09-14T05:29:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.818681+0000", 2024-09-14T05:29:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8726e-05 2024-09-14T05:29:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.829654+0000", 2024-09-14T05:29:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010972986000000001 2024-09-14T05:29:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.829702+0000", 2024-09-14T05:29:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7131e-05 2024-09-14T05:29:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:4423 3.3 3:c35d389d:::rbd_data.112d3d611e54.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:31.842664+0000", 2024-09-14T05:29:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.2204497100000005, 2024-09-14T05:29:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.015928326999999999, 2024-09-14T05:29:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 4423 2024-09-14T05:29:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.842664+0000", 2024-09-14T05:29:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.842664+0000", 2024-09-14T05:29:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.842665+0000", 2024-09-14T05:29:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9600000000000008e-07 2024-09-14T05:29:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.842670+0000", 2024-09-14T05:29:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8239999999999999e-06 2024-09-14T05:29:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.842681+0000", 2024-09-14T05:29:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1503e-05 2024-09-14T05:29:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.842687+0000", 2024-09-14T05:29:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5790000000000003e-06 2024-09-14T05:29:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.842743+0000", 2024-09-14T05:29:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5810999999999998e-05 2024-09-14T05:29:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.842831+0000", 2024-09-14T05:29:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8159999999999996e-05 2024-09-14T05:29:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.845609+0000", 2024-09-14T05:29:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0027784659999999998 2024-09-14T05:29:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.858500+0000", 2024-09-14T05:29:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.012890777000000001 2024-09-14T05:29:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.858510+0000", 2024-09-14T05:29:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0429e-05 2024-09-14T05:29:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.858546+0000", 2024-09-14T05:29:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5543999999999997e-05 2024-09-14T05:29:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.858592+0000", 2024-09-14T05:29:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6238e-05 2024-09-14T05:29:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:4586 3.3 3:c35d389d:::rbd_data.112d3d611e54.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:32.039485+0000", 2024-09-14T05:29:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.0236282939999999, 2024-09-14T05:29:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017931286000000001, 2024-09-14T05:29:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 4586 2024-09-14T05:29:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.039485+0000", 2024-09-14T05:29:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.039485+0000", 2024-09-14T05:29:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.039487+0000", 2024-09-14T05:29:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2810000000000001e-06 2024-09-14T05:29:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.039492+0000", 2024-09-14T05:29:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8619999999999999e-06 2024-09-14T05:29:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.039504+0000", 2024-09-14T05:29:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1282e-05 2024-09-14T05:29:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.039509+0000", 2024-09-14T05:29:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5260000000000001e-06 2024-09-14T05:29:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.039559+0000", 2024-09-14T05:29:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9947999999999997e-05 2024-09-14T05:29:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.039610+0000", 2024-09-14T05:29:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1329999999999998e-05 2024-09-14T05:29:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.041840+0000", 2024-09-14T05:29:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002229932 2024-09-14T05:29:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.057198+0000", 2024-09-14T05:29:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.015357674999999999 2024-09-14T05:29:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.057254+0000", 2024-09-14T05:29:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5865999999999997e-05 2024-09-14T05:29:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.140 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.140 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:41.140 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.057326+0000", 2024-09-14T05:29:41.140 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1962e-05 2024-09-14T05:29:41.140 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.140 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.140 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.140 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:32.057417+0000", 2024-09-14T05:29:41.140 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0622000000000001e-05 2024-09-14T05:29:41.140 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.140 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.141 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.141 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.141 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.141 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7066 3.15 3:abadb03e:::rbd_data.112df7f2fe0a.0000000000000027:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.141 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.142239+0000", 2024-09-14T05:29:41.141 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.9208744649999998, 2024-09-14T05:29:41.141 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0072481910000000002, 2024-09-14T05:29:41.141 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.141 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.142 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:41.142 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.142 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.142 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.142 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7066 2024-09-14T05:29:41.142 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.142 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.142 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.143 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.143 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.142239+0000", 2024-09-14T05:29:41.143 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.143 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.143 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.143 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.142239+0000", 2024-09-14T05:29:41.143 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.144 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.144 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.144 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.142241+0000", 2024-09-14T05:29:41.144 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3800000000000001e-06 2024-09-14T05:29:41.144 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.144 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.144 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.144 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.142261+0000", 2024-09-14T05:29:41.144 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9774000000000001e-05 2024-09-14T05:29:41.144 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.144 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.145 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.145 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.142300+0000", 2024-09-14T05:29:41.145 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8318999999999997e-05 2024-09-14T05:29:41.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.145 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.145 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.142308+0000", 2024-09-14T05:29:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4260000000000003e-06 2024-09-14T05:29:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.142378+0000", 2024-09-14T05:29:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0003e-05 2024-09-14T05:29:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.142456+0000", 2024-09-14T05:29:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8398000000000004e-05 2024-09-14T05:29:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.147 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 6", 2024-09-14T05:29:41.147 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.142614+0000", 2024-09-14T05:29:41.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015796699999999999 2024-09-14T05:29:41.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.147 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:41.147 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.143776+0000", 2024-09-14T05:29:41.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001161565 2024-09-14T05:29:41.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.148 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:41.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.149392+0000", 2024-09-14T05:29:41.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0056158329999999998 2024-09-14T05:29:41.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.148 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:41.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.149446+0000", 2024-09-14T05:29:41.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4466000000000003e-05 2024-09-14T05:29:41.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.149487+0000", 2024-09-14T05:29:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1060000000000003e-05 2024-09-14T05:29:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7724 3.f 3:f1064780:::rbd_data.112d114b4eae.0000000000000007:8 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.600666+0000", 2024-09-14T05:29:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.4624476919999996, 2024-09-14T05:29:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0071472569999999997, 2024-09-14T05:29:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.150 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.150 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.150 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.150 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7724 2024-09-14T05:29:41.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.150 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.150 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.150 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.600666+0000", 2024-09-14T05:29:41.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.600666+0000", 2024-09-14T05:29:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.600666+0000", 2024-09-14T05:29:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6700000000000003e-07 2024-09-14T05:29:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.600668+0000", 2024-09-14T05:29:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9039999999999999e-06 2024-09-14T05:29:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.600675+0000", 2024-09-14T05:29:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5690000000000001e-06 2024-09-14T05:29:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.600688+0000", 2024-09-14T05:29:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3129000000000001e-05 2024-09-14T05:29:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.607637+0000", 2024-09-14T05:29:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0069487780000000001 2024-09-14T05:29:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.607710+0000", 2024-09-14T05:29:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3146999999999996e-05 2024-09-14T05:29:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.607813+0000", 2024-09-14T05:29:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000103163 2024-09-14T05:29:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.154 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7774 3.f 3:f12e2111:::rbd_data.112db03d36fa.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.154 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.607856+0000", 2024-09-14T05:29:41.154 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 5.4552573879999997, 2024-09-14T05:29:41.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0082050860000000003, 2024-09-14T05:29:41.154 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.155 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.155 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:41.155 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.155 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.155 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.155 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7774 2024-09-14T05:29:41.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.155 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.155 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.155 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.607856+0000", 2024-09-14T05:29:41.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.607856+0000", 2024-09-14T05:29:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.607858+0000", 2024-09-14T05:29:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.689e-06 2024-09-14T05:29:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.608577+0000", 2024-09-14T05:29:41.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00071960199999999996 2024-09-14T05:29:41.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.608591+0000", 2024-09-14T05:29:41.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.346e-05 2024-09-14T05:29:41.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.608599+0000", 2024-09-14T05:29:41.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9759999999999995e-06 2024-09-14T05:29:41.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.608643+0000", 2024-09-14T05:29:41.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4092999999999999e-05 2024-09-14T05:29:41.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.608787+0000", 2024-09-14T05:29:41.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014395099999999999 2024-09-14T05:29:41.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 1", 2024-09-14T05:29:41.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.609013+0000", 2024-09-14T05:29:41.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000226393 2024-09-14T05:29:41.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:41.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.612114+0000", 2024-09-14T05:29:41.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0031009079999999999 2024-09-14T05:29:41.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:41.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.615948+0000", 2024-09-14T05:29:41.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.003833749 2024-09-14T05:29:41.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:41.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.616022+0000", 2024-09-14T05:29:41.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3609000000000006e-05 2024-09-14T05:29:41.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.616061+0000", 2024-09-14T05:29:41.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9656000000000001e-05 2024-09-14T05:29:41.161 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.161 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.161 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.161 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:8400 3.f 3:f16f866b:::rbd_data.112d114b4eae.0000000000000005:8 [read 720896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T05:29:41.162 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:36.319068+0000", 2024-09-14T05:29:41.162 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.7440459520000005, 2024-09-14T05:29:41.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0070333419999999997, 2024-09-14T05:29:41.162 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.162 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.162 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.162 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.162 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.162 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.162 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 8400 2024-09-14T05:29:41.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.162 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:36.319068+0000", 2024-09-14T05:29:41.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:36.319068+0000", 2024-09-14T05:29:41.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:36.319068+0000", 2024-09-14T05:29:41.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.08e-07 2024-09-14T05:29:41.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:36.319070+0000", 2024-09-14T05:29:41.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8220000000000001e-06 2024-09-14T05:29:41.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:36.319076+0000", 2024-09-14T05:29:41.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8799999999999996e-06 2024-09-14T05:29:41.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:36.319081+0000", 2024-09-14T05:29:41.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9130000000000002e-06 2024-09-14T05:29:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:36.325914+0000", 2024-09-14T05:29:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0068335419999999997 2024-09-14T05:29:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:36.325993+0000", 2024-09-14T05:29:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8652999999999994e-05 2024-09-14T05:29:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:36.326101+0000", 2024-09-14T05:29:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000108124 2024-09-14T05:29:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:8401 3.f 3:f16f866b:::rbd_data.112d114b4eae.0000000000000005:8 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T05:29:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:36.319087+0000", 2024-09-14T05:29:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.7440270309999999, 2024-09-14T05:29:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0071862669999999997, 2024-09-14T05:29:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 8401 2024-09-14T05:29:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:36.319087+0000", 2024-09-14T05:29:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:36.319087+0000", 2024-09-14T05:29:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:36.319087+0000", 2024-09-14T05:29:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0999999999999999e-07 2024-09-14T05:29:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:36.319089+0000", 2024-09-14T05:29:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.764e-06 2024-09-14T05:29:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:36.319095+0000", 2024-09-14T05:29:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.426e-06 2024-09-14T05:29:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:36.319100+0000", 2024-09-14T05:29:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0669999999999999e-06 2024-09-14T05:29:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:36.326147+0000", 2024-09-14T05:29:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.007047111 2024-09-14T05:29:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:36.326209+0000", 2024-09-14T05:29:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1649000000000006e-05 2024-09-14T05:29:41.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.172 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.172 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:36.326273+0000", 2024-09-14T05:29:41.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3839999999999999e-05 2024-09-14T05:29:41.172 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.172 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.172 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.172 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.172 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.172 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:41.174 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:29:41.175 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-14T05:29:41.175 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:41.173+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:41.274 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:41.273+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:41.324 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-14T05:29:41.324 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:41.325 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:41.325 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:29:41.325 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:41.336 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:29:41.374 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:41.374+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:41.475 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:41.475+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:41.487 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:41.487 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:41.487 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:41.487 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:41.487 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:41.496 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:29:41.511 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:29:41.534 INFO:teuthology.orchestra.run.smithi071.stdout:1149 write 0x9ad6bf0 thru 0x9ad72b6 (0x6c7 bytes) 2024-09-14T05:29:41.541 INFO:teuthology.orchestra.run.smithi071.stdout:1150 write 0xaf26c2b thru 0xaf30820 (0x9bf6 bytes) 2024-09-14T05:29:41.545 INFO:teuthology.orchestra.run.smithi071.stdout:1151 writesame 0x131fd45 thru 0x13224b2 data_size 0x31(0x276e bytes) 2024-09-14T05:29:41.549 INFO:teuthology.orchestra.run.smithi071.stdout:1152 read 0x8f0c142 thru 0x8f14fe8 (0x8ea7 bytes) 2024-09-14T05:29:41.549 INFO:teuthology.orchestra.run.smithi071.stdout:1154 write 0xfa161b thru 0xfa1a97 (0x47d bytes) 2024-09-14T05:29:41.561 INFO:teuthology.orchestra.run.smithi071.stdout:1157 trunc from 0xd61154b to 0x6ea056c 2024-09-14T05:29:41.576 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:41.576+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:41.581 INFO:teuthology.orchestra.run.smithi071.stdout:1158 clone 12 order 19 su 131072 sc 2 2024-09-14T05:29:41.646 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:41.646 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:41.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:41.647 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:41.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.647 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:41.647 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:29:41.647 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.868464572000001, 2024-09-14T05:29:41.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.24562984099999999, 2024-09-14T05:29:41.647 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.647 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.647 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:41.647 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.647 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:41.647 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:41.647 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1 2024-09-14T05:29:41.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.648 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.648 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.648 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:29:41.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.648 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.648 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:29:41.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.648 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.648 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776565+0000", 2024-09-14T05:29:41.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4780000000000001e-06 2024-09-14T05:29:41.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.649 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.649 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776571+0000", 2024-09-14T05:29:41.649 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7669999999999998e-06 2024-09-14T05:29:41.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.649 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.649 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776580+0000", 2024-09-14T05:29:41.649 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9830000000000002e-06 2024-09-14T05:29:41.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.649 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.649 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776589+0000", 2024-09-14T05:29:41.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7579999999999998e-06 2024-09-14T05:29:41.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776662+0000", 2024-09-14T05:29:41.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3332999999999997e-05 2024-09-14T05:29:41.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:29:41.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776670+0000", 2024-09-14T05:29:41.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9850000000000006e-06 2024-09-14T05:29:41.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.651 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.651 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.020867+0000", 2024-09-14T05:29:41.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.244196265 2024-09-14T05:29:41.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.651 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.651 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021022+0000", 2024-09-14T05:29:41.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000155699 2024-09-14T05:29:41.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.651 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:41.651 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021169+0000", 2024-09-14T05:29:41.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014661700000000001 2024-09-14T05:29:41.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.652 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:41.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021891+0000", 2024-09-14T05:29:41.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072146700000000005 2024-09-14T05:29:41.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.652 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:41.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022124+0000", 2024-09-14T05:29:41.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023354199999999999 2024-09-14T05:29:41.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.652 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:41.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022164+0000", 2024-09-14T05:29:41.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9897000000000001e-05 2024-09-14T05:29:41.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.653 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.653 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022193+0000", 2024-09-14T05:29:41.653 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9050000000000001e-05 2024-09-14T05:29:41.653 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.653 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.653 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.653 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:1989 3.1f e30/24 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head v 30'29, mlcod=30'27)", 2024-09-14T05:29:41.653 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:29:41.653 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 11.051693585000001, 2024-09-14T05:29:41.653 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.064242492999999998, 2024-09-14T05:29:41.653 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.654 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.654 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:41.654 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.654 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.654 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.654 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:29:41.654 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.654 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.654 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.654 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:29:41.654 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.655 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.655 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.655 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593336+0000", 2024-09-14T05:29:41.655 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3909999999999999e-06 2024-09-14T05:29:41.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.655 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.655 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.655 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616632+0000", 2024-09-14T05:29:41.655 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.023296598000000002 2024-09-14T05:29:41.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.655 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.655 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.655 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616657+0000", 2024-09-14T05:29:41.655 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5228e-05 2024-09-14T05:29:41.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.656 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.656 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616684+0000", 2024-09-14T05:29:41.656 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6441999999999999e-05 2024-09-14T05:29:41.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.656 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.656 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616755+0000", 2024-09-14T05:29:41.656 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1452000000000006e-05 2024-09-14T05:29:41.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.656 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.656 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616780+0000", 2024-09-14T05:29:41.657 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4436000000000001e-05 2024-09-14T05:29:41.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.657 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.657 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:41.657 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657527+0000", 2024-09-14T05:29:41.657 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.040747323000000002 2024-09-14T05:29:41.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.657 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.657 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.657 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657577+0000", 2024-09-14T05:29:41.657 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9623000000000002e-05 2024-09-14T05:29:41.657 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.657 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.657 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.658 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.658 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:4415 3.1c 3:3a7c4142:::rbd_data.112d3d611e54.0000000000000012:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:41.658 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:31.820344+0000", 2024-09-14T05:29:41.658 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.8246832859999991, 2024-09-14T05:29:41.658 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.01729632, 2024-09-14T05:29:41.658 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.658 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.658 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:41.658 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.658 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.658 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.658 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 4415 2024-09-14T05:29:41.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.659 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.659 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.659 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.659 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.820344+0000", 2024-09-14T05:29:41.659 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.659 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.659 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.659 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.820344+0000", 2024-09-14T05:29:41.659 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.659 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.659 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.660 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.820345+0000", 2024-09-14T05:29:41.660 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1500000000000003e-07 2024-09-14T05:29:41.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.660 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.660 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.660 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.820350+0000", 2024-09-14T05:29:41.660 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3030000000000001e-06 2024-09-14T05:29:41.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.660 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.660 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.660 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.820360+0000", 2024-09-14T05:29:41.660 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0783e-05 2024-09-14T05:29:41.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.661 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.661 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.820366+0000", 2024-09-14T05:29:41.661 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5079999999999996e-06 2024-09-14T05:29:41.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.661 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.661 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.820403+0000", 2024-09-14T05:29:41.661 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7450999999999998e-05 2024-09-14T05:29:41.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.661 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.662 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.820480+0000", 2024-09-14T05:29:41.662 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6093000000000005e-05 2024-09-14T05:29:41.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.662 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 0", 2024-09-14T05:29:41.662 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.822758+0000", 2024-09-14T05:29:41.662 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0022783730000000002 2024-09-14T05:29:41.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.662 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:41.662 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.837542+0000", 2024-09-14T05:29:41.662 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.014784016000000001 2024-09-14T05:29:41.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.663 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:41.663 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.837553+0000", 2024-09-14T05:29:41.663 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1505000000000001e-05 2024-09-14T05:29:41.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.663 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:41.663 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.837589+0000", 2024-09-14T05:29:41.663 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5635e-05 2024-09-14T05:29:41.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.663 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.663 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:31.837641+0000", 2024-09-14T05:29:41.664 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1737999999999998e-05 2024-09-14T05:29:41.664 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.664 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.664 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.664 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7607 3.10 3:0b178fa3:::rbd_data.112d114b4eae.0000000000000001:8 [read 589824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.664 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.594851+0000", 2024-09-14T05:29:41.664 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.0501767070000003, 2024-09-14T05:29:41.664 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.015656860000000002, 2024-09-14T05:29:41.664 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.664 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.664 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.665 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.665 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.665 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.665 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7607 2024-09-14T05:29:41.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.665 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.665 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.665 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594851+0000", 2024-09-14T05:29:41.665 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.665 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.665 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594851+0000", 2024-09-14T05:29:41.666 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.666 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.666 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594851+0000", 2024-09-14T05:29:41.666 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7300000000000002e-07 2024-09-14T05:29:41.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.666 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.666 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594853+0000", 2024-09-14T05:29:41.666 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.945e-06 2024-09-14T05:29:41.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.666 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.666 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594859+0000", 2024-09-14T05:29:41.667 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5470000000000004e-06 2024-09-14T05:29:41.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.667 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.667 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594863+0000", 2024-09-14T05:29:41.667 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3170000000000003e-06 2024-09-14T05:29:41.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.667 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.667 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.610412+0000", 2024-09-14T05:29:41.667 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.015548358 2024-09-14T05:29:41.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.667 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.667 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.610457+0000", 2024-09-14T05:29:41.668 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5130999999999999e-05 2024-09-14T05:29:41.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.668 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.668 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.668 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.610508+0000", 2024-09-14T05:29:41.668 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1189000000000001e-05 2024-09-14T05:29:41.668 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.668 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.668 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.668 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.668 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7608 3.10 3:0b178fa3:::rbd_data.112d114b4eae.0000000000000001:8 [read 720896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.669 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.594870+0000", 2024-09-14T05:29:41.669 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.0501575660000002, 2024-09-14T05:29:41.669 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.015821386, 2024-09-14T05:29:41.669 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.669 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.669 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.669 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.669 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.669 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.669 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7608 2024-09-14T05:29:41.669 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.669 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.669 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.669 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594870+0000", 2024-09-14T05:29:41.670 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.670 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594870+0000", 2024-09-14T05:29:41.670 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.670 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594871+0000", 2024-09-14T05:29:41.670 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8000000000000001e-07 2024-09-14T05:29:41.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.671 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.671 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594872+0000", 2024-09-14T05:29:41.671 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-14T05:29:41.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.671 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.671 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.671 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594886+0000", 2024-09-14T05:29:41.671 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3173000000000001e-05 2024-09-14T05:29:41.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.671 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.671 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.671 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594891+0000", 2024-09-14T05:29:41.671 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1059999999999999e-06 2024-09-14T05:29:41.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.671 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.671 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.672 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.610535+0000", 2024-09-14T05:29:41.672 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.015644075 2024-09-14T05:29:41.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.672 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.672 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.610599+0000", 2024-09-14T05:29:41.672 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4071000000000004e-05 2024-09-14T05:29:41.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.672 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.672 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.610692+0000", 2024-09-14T05:29:41.672 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2638000000000003e-05 2024-09-14T05:29:41.672 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.672 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.673 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.673 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.673 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7609 3.10 3:0b178fa3:::rbd_data.112d114b4eae.0000000000000001:8 [read 638976~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.673 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.594898+0000", 2024-09-14T05:29:41.673 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.0501293939999998, 2024-09-14T05:29:41.673 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.016011338999999999, 2024-09-14T05:29:41.673 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.673 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.673 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.673 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.674 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.674 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.674 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7609 2024-09-14T05:29:41.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.674 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.674 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.674 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594898+0000", 2024-09-14T05:29:41.674 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.674 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.674 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594898+0000", 2024-09-14T05:29:41.674 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.675 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.675 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594899+0000", 2024-09-14T05:29:41.675 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1900000000000003e-07 2024-09-14T05:29:41.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.675 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.675 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594901+0000", 2024-09-14T05:29:41.675 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.379e-06 2024-09-14T05:29:41.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.675 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.675 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594909+0000", 2024-09-14T05:29:41.676 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7090000000000002e-06 2024-09-14T05:29:41.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.676 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.676 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.676 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594914+0000", 2024-09-14T05:29:41.676 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9540000000000003e-06 2024-09-14T05:29:41.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.676 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.676 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.676 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.610734+0000", 2024-09-14T05:29:41.676 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.015820172 2024-09-14T05:29:41.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.676 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.676 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.676 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.610806+0000", 2024-09-14T05:29:41.677 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1829999999999995e-05 2024-09-14T05:29:41.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.677 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.677 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.610910+0000", 2024-09-14T05:29:41.677 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000103776 2024-09-14T05:29:41.677 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.677 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.677 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.677 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7610 3.10 3:0b178fa3:::rbd_data.112d114b4eae.0000000000000001:8 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.677 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.594920+0000", 2024-09-14T05:29:41.677 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.0501077670000001, 2024-09-14T05:29:41.677 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.016214836, 2024-09-14T05:29:41.678 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.678 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.678 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.678 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.678 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.678 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.678 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7610 2024-09-14T05:29:41.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.678 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.678 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.678 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.678 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594920+0000", 2024-09-14T05:29:41.678 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.679 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.679 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594920+0000", 2024-09-14T05:29:41.679 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.679 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.679 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594920+0000", 2024-09-14T05:29:41.679 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.89e-07 2024-09-14T05:29:41.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.679 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.680 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594922+0000", 2024-09-14T05:29:41.680 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9750000000000001e-06 2024-09-14T05:29:41.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.680 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.680 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.680 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594929+0000", 2024-09-14T05:29:41.680 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2029999999999997e-06 2024-09-14T05:29:41.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.680 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.680 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.680 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594940+0000", 2024-09-14T05:29:41.680 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1559000000000001e-05 2024-09-14T05:29:41.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.681 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.681 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.610948+0000", 2024-09-14T05:29:41.681 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.016008397000000001 2024-09-14T05:29:41.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.681 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.681 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.611036+0000", 2024-09-14T05:29:41.681 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7235000000000001e-05 2024-09-14T05:29:41.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.681 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.682 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.611135+0000", 2024-09-14T05:29:41.682 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9078000000000002e-05 2024-09-14T05:29:41.682 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.682 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.682 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.682 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.682 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7611 3.10 3:0b178fa3:::rbd_data.112d114b4eae.0000000000000001:8 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.682 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.594959+0000", 2024-09-14T05:29:41.682 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.050068606, 2024-09-14T05:29:41.682 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.016425392, 2024-09-14T05:29:41.682 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.682 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.683 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.683 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.683 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.683 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.683 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7611 2024-09-14T05:29:41.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.683 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.683 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.683 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.683 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594959+0000", 2024-09-14T05:29:41.683 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.683 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.684 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.684 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594959+0000", 2024-09-14T05:29:41.684 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.684 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.684 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.684 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.684 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594960+0000", 2024-09-14T05:29:41.684 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7999999999999995e-07 2024-09-14T05:29:41.684 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.684 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.684 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.684 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594962+0000", 2024-09-14T05:29:41.684 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3089999999999998e-06 2024-09-14T05:29:41.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.685 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.685 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.685 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594968+0000", 2024-09-14T05:29:41.685 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2589999999999997e-06 2024-09-14T05:29:41.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.685 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.685 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.685 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594973+0000", 2024-09-14T05:29:41.685 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8729999999999998e-06 2024-09-14T05:29:41.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.685 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.685 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.686 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.611172+0000", 2024-09-14T05:29:41.686 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.016199174 2024-09-14T05:29:41.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.686 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.686 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.686 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.611256+0000", 2024-09-14T05:29:41.686 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3251000000000006e-05 2024-09-14T05:29:41.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.686 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.686 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.686 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.611385+0000", 2024-09-14T05:29:41.686 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012894600000000001 2024-09-14T05:29:41.686 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.687 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.687 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.687 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.687 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7612 3.10 3:0b178fa3:::rbd_data.112d114b4eae.0000000000000001:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.687 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.594980+0000", 2024-09-14T05:29:41.687 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.0500474799999999, 2024-09-14T05:29:41.687 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.016589636000000001, 2024-09-14T05:29:41.687 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.687 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.687 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.687 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.687 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.688 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.688 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7612 2024-09-14T05:29:41.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.688 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.688 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.688 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594980+0000", 2024-09-14T05:29:41.688 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.688 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.688 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594980+0000", 2024-09-14T05:29:41.688 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.689 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.689 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594981+0000", 2024-09-14T05:29:41.689 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3599999999999999e-07 2024-09-14T05:29:41.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.689 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.689 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594983+0000", 2024-09-14T05:29:41.689 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0779999999999998e-06 2024-09-14T05:29:41.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.689 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.689 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594989+0000", 2024-09-14T05:29:41.690 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7699999999999998e-06 2024-09-14T05:29:41.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.690 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.690 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.690 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594993+0000", 2024-09-14T05:29:41.690 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.34e-06 2024-09-14T05:29:41.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.690 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.690 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.690 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.611410+0000", 2024-09-14T05:29:41.690 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.016417023999999999 2024-09-14T05:29:41.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.690 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.691 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.691 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.611479+0000", 2024-09-14T05:29:41.691 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8926999999999999e-05 2024-09-14T05:29:41.691 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.691 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.691 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.691 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.611570+0000", 2024-09-14T05:29:41.691 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1061000000000001e-05 2024-09-14T05:29:41.691 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.691 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.691 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.691 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.691 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.691 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7627 3.10 3:088d6032:::rbd_data.112d114b4eae.0000000000000002:8 [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.691 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.594999+0000", 2024-09-14T05:29:41.692 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.050028535, 2024-09-14T05:29:41.692 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.016705718000000001, 2024-09-14T05:29:41.692 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.692 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.692 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.692 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.692 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.692 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.692 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7627 2024-09-14T05:29:41.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.692 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.692 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.692 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.693 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594999+0000", 2024-09-14T05:29:41.693 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.693 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.693 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.693 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.594999+0000", 2024-09-14T05:29:41.693 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.693 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.693 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.693 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595000+0000", 2024-09-14T05:29:41.693 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.27e-07 2024-09-14T05:29:41.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595009+0000", 2024-09-14T05:29:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4429999999999992e-06 2024-09-14T05:29:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595016+0000", 2024-09-14T05:29:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4849999999999997e-06 2024-09-14T05:29:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595020+0000", 2024-09-14T05:29:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7829999999999998e-06 2024-09-14T05:29:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.611599+0000", 2024-09-14T05:29:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.016578859000000001 2024-09-14T05:29:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.611641+0000", 2024-09-14T05:29:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1443999999999998e-05 2024-09-14T05:29:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.696 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.611705+0000", 2024-09-14T05:29:41.696 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4276999999999995e-05 2024-09-14T05:29:41.696 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.696 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.696 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.696 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.696 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7628 3.10 3:088d6032:::rbd_data.112d114b4eae.0000000000000002:8 [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.696 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.595028+0000", 2024-09-14T05:29:41.696 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.0500001719999998, 2024-09-14T05:29:41.696 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.016881493000000001, 2024-09-14T05:29:41.696 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.696 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.696 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.697 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.697 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.697 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.697 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7628 2024-09-14T05:29:41.697 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.697 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.697 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.697 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.697 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595028+0000", 2024-09-14T05:29:41.697 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.697 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.697 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.697 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.697 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595028+0000", 2024-09-14T05:29:41.698 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.698 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.698 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.698 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595028+0000", 2024-09-14T05:29:41.698 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3000000000000001e-07 2024-09-14T05:29:41.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.698 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.698 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.698 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595033+0000", 2024-09-14T05:29:41.698 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1610000000000002e-06 2024-09-14T05:29:41.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.698 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.698 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.699 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595039+0000", 2024-09-14T05:29:41.699 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1299999999999998e-06 2024-09-14T05:29:41.699 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.699 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.699 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.699 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595050+0000", 2024-09-14T05:29:41.699 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0920999999999999e-05 2024-09-14T05:29:41.699 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.699 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.699 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.699 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.611742+0000", 2024-09-14T05:29:41.699 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.016691962000000001 2024-09-14T05:29:41.699 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.699 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.700 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.700 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.611809+0000", 2024-09-14T05:29:41.700 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6789e-05 2024-09-14T05:29:41.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.700 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.700 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.611909+0000", 2024-09-14T05:29:41.700 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010009999999999999 2024-09-14T05:29:41.700 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.700 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.700 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.701 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7629 3.10 3:088d6032:::rbd_data.112d114b4eae.0000000000000002:8 [read 655360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.701 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.595058+0000", 2024-09-14T05:29:41.701 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.0499699299999996, 2024-09-14T05:29:41.701 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017050053999999999, 2024-09-14T05:29:41.701 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.701 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.701 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.701 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.701 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.701 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.701 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7629 2024-09-14T05:29:41.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.701 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.701 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.702 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.702 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595058+0000", 2024-09-14T05:29:41.702 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.702 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.702 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595058+0000", 2024-09-14T05:29:41.702 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.702 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.702 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595058+0000", 2024-09-14T05:29:41.702 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3300000000000003e-07 2024-09-14T05:29:41.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.703 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.703 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.703 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595060+0000", 2024-09-14T05:29:41.703 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1600000000000001e-06 2024-09-14T05:29:41.703 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.703 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.703 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.703 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595067+0000", 2024-09-14T05:29:41.703 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2600000000000002e-06 2024-09-14T05:29:41.703 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.703 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.703 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.703 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595072+0000", 2024-09-14T05:29:41.704 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8609999999999997e-06 2024-09-14T05:29:41.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.704 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.704 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.611950+0000", 2024-09-14T05:29:41.704 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.016878147999999999 2024-09-14T05:29:41.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.704 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.704 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.612027+0000", 2024-09-14T05:29:41.704 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7214999999999998e-05 2024-09-14T05:29:41.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.705 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.705 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.612108+0000", 2024-09-14T05:29:41.705 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0976999999999994e-05 2024-09-14T05:29:41.705 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.705 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.705 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.705 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7630 3.10 3:088d6032:::rbd_data.112d114b4eae.0000000000000002:8 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.705 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.595078+0000", 2024-09-14T05:29:41.705 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.0499497670000002, 2024-09-14T05:29:41.705 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017229891000000001, 2024-09-14T05:29:41.705 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.706 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.706 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.706 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.706 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.706 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.706 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7630 2024-09-14T05:29:41.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.706 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.706 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.706 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595078+0000", 2024-09-14T05:29:41.706 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.707 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.707 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595078+0000", 2024-09-14T05:29:41.707 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.707 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.707 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595078+0000", 2024-09-14T05:29:41.707 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4900000000000001e-07 2024-09-14T05:29:41.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.707 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.707 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595080+0000", 2024-09-14T05:29:41.707 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.911e-06 2024-09-14T05:29:41.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.708 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.708 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595086+0000", 2024-09-14T05:29:41.708 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7579999999999998e-06 2024-09-14T05:29:41.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.708 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.708 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595091+0000", 2024-09-14T05:29:41.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5689999999999998e-06 2024-09-14T05:29:41.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.612144+0000", 2024-09-14T05:29:41.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017053919000000001 2024-09-14T05:29:41.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.612224+0000", 2024-09-14T05:29:41.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9404000000000001e-05 2024-09-14T05:29:41.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.710 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.710 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.612308+0000", 2024-09-14T05:29:41.710 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3980999999999994e-05 2024-09-14T05:29:41.710 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.710 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.710 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.710 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7631 3.10 3:088d6032:::rbd_data.112d114b4eae.0000000000000002:8 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.710 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.595097+0000", 2024-09-14T05:29:41.711 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.0499306830000004, 2024-09-14T05:29:41.711 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017431545, 2024-09-14T05:29:41.711 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.711 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.711 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.711 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.711 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.711 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.711 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7631 2024-09-14T05:29:41.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.711 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.711 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.712 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595097+0000", 2024-09-14T05:29:41.712 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.712 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.712 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595097+0000", 2024-09-14T05:29:41.712 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.712 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.712 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595097+0000", 2024-09-14T05:29:41.712 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9200000000000002e-07 2024-09-14T05:29:41.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.713 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.713 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595099+0000", 2024-09-14T05:29:41.713 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.962e-06 2024-09-14T05:29:41.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.713 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.713 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595106+0000", 2024-09-14T05:29:41.713 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.173e-06 2024-09-14T05:29:41.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.713 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.713 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595110+0000", 2024-09-14T05:29:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8239999999999999e-06 2024-09-14T05:29:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.612335+0000", 2024-09-14T05:29:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017224692999999999 2024-09-14T05:29:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.612401+0000", 2024-09-14T05:29:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6109e-05 2024-09-14T05:29:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.612529+0000", 2024-09-14T05:29:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012739199999999999 2024-09-14T05:29:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7637 3.10 3:088d6032:::rbd_data.112d114b4eae.0000000000000002:8 [read 524288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.595422+0000", 2024-09-14T05:29:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.049606045, 2024-09-14T05:29:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017315264, 2024-09-14T05:29:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7637 2024-09-14T05:29:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595422+0000", 2024-09-14T05:29:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595422+0000", 2024-09-14T05:29:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595422+0000", 2024-09-14T05:29:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9800000000000003e-07 2024-09-14T05:29:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595426+0000", 2024-09-14T05:29:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8190000000000002e-06 2024-09-14T05:29:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595434+0000", 2024-09-14T05:29:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2039999999999997e-06 2024-09-14T05:29:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595440+0000", 2024-09-14T05:29:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0580000000000003e-06 2024-09-14T05:29:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.612564+0000", 2024-09-14T05:29:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017123202000000001 2024-09-14T05:29:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.612640+0000", 2024-09-14T05:29:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6180000000000006e-05 2024-09-14T05:29:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.612737+0000", 2024-09-14T05:29:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7102999999999995e-05 2024-09-14T05:29:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7638 3.10 3:088d6032:::rbd_data.112d114b4eae.0000000000000002:8 [read 573440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.595451+0000", 2024-09-14T05:29:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.049576418, 2024-09-14T05:29:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017485628999999999, 2024-09-14T05:29:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7638 2024-09-14T05:29:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595451+0000", 2024-09-14T05:29:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595451+0000", 2024-09-14T05:29:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595452+0000", 2024-09-14T05:29:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5400000000000002e-07 2024-09-14T05:29:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595454+0000", 2024-09-14T05:29:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0820000000000001e-06 2024-09-14T05:29:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595459+0000", 2024-09-14T05:29:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5890000000000002e-06 2024-09-14T05:29:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595464+0000", 2024-09-14T05:29:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4870000000000004e-06 2024-09-14T05:29:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.612766+0000", 2024-09-14T05:29:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017301667999999999 2024-09-14T05:29:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.612832+0000", 2024-09-14T05:29:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6549000000000003e-05 2024-09-14T05:29:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.612937+0000", 2024-09-14T05:29:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0001048 2024-09-14T05:29:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7639 3.10 3:088d6032:::rbd_data.112d114b4eae.0000000000000002:8 [read 622592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.595549+0000", 2024-09-14T05:29:41.726 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.0494789280000001, 2024-09-14T05:29:41.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017576338, 2024-09-14T05:29:41.726 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.726 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.726 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.726 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.726 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.726 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.726 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7639 2024-09-14T05:29:41.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.726 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.726 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.726 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595549+0000", 2024-09-14T05:29:41.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595549+0000", 2024-09-14T05:29:41.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595549+0000", 2024-09-14T05:29:41.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4000000000000001e-07 2024-09-14T05:29:41.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595552+0000", 2024-09-14T05:29:41.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9529999999999999e-06 2024-09-14T05:29:41.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595559+0000", 2024-09-14T05:29:41.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0400000000000004e-06 2024-09-14T05:29:41.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595565+0000", 2024-09-14T05:29:41.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6280000000000002e-06 2024-09-14T05:29:41.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.729 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.729 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.612973+0000", 2024-09-14T05:29:41.729 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017407435999999998 2024-09-14T05:29:41.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.729 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.729 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.613044+0000", 2024-09-14T05:29:41.729 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1501000000000005e-05 2024-09-14T05:29:41.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.613125+0000", 2024-09-14T05:29:41.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1139999999999999e-05 2024-09-14T05:29:41.730 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.730 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.730 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.730 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7640 3.10 3:088d6032:::rbd_data.112d114b4eae.0000000000000002:8 [read 671744~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.730 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.595575+0000", 2024-09-14T05:29:41.730 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.0494528570000003, 2024-09-14T05:29:41.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017700706, 2024-09-14T05:29:41.730 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.730 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.731 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.731 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.731 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.731 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.731 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7640 2024-09-14T05:29:41.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.731 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595575+0000", 2024-09-14T05:29:41.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.732 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.732 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595575+0000", 2024-09-14T05:29:41.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.732 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.732 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595575+0000", 2024-09-14T05:29:41.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4000000000000002e-07 2024-09-14T05:29:41.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595577+0000", 2024-09-14T05:29:41.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.007e-06 2024-09-14T05:29:41.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595583+0000", 2024-09-14T05:29:41.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8300000000000001e-06 2024-09-14T05:29:41.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595588+0000", 2024-09-14T05:29:41.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6120000000000001e-06 2024-09-14T05:29:41.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.734 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.613162+0000", 2024-09-14T05:29:41.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017574320000000001 2024-09-14T05:29:41.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.734 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.613216+0000", 2024-09-14T05:29:41.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3897000000000003e-05 2024-09-14T05:29:41.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.613276+0000", 2024-09-14T05:29:41.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9599999999999999e-05 2024-09-14T05:29:41.735 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.735 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.735 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.735 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7641 3.10 3:088d6032:::rbd_data.112d114b4eae.0000000000000002:8 [read 720896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.735 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.595594+0000", 2024-09-14T05:29:41.735 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.049433617, 2024-09-14T05:29:41.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017819452999999999, 2024-09-14T05:29:41.736 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.736 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.736 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.736 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.736 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.736 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.736 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7641 2024-09-14T05:29:41.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.736 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595594+0000", 2024-09-14T05:29:41.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595594+0000", 2024-09-14T05:29:41.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595594+0000", 2024-09-14T05:29:41.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7800000000000002e-07 2024-09-14T05:29:41.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.738 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.738 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595596+0000", 2024-09-14T05:29:41.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9439999999999999e-06 2024-09-14T05:29:41.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.738 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.738 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595602+0000", 2024-09-14T05:29:41.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8749999999999997e-06 2024-09-14T05:29:41.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.738 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.738 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595607+0000", 2024-09-14T05:29:41.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6129999999999997e-06 2024-09-14T05:29:41.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.613305+0000", 2024-09-14T05:29:41.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017698242999999999 2024-09-14T05:29:41.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.613342+0000", 2024-09-14T05:29:41.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7273e-05 2024-09-14T05:29:41.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.613414+0000", 2024-09-14T05:29:41.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1126999999999998e-05 2024-09-14T05:29:41.740 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.740 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.740 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.740 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:7642 3.10 3:088d6032:::rbd_data.112d114b4eae.0000000000000002:8 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T05:29:41.740 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:35.595616+0000", 2024-09-14T05:29:41.740 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 6.0494121200000004, 2024-09-14T05:29:41.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017952414999999999, 2024-09-14T05:29:41.740 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:41.740 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:41.740 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:41.741 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:41.741 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:41.741 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:41.741 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 7642 2024-09-14T05:29:41.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.741 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:41.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.741 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:41.741 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595616+0000", 2024-09-14T05:29:41.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.741 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:41.741 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595616+0000", 2024-09-14T05:29:41.742 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:41.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:41.742 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595616+0000", 2024-09-14T05:29:41.742 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:29:41.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:41.742 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595618+0000", 2024-09-14T05:29:41.742 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.688e-06 2024-09-14T05:29:41.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:41.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595623+0000", 2024-09-14T05:29:41.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6060000000000002e-06 2024-09-14T05:29:41.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:41.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.595635+0000", 2024-09-14T05:29:41.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1435e-05 2024-09-14T05:29:41.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:41.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.613431+0000", 2024-09-14T05:29:41.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.017796725999999999 2024-09-14T05:29:41.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:41.744 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.613486+0000", 2024-09-14T05:29:41.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5037999999999997e-05 2024-09-14T05:29:41.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:41.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:41.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:41.744 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:35.613568+0000", 2024-09-14T05:29:41.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1561000000000001e-05 2024-09-14T05:29:41.744 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.744 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.744 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.744 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:41.745 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:41.745 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:41.746 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:29:41.747 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:29:41.747 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:29:41.747 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:29:41.748 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:29:41.748 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:29:41.748 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:29:41.749 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:29:41.749 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:29:41.749 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:29:41.749 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:29:41.750 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:29:41.750 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:29:41.750 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:41.745+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:41.759 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:29:41.759 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:29:41.776 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:29:41.776 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:29:41.847 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:41.847+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:41.862 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:41.862 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:41.871 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:29:41.948 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:41.947+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:41.973 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:41.973 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:41.981 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:29:42.048 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:42.048+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:42.083 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:42.084 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:42.092 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:29:42.149 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:42.148+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:42.174 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone11 from 0x6ea056c (overlap 0x6ea056c) to 0x1620115 2024-09-14T05:29:42.207 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:42.207 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:42.217 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:29:42.245 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-14T05:29:42.250 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:42.249+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:42.351 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:42.350+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:42.360 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:42.360 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:42.369 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:29:42.451 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:42.451+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:42.476 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:42.476 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:42.486 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:29:42.486 INFO:teuthology.orchestra.run.smithi071.stdout:1159 trunc from 0x6ea056c to 0x4e353fe 2024-09-14T05:29:42.500 INFO:teuthology.orchestra.run.smithi071.stdout:1160 read 0x47875e9 thru 0x478cb19 (0x5531 bytes) 2024-09-14T05:29:42.500 INFO:teuthology.orchestra.run.smithi071.stdout:1161 read 0x13d8717 thru 0x13e1f25 (0x980f bytes) 2024-09-14T05:29:42.501 INFO:teuthology.orchestra.run.smithi071.stdout:1162 write 0x2f962ea thru 0x2f9cd85 (0x6a9c bytes) 2024-09-14T05:29:42.505 INFO:teuthology.orchestra.run.smithi071.stdout:1164 read 0x3ae2958 thru 0x3ae2e74 (0x51d bytes) 2024-09-14T05:29:42.505 INFO:teuthology.orchestra.run.smithi071.stdout:1165 write 0x3c78e16 thru 0x3c80c86 (0x7e71 bytes) 2024-09-14T05:29:42.512 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:42.512 INFO:teuthology.orchestra.run.smithi071.stdout:1167 read 0x2c5c2ca thru 0x2c68658 (0xc38f bytes) 2024-09-14T05:29:42.512 INFO:teuthology.orchestra.run.smithi071.stdout:1168 write 0x1f191bf thru 0x1f1ab12 (0x1954 bytes) 2024-09-14T05:29:42.526 INFO:teuthology.orchestra.run.smithi071.stdout:1171 write 0x793e036 thru 0x794cbc4 (0xeb8f bytes) 2024-09-14T05:29:42.545 INFO:teuthology.orchestra.run.smithi071.stdout:1172 writesame 0xd8c6b0e thru 0xd8d4467 data_size 0x2b(0xd95a bytes) 2024-09-14T05:29:42.549 INFO:teuthology.orchestra.run.smithi071.stdout:1173 trunc from 0xd8d4468 to 0xa766d93 2024-09-14T05:29:42.552 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:42.552+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:42.555 INFO:teuthology.orchestra.run.smithi071.stdout:1174 writesame 0x4530afe thru 0x45373df data_size 0x4b(0x68e2 bytes) 2024-09-14T05:29:42.566 INFO:teuthology.orchestra.run.smithi071.stdout:1175 trunc from 0xa766d93 to 0x44445a6 2024-09-14T05:29:42.600 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:42.600 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:42.603 INFO:teuthology.orchestra.run.smithi071.stdout:1178 write 0xc42e962 thru 0xc434fa6 (0x6645 bytes) 2024-09-14T05:29:42.606 INFO:teuthology.orchestra.run.smithi071.stdout:1179 writesame 0x752267c thru 0x7527b86 data_size 0x3b(0x550b bytes) 2024-09-14T05:29:42.609 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:42.609 INFO:teuthology.orchestra.run.smithi071.stdout:1181 write 0x33a998e thru 0x33b69e4 (0xd057 bytes) 2024-09-14T05:29:42.610 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:29:42.613 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:42.613 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:42.613 INFO:teuthology.orchestra.run.smithi071.stdout:1184 read 0x3b34dc5 thru 0x3b36726 (0x1962 bytes) 2024-09-14T05:29:42.613 INFO:teuthology.orchestra.run.smithi071.stdout:1186 writesame 0xbca6987 thru 0xbca7e7f data_size 0xd(0x14f9 bytes) 2024-09-14T05:29:42.615 INFO:teuthology.orchestra.run.smithi071.stdout:1187 read 0x8b232b2 thru 0x8b24923 (0x1672 bytes) 2024-09-14T05:29:42.615 INFO:teuthology.orchestra.run.smithi071.stdout:1188 read 0xbb34709 thru 0xbb37fe2 (0x38da bytes) 2024-09-14T05:29:42.616 INFO:teuthology.orchestra.run.smithi071.stdout:1189 read 0x757cd45 thru 0x75845b2 (0x786e bytes) 2024-09-14T05:29:42.616 INFO:teuthology.orchestra.run.smithi071.stdout:1190 writesame 0x50c7475 thru 0x50cee70 data_size 0x4(0x79fc bytes) 2024-09-14T05:29:42.621 INFO:teuthology.orchestra.run.smithi071.stdout:1191 writesame 0x1ce2235 thru 0x1ce79c1 data_size 0x1f(0x578d bytes) 2024-09-14T05:29:42.624 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:42.624 INFO:teuthology.orchestra.run.smithi071.stdout:1194 writesame 0x8c8ccea thru 0x8c8f718 data_size 0x1(0x2a2f bytes) 2024-09-14T05:29:42.633 INFO:teuthology.orchestra.run.smithi071.stdout:1195 punch from 0x8a13106 to 0x8a1fbcf, (0xcac9 bytes) 2024-09-14T05:29:42.633 INFO:teuthology.orchestra.run.smithi071.stdout:1196 punch from 0x5889f18 to 0x5892bf2, (0x8cda bytes) 2024-09-14T05:29:42.641 INFO:teuthology.orchestra.run.smithi071.stdout:1199 write 0xed53286 thru 0xed5adaa (0x7b25 bytes) 2024-09-14T05:29:42.643 INFO:teuthology.orchestra.run.smithi071.stdout:1200 write 0x39f176c thru 0x39ffc9d (0xe532 bytes) 2024-09-14T05:29:42.647 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:42.647 INFO:teuthology.orchestra.run.smithi071.stdout:1204 write 0x5110c43 thru 0x5112eb7 (0x2275 bytes) 2024-09-14T05:29:42.650 INFO:teuthology.orchestra.run.smithi071.stdout:1207 writesame 0x6ee4de0 thru 0x6ef1d52 data_size 0x1(0xcf73 bytes) 2024-09-14T05:29:42.654 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:42.653+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:42.677 INFO:teuthology.orchestra.run.smithi071.stdout:1208 read 0xa3679ed thru 0xa375b67 (0xe17b bytes) 2024-09-14T05:29:42.677 INFO:teuthology.orchestra.run.smithi071.stdout:1209 read 0x33a3a79 thru 0x33a7cd8 (0x4260 bytes) 2024-09-14T05:29:42.678 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:42.678 INFO:teuthology.orchestra.run.smithi071.stdout:1211 write 0x61929fa thru 0x61a0ade (0xe0e5 bytes) 2024-09-14T05:29:42.684 INFO:teuthology.orchestra.run.smithi071.stdout:1212 write 0x6e9e023 thru 0x6eaad4b (0xcd29 bytes) 2024-09-14T05:29:42.691 INFO:teuthology.orchestra.run.smithi071.stdout:1213 trunc from 0xed5adab to 0xb170a12 2024-09-14T05:29:42.701 INFO:teuthology.orchestra.run.smithi071.stdout:1215 read 0x6aeff78 thru 0x6afc1fa (0xc283 bytes) 2024-09-14T05:29:42.701 INFO:teuthology.orchestra.run.smithi071.stdout:1217 read 0x7b0f420 thru 0x7b11a77 (0x2658 bytes) 2024-09-14T05:29:42.701 INFO:teuthology.orchestra.run.smithi071.stdout:1218 writesame 0x89f1574 thru 0x89f5bcf data_size 0x13(0x465c bytes) 2024-09-14T05:29:42.704 INFO:teuthology.orchestra.run.smithi071.stdout:1219 punch from 0x2cd642a to 0x2cdb5d9, (0x51af bytes) 2024-09-14T05:29:42.705 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:42.705 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:42.705 INFO:teuthology.orchestra.run.smithi071.stdout:1222 punch from 0x6f2cdf7 to 0x6f2d8a5, (0xaae bytes) 2024-09-14T05:29:42.705 INFO:teuthology.orchestra.run.smithi071.stdout:1223 read 0x66062a6 thru 0x6615634 (0xf38f bytes) 2024-09-14T05:29:42.705 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:42.705 INFO:teuthology.orchestra.run.smithi071.stdout:1226 write 0x5d6dbab thru 0x5d70890 (0x2ce6 bytes) 2024-09-14T05:29:42.709 INFO:teuthology.orchestra.run.smithi071.stdout:1227 trunc from 0xb170a12 to 0x9869e40 2024-09-14T05:29:42.725 INFO:teuthology.orchestra.run.smithi071.stdout:1228 trunc from 0x9869e40 to 0x5918d7 2024-09-14T05:29:42.748 INFO:teuthology.orchestra.run.smithi071.stdout:1232 trunc from 0x5918d7 to 0x380ef00 2024-09-14T05:29:42.754 INFO:teuthology.orchestra.run.smithi071.stdout:1233 read 0x32f57ed thru 0x330180d (0xc021 bytes) 2024-09-14T05:29:42.754 INFO:teuthology.orchestra.run.smithi071.stdout:1234 read 0xe3e9be thru 0xe49bb1 (0xb1f4 bytes) 2024-09-14T05:29:42.754 INFO:teuthology.orchestra.run.smithi071.stdout:1235 punch from 0x11ad4bb to 0x11b8d7c, (0xb8c1 bytes) 2024-09-14T05:29:42.754 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:42.753+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:42.762 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:42.762 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:42.767 INFO:teuthology.orchestra.run.smithi071.stdout:1236 write 0xa358588 thru 0xa358dc1 (0x83a bytes) 2024-09-14T05:29:42.770 INFO:teuthology.orchestra.run.smithi071.stdout:1238 write 0x3af653d thru 0x3b0439d (0xde61 bytes) 2024-09-14T05:29:42.772 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:29:42.779 INFO:teuthology.orchestra.run.smithi071.stdout:1240 write 0xa40b836 thru 0xa41a9e1 (0xf1ac bytes) 2024-09-14T05:29:42.790 INFO:teuthology.orchestra.run.smithi071.stdout:1241 write 0xb209549 thru 0xb217e21 (0xe8d9 bytes) 2024-09-14T05:29:42.794 INFO:teuthology.orchestra.run.smithi071.stdout:1242 punch from 0x9548401 to 0x95543b7, (0xbfb6 bytes) 2024-09-14T05:29:42.794 INFO:teuthology.orchestra.run.smithi071.stdout:1245 write 0x406bc0e thru 0x4073a40 (0x7e33 bytes) 2024-09-14T05:29:42.796 INFO:teuthology.orchestra.run.smithi071.stdout:1246 write 0x5894104 thru 0x58a0b16 (0xca13 bytes) 2024-09-14T05:29:42.807 INFO:teuthology.orchestra.run.smithi071.stdout:1247 write 0xcb753bb thru 0xcb7d518 (0x815e bytes) 2024-09-14T05:29:42.811 INFO:teuthology.orchestra.run.smithi071.stdout:1249 punch from 0xb29c915 to 0xb29ee66, (0x2551 bytes) 2024-09-14T05:29:42.813 INFO:teuthology.orchestra.run.smithi071.stdout:1250 trunc from 0xcb7d519 to 0xb6a4749 2024-09-14T05:29:42.823 INFO:teuthology.orchestra.run.smithi071.stdout:1251 trunc from 0xb6a4749 to 0xc69bded 2024-09-14T05:29:42.839 INFO:teuthology.orchestra.run.smithi071.stdout:1252 trunc from 0xc69bded to 0x28432c9 2024-09-14T05:29:42.854 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:42.854+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:42.866 INFO:teuthology.orchestra.run.smithi071.stdout:1253 write 0x90ba058 thru 0x90c768f (0xd638 bytes) 2024-09-14T05:29:42.878 INFO:teuthology.orchestra.run.smithi071.stdout:1254 trunc from 0x90c7690 to 0x423c3db 2024-09-14T05:29:42.885 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:42.885 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:42.894 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:29:42.897 INFO:teuthology.orchestra.run.smithi071.stdout:1255 trunc from 0x423c3db to 0xa0d65ae 2024-09-14T05:29:42.905 INFO:teuthology.orchestra.run.smithi071.stdout:1256 write 0x106f7e1 thru 0x107bd49 (0xc569 bytes) 2024-09-14T05:29:42.908 INFO:teuthology.orchestra.run.smithi071.stdout:1257 write 0x7cfa771 thru 0x7d06721 (0xbfb1 bytes) 2024-09-14T05:29:42.912 INFO:teuthology.orchestra.run.smithi071.stdout:1258 write 0x4bab385 thru 0x4bb51bf (0x9e3b bytes) 2024-09-14T05:29:42.915 INFO:teuthology.orchestra.run.smithi071.stdout:1259 punch from 0x938938 to 0x9399b9, (0x1081 bytes) 2024-09-14T05:29:42.915 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:42.915 INFO:teuthology.orchestra.run.smithi071.stdout:1261 trunc from 0xa0d65ae to 0xa2c955a 2024-09-14T05:29:42.921 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:42.921 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:42.921 INFO:teuthology.orchestra.run.smithi071.stdout:1264 write 0x214eb68 thru 0x21502db (0x1774 bytes) 2024-09-14T05:29:42.930 INFO:teuthology.orchestra.run.smithi071.stdout:1266 trunc from 0xa2c955a to 0x4de7ef1 2024-09-14T05:29:42.941 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:42.941 INFO:teuthology.orchestra.run.smithi071.stdout:1268 read 0x4cf7f0d thru 0x4d02502 (0xa5f6 bytes) 2024-09-14T05:29:42.941 INFO:teuthology.orchestra.run.smithi071.stdout:1271 read 0x3a33535 thru 0x3a3e9f3 (0xb4bf bytes) 2024-09-14T05:29:42.941 INFO:teuthology.orchestra.run.smithi071.stdout:1274 punch from 0xd067c to 0xd60f8, (0x5a7c bytes) 2024-09-14T05:29:42.941 INFO:teuthology.orchestra.run.smithi071.stdout:1275 write 0x30fef25 thru 0x310eb34 (0xfc10 bytes) 2024-09-14T05:29:42.948 INFO:teuthology.orchestra.run.smithi071.stdout:1276 punch from 0x382f91b to 0x3832ca4, (0x3389 bytes) 2024-09-14T05:29:42.948 INFO:teuthology.orchestra.run.smithi071.stdout:1277 trunc from 0x4de7ef1 to 0x56efc0a 2024-09-14T05:29:42.953 INFO:teuthology.orchestra.run.smithi071.stdout:1278 read 0x33a201c thru 0x33b00fb (0xe0e0 bytes) 2024-09-14T05:29:42.953 INFO:teuthology.orchestra.run.smithi071.stdout:1279 writesame 0x4ec52bd thru 0x4ec533f data_size 0x1(0x83 bytes) 2024-09-14T05:29:42.956 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:42.955+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:42.972 INFO:teuthology.orchestra.run.smithi071.stdout:1280 write 0xe1fc0af thru 0xe20292e (0x6880 bytes) 2024-09-14T05:29:42.975 INFO:teuthology.orchestra.run.smithi071.stdout:1282 read 0x83b7851 thru 0x83c097a (0x912a bytes) 2024-09-14T05:29:42.989 INFO:teuthology.orchestra.run.smithi071.stdout:1284 trunc from 0xe20292f to 0x26e72f2 2024-09-14T05:29:42.999 INFO:teuthology.orchestra.run.smithi071.stdout:1286 punch from 0x231cc68 to 0x2329649, (0xc9e1 bytes) 2024-09-14T05:29:42.999 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:42.999 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.007 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:43.007 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:43.012 INFO:teuthology.orchestra.run.smithi071.stdout:1289 write 0x8ab3972 thru 0x8ac042e (0xcabd bytes) 2024-09-14T05:29:43.017 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:29:43.017 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.022 INFO:teuthology.orchestra.run.smithi071.stdout:1291 trunc from 0x8ac042f to 0x455d938 2024-09-14T05:29:43.033 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.033 INFO:teuthology.orchestra.run.smithi071.stdout:1294 clone 13 order 24 su 65536 sc 8 2024-09-14T05:29:43.056 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:43.056+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:43.123 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:43.123 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:43.132 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:29:43.175 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone12 from 0x455d938 (overlap 0x5918d7) to 0x3e5e30 2024-09-14T05:29:43.176 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T05:29:43.177 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:43.176+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:43.239 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-14T05:29:43.244 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:43.244 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:43.253 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:29:43.267 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.267 INFO:teuthology.orchestra.run.smithi071.stdout:1297 write 0x42b5642 thru 0x42b7846 (0x2205 bytes) 2024-09-14T05:29:43.276 INFO:teuthology.orchestra.run.smithi071.stdout:1298 read 0x15ae5b4 thru 0x15bc17a (0xdbc7 bytes) 2024-09-14T05:29:43.277 INFO:teuthology.orchestra.run.smithi071.stdout:1300 read 0x363f78a thru 0x364c5d3 (0xce4a bytes) 2024-09-14T05:29:43.277 INFO:teuthology.orchestra.run.smithi071.stdout:1303 read 0x1079706 thru 0x10797c1 (0xbc bytes) 2024-09-14T05:29:43.290 INFO:teuthology.orchestra.run.smithi071.stdout:1305 writesame 0xaf27caa thru 0xaf29ac7 data_size 0xf(0x1e1e bytes) 2024-09-14T05:29:43.290 INFO:teuthology.orchestra.run.smithi071.stdout:1306 writesame 0x2fb7c5c thru 0x2fc4681 data_size 0x96(0xca26 bytes) 2024-09-14T05:29:43.295 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:43.294+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:43.297 INFO:teuthology.orchestra.run.smithi071.stdout:1307 read 0x9b4d413 thru 0x9b535c7 (0x61b5 bytes) 2024-09-14T05:29:43.297 INFO:teuthology.orchestra.run.smithi071.stdout:1311 punch from 0x3182442 to 0x3190497, (0xe055 bytes) 2024-09-14T05:29:43.297 INFO:teuthology.orchestra.run.smithi071.stdout:1312 punch from 0x43a9883 to 0x43aff4f, (0x66cc bytes) 2024-09-14T05:29:43.302 INFO:teuthology.orchestra.run.smithi071.stdout:1313 write 0xb40b33b thru 0xb40ea15 (0x36db bytes) 2024-09-14T05:29:43.305 INFO:teuthology.orchestra.run.smithi071.stdout:1314 read 0x372ad1b thru 0x3734e7c (0xa162 bytes) 2024-09-14T05:29:43.313 INFO:teuthology.orchestra.run.smithi071.stdout:1316 trunc from 0xb40ea16 to 0x4de1f53 2024-09-14T05:29:43.342 INFO:teuthology.orchestra.run.smithi071.stdout:1317 punch from 0x2d5541e to 0x2d5a90d, (0x54ef bytes) 2024-09-14T05:29:43.351 INFO:teuthology.orchestra.run.smithi071.stdout:1318 write 0x7855715 thru 0x785fe2d (0xa719 bytes) 2024-09-14T05:29:43.359 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:43.358+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:43.362 INFO:teuthology.orchestra.run.smithi071.stdout:1319 write 0xb98efb2 thru 0xb99d50b (0xe55a bytes) 2024-09-14T05:29:43.366 INFO:teuthology.orchestra.run.smithi071.stdout:1320 read 0x7b52b94 thru 0x7b56468 (0x38d5 bytes) 2024-09-14T05:29:43.366 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.366 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.366 INFO:teuthology.orchestra.run.smithi071.stdout:1326 write 0x52eb6d thru 0x53cb79 (0xe00d bytes) 2024-09-14T05:29:43.370 INFO:teuthology.orchestra.run.smithi071.stdout:1327 trunc from 0xb99d50c to 0x99fb73e 2024-09-14T05:29:43.377 INFO:teuthology.orchestra.run.smithi071.stdout:1328 punch from 0x851cb68 to 0x85229eb, (0x5e83 bytes) 2024-09-14T05:29:43.377 INFO:teuthology.orchestra.run.smithi071.stdout:1329 punch from 0x7be5813 to 0x7be665f, (0xe4c bytes) 2024-09-14T05:29:43.388 INFO:teuthology.orchestra.run.smithi071.stdout:1330 write 0xe74249d thru 0xe748e50 (0x69b4 bytes) 2024-09-14T05:29:43.390 INFO:teuthology.orchestra.run.smithi071.stdout:1332 read 0xac8d67e thru 0xac981a7 (0xab2a bytes) 2024-09-14T05:29:43.391 INFO:teuthology.orchestra.run.smithi071.stdout:1333 write 0x78c3c03 thru 0x78ceecd (0xb2cb bytes) 2024-09-14T05:29:43.393 INFO:teuthology.orchestra.run.smithi071.stdout:1334 write 0xa018f18 thru 0xa01c5a5 (0x368e bytes) 2024-09-14T05:29:43.399 INFO:teuthology.orchestra.run.smithi071.stdout:1335 writesame 0xec37f42 thru 0xec3ca2b data_size 0x2(0x4aea bytes) 2024-09-14T05:29:43.399 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.399 INFO:teuthology.orchestra.run.smithi071.stdout:1337 write 0x7450b83 thru 0x745f2b9 (0xe737 bytes) 2024-09-14T05:29:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:1339 read 0xebdc721 thru 0xebdd505 (0xde5 bytes) 2024-09-14T05:29:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:1340 write 0xc149293 thru 0xc15498c (0xb6fa bytes) 2024-09-14T05:29:43.402 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:43.402 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:43.403 INFO:teuthology.orchestra.run.smithi071.stdout:1341 write 0x9996c73 thru 0x999c5de (0x596c bytes) 2024-09-14T05:29:43.405 INFO:teuthology.orchestra.run.smithi071.stdout:1342 write 0x99afd8 thru 0x9a1ff1 (0x701a bytes) 2024-09-14T05:29:43.406 INFO:teuthology.orchestra.run.smithi071.stdout:1344 read 0x86a0336 thru 0x86ae782 (0xe44d bytes) 2024-09-14T05:29:43.406 INFO:teuthology.orchestra.run.smithi071.stdout:1345 punch from 0x7371976 to 0x737564f, (0x3cd9 bytes) 2024-09-14T05:29:43.407 INFO:teuthology.orchestra.run.smithi071.stdout:1346 write 0x8cad0f4 thru 0x8cb9f40 (0xce4d bytes) 2024-09-14T05:29:43.412 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:29:43.421 INFO:teuthology.orchestra.run.smithi071.stdout:1347 trunc from 0xec3ca2c to 0x67283a1 2024-09-14T05:29:43.434 INFO:teuthology.orchestra.run.smithi071.stdout:1348 write 0x39aa349 thru 0x39af67a (0x5332 bytes) 2024-09-14T05:29:43.435 INFO:teuthology.orchestra.run.smithi071.stdout:1349 read 0x477687e thru 0x47782ea (0x1a6d bytes) 2024-09-14T05:29:43.435 INFO:teuthology.orchestra.run.smithi071.stdout:1350 punch from 0x54863c2 to 0x548e7c1, (0x83ff bytes) 2024-09-14T05:29:43.444 INFO:teuthology.orchestra.run.smithi071.stdout:1351 write 0x91772b8 thru 0x917c815 (0x555e bytes) 2024-09-14T05:29:43.447 INFO:teuthology.orchestra.run.smithi071.stdout:1353 read 0x33fd31f thru 0x340284f (0x5531 bytes) 2024-09-14T05:29:43.456 INFO:teuthology.orchestra.run.smithi071.stdout:1354 writesame 0xc01da44 thru 0xc020ad4 data_size 0x1(0x3091 bytes) 2024-09-14T05:29:43.456 INFO:teuthology.orchestra.run.smithi071.stdout:1355 flatten 2024-09-14T05:29:43.469 INFO:teuthology.orchestra.run.smithi071.stdout:1357 write 0xdf713cf thru 0xdf7c573 (0xb1a5 bytes) 2024-09-14T05:29:43.470 INFO:teuthology.orchestra.run.smithi071.stdout:1358 writesame 0xc16a000 thru 0xc172d93 data_size 0x52(0x8d94 bytes) 2024-09-14T05:29:43.473 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.473 INFO:teuthology.orchestra.run.smithi071.stdout:1360 write 0x8ba1e6c thru 0x8bb03fe (0xe593 bytes) 2024-09-14T05:29:43.477 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.477 INFO:teuthology.orchestra.run.smithi071.stdout:1362 write 0x6686ae9 thru 0x6687cef (0x1207 bytes) 2024-09-14T05:29:43.478 INFO:teuthology.orchestra.run.smithi071.stdout:1363 punch from 0x9dc4dad to 0x9dcc9ec, (0x7c3f bytes) 2024-09-14T05:29:43.478 INFO:teuthology.orchestra.run.smithi071.stdout:1364 write 0x18add87 thru 0x18b63ce (0x8648 bytes) 2024-09-14T05:29:43.479 INFO:teuthology.orchestra.run.smithi071.stdout:1365 punch from 0xd932acd to 0xd932cea, (0x21d bytes) 2024-09-14T05:29:43.490 INFO:teuthology.orchestra.run.smithi071.stdout:1367 trunc from 0xdf7c574 to 0x503877f 2024-09-14T05:29:43.496 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:43.496+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:43.520 INFO:teuthology.orchestra.run.smithi071.stdout:1369 writesame 0xe3a83a7 thru 0xe3b3f6e data_size 0xc(0xbbc8 bytes) 2024-09-14T05:29:43.520 INFO:teuthology.orchestra.run.smithi071.stdout:1374 read 0x8570241 thru 0x8577839 (0x75f9 bytes) 2024-09-14T05:29:43.529 INFO:teuthology.orchestra.run.smithi071.stdout:1376 trunc from 0xe3b3f6f to 0x5b8dd2b 2024-09-14T05:29:43.543 INFO:teuthology.orchestra.run.smithi071.stdout:1380 write 0x7305d1d thru 0x730a7cf (0x4ab3 bytes) 2024-09-14T05:29:43.555 INFO:teuthology.orchestra.run.smithi071.stdout:1381 write 0xc7079cf thru 0xc7104a6 (0x8ad8 bytes) 2024-09-14T05:29:43.560 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:43.561 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:43.561 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:43.560+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:43.565 INFO:teuthology.orchestra.run.smithi071.stdout:1382 trunc from 0xc7104a7 to 0x95395c5 2024-09-14T05:29:43.570 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:29:43.571 INFO:teuthology.orchestra.run.smithi071.stdout:1383 read 0x7cb94a6 thru 0x7cc2ee6 (0x9a41 bytes) 2024-09-14T05:29:43.578 INFO:teuthology.orchestra.run.smithi071.stdout:1384 write 0xa3697ed thru 0xa36db18 (0x432c bytes) 2024-09-14T05:29:43.582 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.592 INFO:teuthology.orchestra.run.smithi071.stdout:1387 trunc from 0xa36db19 to 0x2013b92 2024-09-14T05:29:43.605 INFO:teuthology.orchestra.run.smithi071.stdout:1388 punch from 0x164e9dd to 0x165735f, (0x8982 bytes) 2024-09-14T05:29:43.606 INFO:teuthology.orchestra.run.smithi071.stdout:1389 trunc from 0x2013b92 to 0x17a1ff6 2024-09-14T05:29:43.615 INFO:teuthology.orchestra.run.smithi071.stdout:1391 punch from 0x1266bfa to 0x12746c0, (0xdac6 bytes) 2024-09-14T05:29:43.615 INFO:teuthology.orchestra.run.smithi071.stdout:1392 read 0x139ab90 thru 0x139f5e9 (0x4a5a bytes) 2024-09-14T05:29:43.615 INFO:teuthology.orchestra.run.smithi071.stdout:1393 read 0xe2afad thru 0xe39aa8 (0xeafc bytes) 2024-09-14T05:29:43.616 INFO:teuthology.orchestra.run.smithi071.stdout:1395 read 0xfde67e thru 0xfedae3 (0xf466 bytes) 2024-09-14T05:29:43.616 INFO:teuthology.orchestra.run.smithi071.stdout:1397 read 0xaf5ed5 thru 0xb04945 (0xea71 bytes) 2024-09-14T05:29:43.616 INFO:teuthology.orchestra.run.smithi071.stdout:1398 read 0xdfdca8 thru 0xe0265e (0x49b7 bytes) 2024-09-14T05:29:43.625 INFO:teuthology.orchestra.run.smithi071.stdout:1399 write 0x3eb10e7 thru 0x3ebc2ba (0xb1d4 bytes) 2024-09-14T05:29:43.626 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.637 INFO:teuthology.orchestra.run.smithi071.stdout:1402 writesame 0x8cfab27 thru 0x8d03215 data_size 0x1(0x86ef bytes) 2024-09-14T05:29:43.637 INFO:teuthology.orchestra.run.smithi071.stdout:1404 read 0x7a63ff0 thru 0x7a73d55 (0xfd66 bytes) 2024-09-14T05:29:43.637 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.648 INFO:teuthology.orchestra.run.smithi071.stdout:1408 writesame 0xe2a8cb2 thru 0xe2b68e7 data_size 0x47(0xdc36 bytes) 2024-09-14T05:29:43.648 INFO:teuthology.orchestra.run.smithi071.stdout:1412 read 0x74b9cd3 thru 0x74c593b (0xbc69 bytes) 2024-09-14T05:29:43.649 INFO:teuthology.orchestra.run.smithi071.stdout:1413 write 0x669ee2e thru 0x66a80c1 (0x9294 bytes) 2024-09-14T05:29:43.650 INFO:teuthology.orchestra.run.smithi071.stdout:1414 writesame 0xdb1c7e6 thru 0xdb2aeae data_size 0x29(0xe6c9 bytes) 2024-09-14T05:29:43.656 INFO:teuthology.orchestra.run.smithi071.stdout:1415 write 0x33f9d74 thru 0x33fb285 (0x1512 bytes) 2024-09-14T05:29:43.656 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.662 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:43.661+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:43.665 INFO:teuthology.orchestra.run.smithi071.stdout:1417 trunc from 0xe2b68e8 to 0x6d87619 2024-09-14T05:29:43.678 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.678 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.678 INFO:teuthology.orchestra.run.smithi071.stdout:1422 punch from 0x208feb3 to 0x2092577, (0x26c4 bytes) 2024-09-14T05:29:43.679 INFO:teuthology.orchestra.run.smithi071.stdout:1423 write 0x16510ef thru 0x165d0b8 (0xbfca bytes) 2024-09-14T05:29:43.679 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.680 INFO:teuthology.orchestra.run.smithi071.stdout:1428 read 0x1eccf2 thru 0x1f0ba4 (0x3eb3 bytes) 2024-09-14T05:29:43.680 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.680 INFO:teuthology.orchestra.run.smithi071.stdout:1430 punch from 0x70647b to 0x70c497, (0x601c bytes) 2024-09-14T05:29:43.681 INFO:teuthology.orchestra.run.smithi071.stdout:1431 read 0x66e92e thru 0x67cdb6 (0xe489 bytes) 2024-09-14T05:29:43.686 INFO:teuthology.orchestra.run.smithi071.stdout:1432 writesame 0x769060d thru 0x76995fb data_size 0x1(0x8fef bytes) 2024-09-14T05:29:43.692 INFO:teuthology.orchestra.run.smithi071.stdout:1433 trunc from 0x76995fc to 0xbadd757 2024-09-14T05:29:43.697 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.698 INFO:teuthology.orchestra.run.smithi071.stdout:1435 punch from 0x1b36297 to 0x1b454e3, (0xf24c bytes) 2024-09-14T05:29:43.698 INFO:teuthology.orchestra.run.smithi071.stdout:1436 write 0x1424059 thru 0x1427058 (0x3000 bytes) 2024-09-14T05:29:43.699 INFO:teuthology.orchestra.run.smithi071.stdout:1437 write 0x7e6ffff thru 0x7e7fc87 (0xfc89 bytes) 2024-09-14T05:29:43.701 INFO:teuthology.orchestra.run.smithi071.stdout:1438 read 0xaeac529 thru 0xaeb4350 (0x7e28 bytes) 2024-09-14T05:29:43.701 INFO:teuthology.orchestra.run.smithi071.stdout:1440 read 0x989bdd6 thru 0x98a1445 (0x5670 bytes) 2024-09-14T05:29:43.701 INFO:teuthology.orchestra.run.smithi071.stdout:1442 write 0x35f7dfb thru 0x35fa4d1 (0x26d7 bytes) 2024-09-14T05:29:43.702 INFO:teuthology.orchestra.run.smithi071.stdout:1443 punch from 0xb86b976 to 0xb8785bf, (0xcc49 bytes) 2024-09-14T05:29:43.702 INFO:teuthology.orchestra.run.smithi071.stdout:1444 writesame 0x6646fba thru 0x6655da7 data_size 0x5(0xedee bytes) 2024-09-14T05:29:43.706 INFO:teuthology.orchestra.run.smithi071.stdout:1445 punch from 0x5c2068e to 0x5c2a437, (0x9da9 bytes) 2024-09-14T05:29:43.706 INFO:teuthology.orchestra.run.smithi071.stdout:1446 read 0x3af3192 thru 0x3b0140d (0xe27c bytes) 2024-09-14T05:29:43.707 INFO:teuthology.orchestra.run.smithi071.stdout:1447 writesame 0x91cb401 thru 0x91d7a00 data_size 0xc0(0xc600 bytes) 2024-09-14T05:29:43.710 INFO:teuthology.orchestra.run.smithi071.stdout:1448 write 0x80905b0 thru 0x8094e38 (0x4889 bytes) 2024-09-14T05:29:43.720 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:43.720 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:43.722 INFO:teuthology.orchestra.run.smithi071.stdout:1449 trunc from 0xbadd757 to 0x5017abe 2024-09-14T05:29:43.730 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:29:43.734 INFO:teuthology.orchestra.run.smithi071.stdout:1450 read 0x49be185 thru 0x49bf876 (0x16f2 bytes) 2024-09-14T05:29:43.735 INFO:teuthology.orchestra.run.smithi071.stdout:1451 writesame 0x567903 thru 0x56a9ce data_size 0x12(0x30cc bytes) 2024-09-14T05:29:43.749 INFO:teuthology.orchestra.run.smithi071.stdout:1452 writesame 0xc01154e thru 0xc01dd42 data_size 0x71(0xc7f5 bytes) 2024-09-14T05:29:43.749 INFO:teuthology.orchestra.run.smithi071.stdout:1453 write 0x57031a thru 0x575ee4 (0x5bcb bytes) 2024-09-14T05:29:43.751 INFO:teuthology.orchestra.run.smithi071.stdout:1454 write 0x57a83c5 thru 0x57aaa05 (0x2641 bytes) 2024-09-14T05:29:43.753 INFO:teuthology.orchestra.run.smithi071.stdout:1455 punch from 0x77e1f8d to 0x77e2eca, (0xf3d bytes) 2024-09-14T05:29:43.753 INFO:teuthology.orchestra.run.smithi071.stdout:1456 write 0xb1ea128 thru 0xb1efa8d (0x5966 bytes) 2024-09-14T05:29:43.762 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:43.761+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:43.769 INFO:teuthology.orchestra.run.smithi071.stdout:1459 trunc from 0xc01dd43 to 0xc5c8a6 2024-09-14T05:29:43.787 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:29:43.794 INFO:teuthology.orchestra.run.smithi071.stdout:1460 write 0x93a2b9a thru 0x93a6e7e (0x42e5 bytes) 2024-09-14T05:29:43.797 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.797 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.797 INFO:teuthology.orchestra.run.smithi071.stdout:1466 punch from 0x127b59b to 0x127ed03, (0x3768 bytes) 2024-09-14T05:29:43.797 INFO:teuthology.orchestra.run.smithi071.stdout:1467 writesame 0x28b8dee thru 0x28be996 data_size 0x41(0x5ba9 bytes) 2024-09-14T05:29:43.798 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.806 INFO:teuthology.orchestra.run.smithi071.stdout:1470 write 0xb4f4c7e thru 0xb4fec3d (0x9fc0 bytes) 2024-09-14T05:29:43.810 INFO:teuthology.orchestra.run.smithi071.stdout:1473 write 0x7e860f8 thru 0x7e89292 (0x319b bytes) 2024-09-14T05:29:43.811 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.812 INFO:teuthology.orchestra.run.smithi071.stdout:1475 punch from 0x415f499 to 0x4167ab6, (0x861d bytes) 2024-09-14T05:29:43.812 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.812 INFO:teuthology.orchestra.run.smithi071.stdout:1477 writesame 0x96fa9f1 thru 0x96fe7b9 data_size 0x1(0x3dc9 bytes) 2024-09-14T05:29:43.819 INFO:teuthology.orchestra.run.smithi071.stdout:1478 write 0x39f1545 thru 0x39f2dda (0x1896 bytes) 2024-09-14T05:29:43.820 INFO:teuthology.orchestra.run.smithi071.stdout:1480 read 0x154d46d thru 0x155bdb7 (0xe94b bytes) 2024-09-14T05:29:43.821 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.821 INFO:teuthology.orchestra.run.smithi071.stdout:1482 write 0x75bb11c thru 0x75c254f (0x7434 bytes) 2024-09-14T05:29:43.823 INFO:teuthology.orchestra.run.smithi071.stdout:1484 read 0x1441c7e thru 0x144aa9f (0x8e22 bytes) 2024-09-14T05:29:43.823 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.824 INFO:teuthology.orchestra.run.smithi071.stdout:1486 read 0xb25d318 thru 0xb26c9ea (0xf6d3 bytes) 2024-09-14T05:29:43.824 INFO:teuthology.orchestra.run.smithi071.stdout:1487 writesame 0x302db0e thru 0x303a070 data_size 0x17(0xc563 bytes) 2024-09-14T05:29:43.825 INFO:teuthology.orchestra.run.smithi071.stdout:1488 read 0x110564d thru 0x110d48c (0x7e40 bytes) 2024-09-14T05:29:43.833 INFO:teuthology.orchestra.run.smithi071.stdout:1489 trunc from 0xb4fec3e to 0x548fa4b 2024-09-14T05:29:43.835 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:43.836 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:43.846 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:29:43.856 INFO:teuthology.orchestra.run.smithi071.stdout:1491 write 0xaa88c78 thru 0xaa89b7c (0xf05 bytes) 2024-09-14T05:29:43.859 INFO:teuthology.orchestra.run.smithi071.stdout:1493 read 0xdd8a6f thru 0xde5cfb (0xd28d bytes) 2024-09-14T05:29:43.860 INFO:teuthology.orchestra.run.smithi071.stdout:1496 write 0x9d79729 thru 0x9d85d11 (0xc5e9 bytes) 2024-09-14T05:29:43.862 INFO:teuthology.orchestra.run.smithi071.stdout:1499 read 0x34e6909 thru 0x34f3985 (0xd07d bytes) 2024-09-14T05:29:43.863 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:43.862+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:43.863 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.863 INFO:teuthology.orchestra.run.smithi071.stdout:1502 write 0x17dda92 thru 0x17e29a3 (0x4f12 bytes) 2024-09-14T05:29:43.864 INFO:teuthology.orchestra.run.smithi071.stdout:1504 write 0x4dffa9e thru 0x4e0139c (0x18ff bytes) 2024-09-14T05:29:43.866 INFO:teuthology.orchestra.run.smithi071.stdout:1505 read 0x542bb51 thru 0x5434914 (0x8dc4 bytes) 2024-09-14T05:29:43.866 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.867 INFO:teuthology.orchestra.run.smithi071.stdout:1507 read 0x872a7c4 thru 0x8739117 (0xe954 bytes) 2024-09-14T05:29:43.867 INFO:teuthology.orchestra.run.smithi071.stdout:1510 punch from 0x9f5128a to 0x9f5c5e2, (0xb358 bytes) 2024-09-14T05:29:43.867 INFO:teuthology.orchestra.run.smithi071.stdout:1511 read 0x1b78798 thru 0x1b7b19b (0x2a04 bytes) 2024-09-14T05:29:43.868 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:43.868 INFO:teuthology.orchestra.run.smithi071.stdout:1517 write 0xa1002b1 thru 0xa101901 (0x1651 bytes) 2024-09-14T05:29:43.877 INFO:teuthology.orchestra.run.smithi071.stdout:1518 writesame 0xde90818 thru 0xde9c5f2 data_size 0x11(0xbddb bytes) 2024-09-14T05:29:43.877 INFO:teuthology.orchestra.run.smithi071.stdout:1522 punch from 0x3156c98 to 0x315f241, (0x85a9 bytes) 2024-09-14T05:29:43.889 INFO:teuthology.orchestra.run.smithi071.stdout:1523 trunc from 0xde9c5f3 to 0x42a1b7e 2024-09-14T05:29:43.902 INFO:teuthology.orchestra.run.smithi071.stdout:1524 read 0x1f39dd6 thru 0x1f412f2 (0x751d bytes) 2024-09-14T05:29:43.909 INFO:teuthology.orchestra.run.smithi071.stdout:1525 write 0x4c6ecda thru 0x4c7b171 (0xc498 bytes) 2024-09-14T05:29:43.918 INFO:teuthology.orchestra.run.smithi071.stdout:1526 trunc from 0x4c7b172 to 0x99bc40a 2024-09-14T05:29:43.923 INFO:teuthology.orchestra.run.smithi071.stdout:1527 read 0x4794548 thru 0x47a1e61 (0xd91a bytes) 2024-09-14T05:29:43.923 INFO:teuthology.orchestra.run.smithi071.stdout:1528 read 0x23e78aa thru 0x23f369a (0xbdf1 bytes) 2024-09-14T05:29:43.929 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:29:43.929 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:29:43.932 INFO:teuthology.orchestra.run.smithi071.stdout:1529 write 0xb7b9708 thru 0xb7c764c (0xdf45 bytes) 2024-09-14T05:29:43.937 INFO:teuthology.orchestra.run.smithi071.stdout:1531 clone 14 order 18 su 2048 sc 10 2024-09-14T05:29:43.947 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:29:43.947 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:29:43.954 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:43.954 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:43.963 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:29:43.964 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:29:43.964 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:29:43.964 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:43.963+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:43.979 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:29:43.979 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:29:43.995 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:29:43.995 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:29:44.011 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:29:44.011 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:29:44.027 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:29:44.028 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:29:44.044 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:29:44.044 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:29:44.070 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:44.070 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:29:44.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:29:44.070 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:29:44.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.070 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2131 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:44.070 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:29:44.070 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.294045279000001, 2024-09-14T05:29:44.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049526593000000001, 2024-09-14T05:29:44.071 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.071 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.071 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:44.071 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.071 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.071 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.071 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2131 2024-09-14T05:29:44.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.071 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:29:44.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:29:44.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774867+0000", 2024-09-14T05:29:44.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4430000000000002e-06 2024-09-14T05:29:44.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790357+0000", 2024-09-14T05:29:44.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015489466 2024-09-14T05:29:44.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790383+0000", 2024-09-14T05:29:44.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6247000000000001e-05 2024-09-14T05:29:44.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790394+0000", 2024-09-14T05:29:44.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0728e-05 2024-09-14T05:29:44.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790471+0000", 2024-09-14T05:29:44.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7539000000000004e-05 2024-09-14T05:29:44.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790639+0000", 2024-09-14T05:29:44.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016787700000000001 2024-09-14T05:29:44.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-14T05:29:44.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790998+0000", 2024-09-14T05:29:44.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035931299999999999 2024-09-14T05:29:44.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:44.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.823529+0000", 2024-09-14T05:29:44.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032530798 2024-09-14T05:29:44.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:44.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824324+0000", 2024-09-14T05:29:44.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00079442399999999995 2024-09-14T05:29:44.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:44.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824362+0000", 2024-09-14T05:29:44.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8398000000000001e-05 2024-09-14T05:29:44.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824391+0000", 2024-09-14T05:29:44.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.936e-05 2024-09-14T05:29:44.076 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.076 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.076 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.076 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3892 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7602176~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.076 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:29:44.076 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.712467281, 2024-09-14T05:29:44.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047082951999999997, 2024-09-14T05:29:44.076 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.076 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.076 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.076 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.076 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.077 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3892 2024-09-14T05:29:44.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.077 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:29:44.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:29:44.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:29:44.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0199999999999998e-07 2024-09-14T05:29:44.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356454+0000", 2024-09-14T05:29:44.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0414999999999999e-05 2024-09-14T05:29:44.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356463+0000", 2024-09-14T05:29:44.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6739999999999992e-06 2024-09-14T05:29:44.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356468+0000", 2024-09-14T05:29:44.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5680000000000001e-06 2024-09-14T05:29:44.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403143+0000", 2024-09-14T05:29:44.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046675051000000002 2024-09-14T05:29:44.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403206+0000", 2024-09-14T05:29:44.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3127999999999996e-05 2024-09-14T05:29:44.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403526+0000", 2024-09-14T05:29:44.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031981400000000002 2024-09-14T05:29:44.080 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.080 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.080 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.080 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3893 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8093696~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.080 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:29:44.080 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.712422444, 2024-09-14T05:29:44.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047516305000000002, 2024-09-14T05:29:44.081 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.081 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.081 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.081 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3893 2024-09-14T05:29:44.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.081 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:29:44.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:29:44.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:29:44.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6899999999999997e-07 2024-09-14T05:29:44.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356491+0000", 2024-09-14T05:29:44.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8109999999999999e-06 2024-09-14T05:29:44.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356498+0000", 2024-09-14T05:29:44.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5320000000000003e-06 2024-09-14T05:29:44.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356502+0000", 2024-09-14T05:29:44.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000002e-06 2024-09-14T05:29:44.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403579+0000", 2024-09-14T05:29:44.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047076706000000003 2024-09-14T05:29:44.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403664+0000", 2024-09-14T05:29:44.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5240000000000004e-05 2024-09-14T05:29:44.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404004+0000", 2024-09-14T05:29:44.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033974500000000002 2024-09-14T05:29:44.085 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.085 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.085 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.085 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3993 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 262144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.085 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:29:44.085 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.712400573, 2024-09-14T05:29:44.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048088476999999998, 2024-09-14T05:29:44.085 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.085 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.085 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.085 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.085 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.085 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.086 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3993 2024-09-14T05:29:44.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.086 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:29:44.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:29:44.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:29:44.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.46e-07 2024-09-14T05:29:44.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356512+0000", 2024-09-14T05:29:44.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.827e-06 2024-09-14T05:29:44.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356518+0000", 2024-09-14T05:29:44.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7780000000000004e-06 2024-09-14T05:29:44.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356522+0000", 2024-09-14T05:29:44.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.865e-06 2024-09-14T05:29:44.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404036+0000", 2024-09-14T05:29:44.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047513725999999999 2024-09-14T05:29:44.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404135+0000", 2024-09-14T05:29:44.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9352999999999996e-05 2024-09-14T05:29:44.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404598+0000", 2024-09-14T05:29:44.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046258200000000001 2024-09-14T05:29:44.089 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.089 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.089 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.089 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3994 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.089 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:29:44.089 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.712358561, 2024-09-14T05:29:44.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048274625000000002, 2024-09-14T05:29:44.089 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.089 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.090 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.090 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3994 2024-09-14T05:29:44.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.090 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:29:44.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:29:44.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:29:44.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3799999999999997e-07 2024-09-14T05:29:44.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356554+0000", 2024-09-14T05:29:44.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2259999999999999e-06 2024-09-14T05:29:44.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356560+0000", 2024-09-14T05:29:44.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-14T05:29:44.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356566+0000", 2024-09-14T05:29:44.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3909999999999997e-06 2024-09-14T05:29:44.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404632+0000", 2024-09-14T05:29:44.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048066154 2024-09-14T05:29:44.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404719+0000", 2024-09-14T05:29:44.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7440000000000003e-05 2024-09-14T05:29:44.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404826+0000", 2024-09-14T05:29:44.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106813 2024-09-14T05:29:44.093 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.093 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.093 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.094 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3995 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.094 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:29:44.094 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.712333422, 2024-09-14T05:29:44.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048828209999999997, 2024-09-14T05:29:44.094 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.094 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.094 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.094 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.094 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.094 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.094 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3995 2024-09-14T05:29:44.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.094 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:29:44.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:29:44.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:29:44.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6599999999999996e-07 2024-09-14T05:29:44.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356579+0000", 2024-09-14T05:29:44.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0719999999999998e-06 2024-09-14T05:29:44.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356585+0000", 2024-09-14T05:29:44.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.959e-06 2024-09-14T05:29:44.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356590+0000", 2024-09-14T05:29:44.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.464e-06 2024-09-14T05:29:44.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404875+0000", 2024-09-14T05:29:44.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048285649999999999 2024-09-14T05:29:44.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404985+0000", 2024-09-14T05:29:44.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109417 2024-09-14T05:29:44.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405405+0000", 2024-09-14T05:29:44.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042008200000000001 2024-09-14T05:29:44.098 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.098 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.098 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.098 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3996 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.098 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:29:44.098 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.712024539, 2024-09-14T05:29:44.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049061595, 2024-09-14T05:29:44.098 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.098 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.098 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.099 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.099 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.099 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.099 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3996 2024-09-14T05:29:44.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.099 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:29:44.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:29:44.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:29:44.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.61e-07 2024-09-14T05:29:44.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356892+0000", 2024-09-14T05:29:44.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9050000000000002e-06 2024-09-14T05:29:44.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356901+0000", 2024-09-14T05:29:44.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4149999999999999e-06 2024-09-14T05:29:44.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356907+0000", 2024-09-14T05:29:44.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0140000000000004e-06 2024-09-14T05:29:44.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405448+0000", 2024-09-14T05:29:44.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048541647 2024-09-14T05:29:44.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405542+0000", 2024-09-14T05:29:44.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3639000000000003e-05 2024-09-14T05:29:44.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405947+0000", 2024-09-14T05:29:44.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040521399999999999 2024-09-14T05:29:44.102 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.102 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.102 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.102 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3997 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2228224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.103 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:29:44.103 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.711810557, 2024-09-14T05:29:44.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049308774999999999, 2024-09-14T05:29:44.103 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.103 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.103 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.103 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.103 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.103 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.103 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3997 2024-09-14T05:29:44.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.103 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:29:44.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:29:44.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:29:44.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6499999999999999e-07 2024-09-14T05:29:44.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357104+0000", 2024-09-14T05:29:44.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4989999999999999e-06 2024-09-14T05:29:44.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357112+0000", 2024-09-14T05:29:44.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4279999999999996e-06 2024-09-14T05:29:44.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357117+0000", 2024-09-14T05:29:44.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6190000000000002e-06 2024-09-14T05:29:44.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405989+0000", 2024-09-14T05:29:44.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048872080999999998 2024-09-14T05:29:44.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406074+0000", 2024-09-14T05:29:44.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4676e-05 2024-09-14T05:29:44.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406408+0000", 2024-09-14T05:29:44.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033480700000000001 2024-09-14T05:29:44.106 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.107 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.107 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.107 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3998 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.107 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:29:44.107 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.711489128, 2024-09-14T05:29:44.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049462314, 2024-09-14T05:29:44.107 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.107 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.107 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.107 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.107 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.107 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.108 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3998 2024-09-14T05:29:44.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.108 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:29:44.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:29:44.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357422+0000", 2024-09-14T05:29:44.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3099999999999996e-07 2024-09-14T05:29:44.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357429+0000", 2024-09-14T05:29:44.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2880000000000004e-06 2024-09-14T05:29:44.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357439+0000", 2024-09-14T05:29:44.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.302e-06 2024-09-14T05:29:44.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357445+0000", 2024-09-14T05:29:44.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4269999999999996e-06 2024-09-14T05:29:44.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406448+0000", 2024-09-14T05:29:44.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049003096000000003 2024-09-14T05:29:44.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406537+0000", 2024-09-14T05:29:44.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9135999999999995e-05 2024-09-14T05:29:44.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406883+0000", 2024-09-14T05:29:44.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000346134 2024-09-14T05:29:44.111 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.111 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.111 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.111 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3999 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.111 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:29:44.111 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.711418771, 2024-09-14T05:29:44.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049797556, 2024-09-14T05:29:44.111 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.111 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.111 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.112 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.112 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.112 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.112 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3999 2024-09-14T05:29:44.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.112 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:29:44.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:29:44.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357492+0000", 2024-09-14T05:29:44.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8299999999999996e-07 2024-09-14T05:29:44.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357495+0000", 2024-09-14T05:29:44.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0450000000000001e-06 2024-09-14T05:29:44.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357503+0000", 2024-09-14T05:29:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9689999999999994e-06 2024-09-14T05:29:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357510+0000", 2024-09-14T05:29:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-14T05:29:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406914+0000", 2024-09-14T05:29:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049404293000000002 2024-09-14T05:29:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406965+0000", 2024-09-14T05:29:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0470999999999998e-05 2024-09-14T05:29:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407289+0000", 2024-09-14T05:29:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032434899999999998 2024-09-14T05:29:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4000 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3702784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:29:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.711072425999999, 2024-09-14T05:29:44.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049936728999999999, 2024-09-14T05:29:44.116 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.116 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.116 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.116 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.116 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.116 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.116 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4000 2024-09-14T05:29:44.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.116 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:29:44.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:29:44.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357839+0000", 2024-09-14T05:29:44.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-14T05:29:44.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357842+0000", 2024-09-14T05:29:44.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4850000000000001e-06 2024-09-14T05:29:44.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357851+0000", 2024-09-14T05:29:44.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1819999999999995e-06 2024-09-14T05:29:44.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357859+0000", 2024-09-14T05:29:44.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8129999999999996e-06 2024-09-14T05:29:44.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407337+0000", 2024-09-14T05:29:44.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049478084999999998 2024-09-14T05:29:44.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407442+0000", 2024-09-14T05:29:44.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000105302 2024-09-14T05:29:44.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407774+0000", 2024-09-14T05:29:44.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033200100000000001 2024-09-14T05:29:44.119 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.119 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.120 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.120 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4001 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.120 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:29:44.120 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.710977071, 2024-09-14T05:29:44.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050304205999999997, 2024-09-14T05:29:44.120 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.120 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.120 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.120 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.120 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.120 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.120 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4001 2024-09-14T05:29:44.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.121 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:29:44.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:29:44.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357934+0000", 2024-09-14T05:29:44.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000004e-07 2024-09-14T05:29:44.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357937+0000", 2024-09-14T05:29:44.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0869999999999998e-06 2024-09-14T05:29:44.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357946+0000", 2024-09-14T05:29:44.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1439999999999992e-06 2024-09-14T05:29:44.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357952+0000", 2024-09-14T05:29:44.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.609e-06 2024-09-14T05:29:44.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407820+0000", 2024-09-14T05:29:44.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049868580000000003 2024-09-14T05:29:44.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407909+0000", 2024-09-14T05:29:44.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8887000000000005e-05 2024-09-14T05:29:44.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408237+0000", 2024-09-14T05:29:44.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032824599999999997 2024-09-14T05:29:44.124 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.124 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.124 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.124 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4002 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.124 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:29:44.124 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.710760476000001, 2024-09-14T05:29:44.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050539529999999999, 2024-09-14T05:29:44.124 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.124 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.125 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.125 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.125 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.125 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.125 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4002 2024-09-14T05:29:44.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.125 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:29:44.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:29:44.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:29:44.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5099999999999999e-07 2024-09-14T05:29:44.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358154+0000", 2024-09-14T05:29:44.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6799999999999999e-06 2024-09-14T05:29:44.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358165+0000", 2024-09-14T05:29:44.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0899e-05 2024-09-14T05:29:44.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358172+0000", 2024-09-14T05:29:44.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8689999999999998e-06 2024-09-14T05:29:44.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408281+0000", 2024-09-14T05:29:44.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050108699 2024-09-14T05:29:44.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408369+0000", 2024-09-14T05:29:44.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8542999999999996e-05 2024-09-14T05:29:44.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408689+0000", 2024-09-14T05:29:44.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032008900000000002 2024-09-14T05:29:44.128 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.128 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.128 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.129 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4003 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5177344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.129 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:29:44.129 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.710720641, 2024-09-14T05:29:44.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050868759, 2024-09-14T05:29:44.129 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.129 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.129 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.129 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.129 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.129 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.129 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4003 2024-09-14T05:29:44.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.130 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:29:44.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:29:44.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:29:44.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-07 2024-09-14T05:29:44.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358193+0000", 2024-09-14T05:29:44.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.588e-06 2024-09-14T05:29:44.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358204+0000", 2024-09-14T05:29:44.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1525999999999999e-05 2024-09-14T05:29:44.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358213+0000", 2024-09-14T05:29:44.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5010000000000004e-06 2024-09-14T05:29:44.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408719+0000", 2024-09-14T05:29:44.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050506708999999997 2024-09-14T05:29:44.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408766+0000", 2024-09-14T05:29:44.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6579999999999998e-05 2024-09-14T05:29:44.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409058+0000", 2024-09-14T05:29:44.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000292365 2024-09-14T05:29:44.133 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.133 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.133 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.133 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4004 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5668864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.133 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:29:44.133 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.710297797999999, 2024-09-14T05:29:44.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050839848, 2024-09-14T05:29:44.133 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.134 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.134 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.134 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.134 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.134 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.134 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4004 2024-09-14T05:29:44.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.134 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:29:44.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:29:44.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358613+0000", 2024-09-14T05:29:44.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:29:44.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358618+0000", 2024-09-14T05:29:44.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6009999999999997e-06 2024-09-14T05:29:44.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358630+0000", 2024-09-14T05:29:44.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2122e-05 2024-09-14T05:29:44.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358638+0000", 2024-09-14T05:29:44.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3969999999999994e-06 2024-09-14T05:29:44.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409088+0000", 2024-09-14T05:29:44.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050449565000000002 2024-09-14T05:29:44.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409153+0000", 2024-09-14T05:29:44.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4728999999999996e-05 2024-09-14T05:29:44.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409452+0000", 2024-09-14T05:29:44.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029958900000000001 2024-09-14T05:29:44.137 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.137 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.137 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.137 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4005 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6160384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.137 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:29:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.710080864, 2024-09-14T05:29:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051054752000000002, 2024-09-14T05:29:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4005 2024-09-14T05:29:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:29:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:29:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358830+0000", 2024-09-14T05:29:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-14T05:29:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358833+0000", 2024-09-14T05:29:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9529999999999999e-06 2024-09-14T05:29:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358843+0000", 2024-09-14T05:29:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8069999999999994e-06 2024-09-14T05:29:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358849+0000", 2024-09-14T05:29:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1040000000000003e-06 2024-09-14T05:29:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409487+0000", 2024-09-14T05:29:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050638626999999999 2024-09-14T05:29:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409535+0000", 2024-09-14T05:29:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7525000000000003e-05 2024-09-14T05:29:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409884+0000", 2024-09-14T05:29:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034914800000000003 2024-09-14T05:29:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4006 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:29:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.710043752000001, 2024-09-14T05:29:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051425323000000002, 2024-09-14T05:29:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4006 2024-09-14T05:29:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:29:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:29:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358867+0000", 2024-09-14T05:29:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2900000000000004e-07 2024-09-14T05:29:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358869+0000", 2024-09-14T05:29:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4210000000000002e-06 2024-09-14T05:29:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358876+0000", 2024-09-14T05:29:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4980000000000003e-06 2024-09-14T05:29:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358881+0000", 2024-09-14T05:29:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3090000000000003e-06 2024-09-14T05:29:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409915+0000", 2024-09-14T05:29:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051033348999999999 2024-09-14T05:29:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409964+0000", 2024-09-14T05:29:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9858000000000003e-05 2024-09-14T05:29:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410292+0000", 2024-09-14T05:29:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032735899999999999 2024-09-14T05:29:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4007 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:29:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.710021159, 2024-09-14T05:29:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05184076, 2024-09-14T05:29:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4007 2024-09-14T05:29:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:29:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:29:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:29:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:29:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358891+0000", 2024-09-14T05:29:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9819999999999998e-06 2024-09-14T05:29:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358897+0000", 2024-09-14T05:29:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9390000000000002e-06 2024-09-14T05:29:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358902+0000", 2024-09-14T05:29:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6770000000000003e-06 2024-09-14T05:29:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410335+0000", 2024-09-14T05:29:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051432991999999997 2024-09-14T05:29:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410416+0000", 2024-09-14T05:29:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1316999999999994e-05 2024-09-14T05:29:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410730+0000", 2024-09-14T05:29:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000313419 2024-09-14T05:29:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4008 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7634944~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:29:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.709996798000001, 2024-09-14T05:29:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052208082000000003, 2024-09-14T05:29:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4008 2024-09-14T05:29:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:29:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:29:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358914+0000", 2024-09-14T05:29:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6100000000000001e-07 2024-09-14T05:29:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358916+0000", 2024-09-14T05:29:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.902e-06 2024-09-14T05:29:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358923+0000", 2024-09-14T05:29:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4780000000000002e-06 2024-09-14T05:29:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358929+0000", 2024-09-14T05:29:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0970000000000002e-06 2024-09-14T05:29:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410761+0000", 2024-09-14T05:29:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051831436000000002 2024-09-14T05:29:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410829+0000", 2024-09-14T05:29:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7874999999999996e-05 2024-09-14T05:29:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411121+0000", 2024-09-14T05:29:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029283299999999999 2024-09-14T05:29:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4009 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8126464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:29:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.709849573, 2024-09-14T05:29:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052518169000000003, 2024-09-14T05:29:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4009 2024-09-14T05:29:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:29:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:29:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:29:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0200000000000002e-07 2024-09-14T05:29:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359066+0000", 2024-09-14T05:29:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7899999999999999e-06 2024-09-14T05:29:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359074+0000", 2024-09-14T05:29:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2579999999999995e-06 2024-09-14T05:29:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359082+0000", 2024-09-14T05:29:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.464e-06 2024-09-14T05:29:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411160+0000", 2024-09-14T05:29:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052078078 2024-09-14T05:29:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411244+0000", 2024-09-14T05:29:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4041000000000004e-05 2024-09-14T05:29:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411579+0000", 2024-09-14T05:29:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033493599999999998 2024-09-14T05:29:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:44.159 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:44.160 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T05:29:44.160 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T05:29:44.160 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T05:29:44.160 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T05:29:44.161 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:44.095+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:44.162 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-14T05:29:44.162 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:29:44.166 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:44.165+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:44.266 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:44.266+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:44.305 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-14T05:29:44.313 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:44.313 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:44.313 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:29:44.313 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:44.316 INFO:teuthology.orchestra.run.smithi071.stdout:1532 write 0x4d03c17 thru 0x4d10f4c (0xd336 bytes) 2024-09-14T05:29:44.333 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:29:44.378 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:44.378+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:44.380 INFO:teuthology.orchestra.run.smithi071.stdout:1536 read 0x53ab45e thru 0x53bb3a9 (0xff4c bytes) 2024-09-14T05:29:44.384 INFO:teuthology.orchestra.run.smithi071.stdout:1537 read 0xa451e48 thru 0xa45e5b8 (0xc771 bytes) 2024-09-14T05:29:44.384 INFO:teuthology.orchestra.run.smithi071.stdout:1538 writesame 0x20894e6 thru 0x208d5ee data_size 0x1(0x4109 bytes) 2024-09-14T05:29:44.442 INFO:teuthology.orchestra.run.smithi071.stdout:1539 read 0xa7fd4ef thru 0xa8041fe (0x6d10 bytes) 2024-09-14T05:29:44.443 INFO:teuthology.orchestra.run.smithi071.stdout:1540 write 0x46e0200 thru 0x46ef0d1 (0xeed2 bytes) 2024-09-14T05:29:44.468 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:44.467+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:44.504 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:44.505 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:44.505 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:44.505 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:44.505 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:44.511 INFO:teuthology.orchestra.run.smithi071.stdout:1542 punch from 0x7bcd14e to 0x7bdc3a5, (0xf257 bytes) 2024-09-14T05:29:44.511 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:44.513 INFO:teuthology.orchestra.run.smithi071.stdout:1544 trunc from 0xb7c764d to 0xd9e54a7 2024-09-14T05:29:44.516 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:29:44.519 INFO:teuthology.orchestra.run.smithi071.stdout:1546 write 0x2bef7b2 thru 0x2bf4d5f (0x55ae bytes) 2024-09-14T05:29:44.568 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:44.567+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:44.588 INFO:teuthology.orchestra.run.smithi071.stdout:1547 read 0xd0281ee thru 0xd029155 (0xf68 bytes) 2024-09-14T05:29:44.589 INFO:teuthology.orchestra.run.smithi071.stdout:1548 write 0xc9c11c5 thru 0xc9cb9cf (0xa80b bytes) 2024-09-14T05:29:44.606 INFO:teuthology.orchestra.run.smithi071.stdout:1549 read 0x16bf9f4 thru 0x16c9d02 (0xa30f bytes) 2024-09-14T05:29:44.610 INFO:teuthology.orchestra.run.smithi071.stdout:1550 write 0x20449d1 thru 0x2045907 (0xf37 bytes) 2024-09-14T05:29:44.637 INFO:teuthology.orchestra.run.smithi071.stdout:1551 punch from 0x852e83e to 0x8532740, (0x3f02 bytes) 2024-09-14T05:29:44.637 INFO:teuthology.orchestra.run.smithi071.stdout:1553 write 0xaeef21d thru 0xaefea33 (0xf817 bytes) 2024-09-14T05:29:44.649 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:29:44.649 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:44.649 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 44, 2024-09-14T05:29:44.649 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "1127be44-5f14-4e7d-ad23-f608afe154f5", 2024-09-14T05:29:44.649 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2024-09-14T05:28:54.203072+0000", 2024-09-14T05:29:44.649 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-14T05:29:44.110881+0000", 2024-09-14T05:29:44.649 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2024-09-14T05:29:06.755531+0000", 2024-09-14T05:29:44.650 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2024-09-14T05:28:56.535048+0000", 2024-09-14T05:29:44.650 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T05:29:44.650 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2024-09-14T05:29:44.650 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2024-09-14T05:29:44.650 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2024-09-14T05:29:44.650 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2024-09-14T05:29:44.650 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2024-09-14T05:29:44.650 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2024-09-14T05:29:44.650 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:29:44.650 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 6, 2024-09-14T05:29:44.650 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T05:29:44.650 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T05:29:44.650 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T05:29:44.650 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2024-09-14T05:29:44.651 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 3, 2024-09-14T05:29:44.651 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2024-09-14T05:29:44.651 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2024-09-14T05:29:44.651 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2024-09-14T05:29:44.651 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "squid", 2024-09-14T05:29:44.651 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2024-09-14T05:29:44.651 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2024-09-14T05:29:44.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.651 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2024-09-14T05:29:44.651 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2024-09-14T05:29:44.651 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-14T05:29:06.452744+0000", 2024-09-14T05:29:44.651 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-14T05:29:44.651 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2024-09-14T05:29:44.652 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-14T05:29:44.652 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-14T05:29:44.652 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-14T05:29:44.652 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-14T05:29:44.652 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-14T05:29:44.652 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-14T05:29:44.652 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T05:29:44.652 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T05:29:44.652 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-14T05:29:44.652 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-14T05:29:44.652 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-14T05:29:44.652 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2024-09-14T05:29:44.652 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2024-09-14T05:29:44.652 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2024-09-14T05:29:44.653 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2024-09-14T05:29:44.653 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2024-09-14T05:29:44.653 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-14T05:29:44.653 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-14T05:29:44.653 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-14T05:29:44.653 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-14T05:29:44.653 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-14T05:29:44.653 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-14T05:29:44.653 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-14T05:29:44.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.653 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "18", 2024-09-14T05:29:44.653 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-14T05:29:44.653 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T05:29:44.653 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T05:29:44.654 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-14T05:29:44.654 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-14T05:29:44.654 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2024-09-14T05:29:44.654 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2024-09-14T05:29:44.654 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-14T05:29:44.654 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-14T05:29:44.654 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-14T05:29:44.654 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-14T05:29:44.654 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-14T05:29:44.654 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-14T05:29:44.654 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-14T05:29:44.654 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-14T05:29:44.654 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-14T05:29:44.654 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-14T05:29:44.655 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-14T05:29:44.655 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T05:29:44.655 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T05:29:44.655 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T05:29:44.655 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-14T05:29:44.655 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-14T05:29:44.655 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-14T05:29:44.655 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-14T05:29:44.655 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-14T05:29:44.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.655 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-14T05:29:44.655 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-14T05:29:44.655 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-14T05:29:44.655 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-14T05:29:44.655 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-14T05:29:44.655 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T05:29:44.656 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-14T05:29:44.656 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-14T05:29:44.656 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-14T05:29:44.656 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-14T05:29:44.656 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-14T05:29:44.656 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2024-09-14T05:29:44.656 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2024-09-14T05:29:44.656 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2024-09-14T05:29:44.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.656 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-14T05:29:44.656 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2024-09-14T05:29:44.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.657 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-14T05:29:44.657 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-14T05:29:44.657 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2024-09-14T05:29:44.657 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2024-09-14T05:29:44.657 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2024-09-14T05:29:44.657 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2024-09-14T05:29:44.657 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2024-09-14T05:29:44.657 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.87708091735839844, 2024-09-14T05:29:44.657 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T05:29:44.657 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T05:29:44.657 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.657 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.657 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 2, 2024-09-14T05:29:44.658 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "rbd", 2024-09-14T05:29:44.658 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-14T05:29:12.380843+0000", 2024-09-14T05:29:44.658 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 8193, 2024-09-14T05:29:44.658 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T05:29:44.658 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-14T05:29:44.658 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-14T05:29:44.658 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-14T05:29:44.658 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-14T05:29:44.658 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-14T05:29:44.658 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-14T05:29:44.658 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T05:29:44.658 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T05:29:44.658 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-14T05:29:44.658 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-14T05:29:44.659 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-14T05:29:44.659 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 8, 2024-09-14T05:29:44.659 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 8, 2024-09-14T05:29:44.659 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 8, 2024-09-14T05:29:44.659 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 8, 2024-09-14T05:29:44.659 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 8, 2024-09-14T05:29:44.659 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-14T05:29:44.659 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-14T05:29:44.659 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-14T05:29:44.659 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-14T05:29:44.659 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-14T05:29:44.659 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-14T05:29:44.659 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-14T05:29:44.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.660 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "22", 2024-09-14T05:29:44.660 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-14T05:29:44.660 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T05:29:44.660 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T05:29:44.660 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-14T05:29:44.660 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-14T05:29:44.660 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 2, 2024-09-14T05:29:44.660 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 22, 2024-09-14T05:29:44.660 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-14T05:29:44.660 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-14T05:29:44.660 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-14T05:29:44.660 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-14T05:29:44.660 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-14T05:29:44.661 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-14T05:29:44.661 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-14T05:29:44.661 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-14T05:29:44.661 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-14T05:29:44.661 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-14T05:29:44.661 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-14T05:29:44.661 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T05:29:44.661 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T05:29:44.661 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T05:29:44.661 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-14T05:29:44.661 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-14T05:29:44.661 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-14T05:29:44.661 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-14T05:29:44.661 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-14T05:29:44.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.662 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-14T05:29:44.662 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-14T05:29:44.662 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-14T05:29:44.662 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-14T05:29:44.662 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-14T05:29:44.662 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T05:29:44.662 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-14T05:29:44.662 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-14T05:29:44.662 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-14T05:29:44.662 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-14T05:29:44.662 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-14T05:29:44.662 INFO:teuthology.orchestra.run.smithi071.stdout: "options": {}, 2024-09-14T05:29:44.662 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-14T05:29:44.662 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd": {} 2024-09-14T05:29:44.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.663 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-14T05:29:44.663 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-14T05:29:44.663 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 4.9800000190734863, 2024-09-14T05:29:44.663 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 4.9800000190734863, 2024-09-14T05:29:44.663 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T05:29:44.663 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 4.380000114440918, 2024-09-14T05:29:44.663 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 4.380000114440918, 2024-09-14T05:29:44.663 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.87708091735839844, 2024-09-14T05:29:44.663 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T05:29:44.663 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T05:29:44.663 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.663 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 3, 2024-09-14T05:29:44.664 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "pool_client.0", 2024-09-14T05:29:44.664 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-14T05:29:22.335419+0000", 2024-09-14T05:29:44.664 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 8193, 2024-09-14T05:29:44.664 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T05:29:44.664 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-14T05:29:44.664 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-14T05:29:44.664 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-14T05:29:44.664 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-14T05:29:44.664 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-14T05:29:44.664 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-14T05:29:44.664 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T05:29:44.664 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T05:29:44.664 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-14T05:29:44.664 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-14T05:29:44.664 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-14T05:29:44.665 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2024-09-14T05:29:44.665 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2024-09-14T05:29:44.665 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2024-09-14T05:29:44.665 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2024-09-14T05:29:44.665 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2024-09-14T05:29:44.665 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-14T05:29:44.665 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-14T05:29:44.665 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-14T05:29:44.665 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-14T05:29:44.665 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-14T05:29:44.666 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-14T05:29:44.666 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-14T05:29:44.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.666 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "44", 2024-09-14T05:29:44.666 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-14T05:29:44.666 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T05:29:44.666 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T05:29:44.666 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-14T05:29:44.666 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-14T05:29:44.666 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 16, 2024-09-14T05:29:44.666 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 44, 2024-09-14T05:29:44.666 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-14T05:29:44.666 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-14T05:29:44.666 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-14T05:29:44.666 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-14T05:29:44.667 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-14T05:29:44.667 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-14T05:29:44.667 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-14T05:29:44.667 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-14T05:29:44.667 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-14T05:29:44.667 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-14T05:29:44.667 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-14T05:29:44.667 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T05:29:44.667 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T05:29:44.667 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T05:29:44.667 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-14T05:29:44.667 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-14T05:29:44.668 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-14T05:29:44.668 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-14T05:29:44.668 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-14T05:29:44.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.668 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-14T05:29:44.668 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-14T05:29:44.668 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-14T05:29:44.668 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-14T05:29:44.668 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-14T05:29:44.668 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T05:29:44.668 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-14T05:29:44.668 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-14T05:29:44.668 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-14T05:29:44.668 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-14T05:29:44.668 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-14T05:29:44.669 INFO:teuthology.orchestra.run.smithi071.stdout: "options": {}, 2024-09-14T05:29:44.669 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-14T05:29:44.669 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd": {} 2024-09-14T05:29:44.669 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.669 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-14T05:29:44.669 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-14T05:29:44.669 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2024-09-14T05:29:44.669 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2024-09-14T05:29:44.669 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2024-09-14T05:29:44.669 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2024-09-14T05:29:44.669 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2024-09-14T05:29:44.669 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.87708091735839844, 2024-09-14T05:29:44.669 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T05:29:44.670 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T05:29:44.670 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.670 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.670 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:29:44.670 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2024-09-14T05:29:44.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.670 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2024-09-14T05:29:44.670 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "e7c7817f-4a65-4f66-9af5-d31d0a92cf30", 2024-09-14T05:29:44.670 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:44.670 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:44.671 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:44.671 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:44.671 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:44.671 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:44.671 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 16, 2024-09-14T05:29:44.671 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:29:44.671 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:44.671 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:44.671 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:44.671 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.671 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.671 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.671 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2024-09-14T05:29:44.671 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:44.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.672 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.672 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2024-09-14T05:29:44.672 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:44.672 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.672 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.672 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:44.672 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.672 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.672 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2024-09-14T05:29:44.672 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:44.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.673 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.673 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.673 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2024-09-14T05:29:44.673 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:44.673 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.673 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.673 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:44.673 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.673 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.673 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.673 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2024-09-14T05:29:44.673 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:44.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.674 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.674 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2024-09-14T05:29:44.674 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:44.674 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.674 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.674 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:44.674 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.674 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.674 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2024-09-14T05:29:44.674 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:44.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.675 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.675 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2024-09-14T05:29:44.675 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:44.675 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.676 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.676 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6819/2926109885", 2024-09-14T05:29:44.676 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6821/2926109885", 2024-09-14T05:29:44.676 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6825/2926109885", 2024-09-14T05:29:44.676 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6823/2926109885", 2024-09-14T05:29:44.676 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:44.676 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:44.676 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:44.676 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.676 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.676 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2024-09-14T05:29:44.677 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "e184604e-b973-4b7a-bba2-6f471753a0fd", 2024-09-14T05:29:44.677 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:44.677 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:44.677 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:44.677 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:44.677 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:44.677 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:44.677 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:44.677 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:29:44.677 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:44.677 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:44.677 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:44.677 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.678 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.678 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2024-09-14T05:29:44.678 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:44.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.678 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.678 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.678 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2024-09-14T05:29:44.678 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:44.678 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.678 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.679 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:44.679 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.679 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.679 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2024-09-14T05:29:44.679 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:44.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.679 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.679 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2024-09-14T05:29:44.679 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:44.679 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.680 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.680 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:44.680 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.680 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.680 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.680 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6832", 2024-09-14T05:29:44.680 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:44.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.680 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.680 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.680 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6833", 2024-09-14T05:29:44.680 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:44.681 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.681 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.681 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:44.681 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.681 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.681 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2024-09-14T05:29:44.681 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:44.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.681 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.682 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2024-09-14T05:29:44.682 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:44.682 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.682 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.682 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6827/345649648", 2024-09-14T05:29:44.682 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6829/345649648", 2024-09-14T05:29:44.682 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6833/345649648", 2024-09-14T05:29:44.683 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6831/345649648", 2024-09-14T05:29:44.683 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:44.683 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:44.683 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:44.683 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.683 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.683 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2024-09-14T05:29:44.683 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "d8b70f09-5afb-4296-b8c9-719c172e0e70", 2024-09-14T05:29:44.683 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:44.683 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:44.683 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:44.683 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:44.684 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:44.684 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:44.684 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:44.684 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 39, 2024-09-14T05:29:44.684 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:44.684 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:44.684 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:44.684 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.684 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.684 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.684 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2024-09-14T05:29:44.685 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:44.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.685 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.685 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.685 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2024-09-14T05:29:44.685 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:44.685 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.685 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.685 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:44.685 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.685 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.685 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.686 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2024-09-14T05:29:44.686 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:44.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.686 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.686 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.686 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2024-09-14T05:29:44.686 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:44.686 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.686 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.686 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:44.686 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.686 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.687 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.687 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2024-09-14T05:29:44.687 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:44.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.687 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.687 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.687 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2024-09-14T05:29:44.687 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:44.687 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.687 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.687 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:44.687 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.687 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.687 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.688 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2024-09-14T05:29:44.688 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:44.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.688 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.688 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2024-09-14T05:29:44.688 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:44.688 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.688 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.688 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/2721860691", 2024-09-14T05:29:44.688 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/2721860691", 2024-09-14T05:29:44.688 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/2721860691", 2024-09-14T05:29:44.689 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/2721860691", 2024-09-14T05:29:44.689 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:44.689 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:44.689 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:44.689 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.689 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2024-09-14T05:29:44.689 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c", 2024-09-14T05:29:44.689 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:44.689 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:44.689 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:44.690 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:44.690 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:44.690 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:44.690 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:44.690 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 39, 2024-09-14T05:29:44.690 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:44.690 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:44.690 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:44.690 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.690 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.690 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.690 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2024-09-14T05:29:44.690 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:44.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.691 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.691 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.691 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2024-09-14T05:29:44.691 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:44.691 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.691 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.691 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.691 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:44.691 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.691 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.691 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.691 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2024-09-14T05:29:44.691 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:44.691 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.692 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.692 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.692 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2024-09-14T05:29:44.692 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:44.692 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.692 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.692 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:44.692 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.692 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.692 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.693 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2024-09-14T05:29:44.693 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:44.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.693 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.693 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.693 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2024-09-14T05:29:44.693 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:44.693 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.693 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.694 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:44.694 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.694 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.694 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.694 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2024-09-14T05:29:44.694 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:44.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.694 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.694 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.694 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2024-09-14T05:29:44.694 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:44.694 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.695 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.695 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/494607463", 2024-09-14T05:29:44.695 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6811/494607463", 2024-09-14T05:29:44.695 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6815/494607463", 2024-09-14T05:29:44.695 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6813/494607463", 2024-09-14T05:29:44.695 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:44.695 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:44.695 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:44.695 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.695 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.695 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2024-09-14T05:29:44.695 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "515cb7d6-16eb-4aff-973a-7f7ed49325ba", 2024-09-14T05:29:44.695 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:44.696 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:44.696 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:44.696 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:44.696 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:44.696 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:44.696 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:44.696 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:29:44.696 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:44.696 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:44.696 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:44.696 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.696 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.696 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.697 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6816", 2024-09-14T05:29:44.697 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:44.697 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.697 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.697 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.697 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6817", 2024-09-14T05:29:44.697 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:44.697 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.697 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.697 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.697 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:44.697 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.697 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.697 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.697 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6818", 2024-09-14T05:29:44.698 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:44.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.698 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.698 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.698 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6819", 2024-09-14T05:29:44.698 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:44.698 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.698 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.698 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:44.698 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.698 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.698 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.699 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6822", 2024-09-14T05:29:44.699 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:44.699 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.699 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.699 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.699 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6823", 2024-09-14T05:29:44.699 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:44.699 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.699 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.699 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.699 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:44.699 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.699 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.700 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.700 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6820", 2024-09-14T05:29:44.700 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:44.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.700 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.700 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6821", 2024-09-14T05:29:44.700 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:44.700 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.700 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.700 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6817/2240133417", 2024-09-14T05:29:44.700 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6819/2240133417", 2024-09-14T05:29:44.701 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6823/2240133417", 2024-09-14T05:29:44.701 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6821/2240133417", 2024-09-14T05:29:44.701 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:44.701 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:44.701 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:44.701 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.701 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.701 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2024-09-14T05:29:44.701 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "c434b53a-04be-45b9-b89a-193654133950", 2024-09-14T05:29:44.701 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:44.701 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:44.701 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:44.702 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:44.702 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:44.702 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:44.702 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:44.702 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 39, 2024-09-14T05:29:44.702 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:44.702 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:44.702 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:44.702 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.702 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.702 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6808", 2024-09-14T05:29:44.702 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:44.703 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.703 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.703 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.703 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6809", 2024-09-14T05:29:44.703 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:44.703 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.703 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.703 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.703 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:44.703 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.703 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.703 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.703 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6810", 2024-09-14T05:29:44.703 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:44.703 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.704 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.704 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6811", 2024-09-14T05:29:44.704 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:44.704 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.704 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.704 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:44.704 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.704 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.704 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6814", 2024-09-14T05:29:44.704 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:44.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.705 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.705 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6815", 2024-09-14T05:29:44.705 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:44.705 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.705 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.705 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:44.705 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.705 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.706 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6812", 2024-09-14T05:29:44.706 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:44.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.706 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.706 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6813", 2024-09-14T05:29:44.706 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:44.706 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.706 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.706 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6809/2393825223", 2024-09-14T05:29:44.706 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6811/2393825223", 2024-09-14T05:29:44.706 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6815/2393825223", 2024-09-14T05:29:44.706 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6813/2393825223", 2024-09-14T05:29:44.706 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:44.707 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:44.707 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:44.707 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.707 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2024-09-14T05:29:44.707 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "3d2f8405-9286-4b40-8acd-036f67373ac6", 2024-09-14T05:29:44.707 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:44.707 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:44.707 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:44.707 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 0.0166473388671875, 2024-09-14T05:29:44.708 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:44.708 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:44.708 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:44.708 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:29:44.708 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:44.708 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:44.708 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:44.708 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.708 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.708 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6800", 2024-09-14T05:29:44.708 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:44.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.708 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.709 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6801", 2024-09-14T05:29:44.709 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:44.709 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.709 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.709 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:44.709 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.709 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.709 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6802", 2024-09-14T05:29:44.709 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:44.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.710 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.710 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6803", 2024-09-14T05:29:44.710 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:44.710 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.710 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.710 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:44.710 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.710 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.710 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6806", 2024-09-14T05:29:44.710 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:44.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.711 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.711 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6807", 2024-09-14T05:29:44.711 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:44.711 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.711 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.711 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:44.711 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.711 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.711 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6804", 2024-09-14T05:29:44.711 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:44.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.712 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.712 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6805", 2024-09-14T05:29:44.712 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:44.712 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.712 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.712 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6801/3235729544", 2024-09-14T05:29:44.712 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6803/3235729544", 2024-09-14T05:29:44.712 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6807/3235729544", 2024-09-14T05:29:44.712 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6805/3235729544", 2024-09-14T05:29:44.712 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:44.712 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:44.713 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:44.713 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.713 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2024-09-14T05:29:44.713 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "6724f080-4dc3-465a-939f-b023985bf998", 2024-09-14T05:29:44.713 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:44.713 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:44.713 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:44.713 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:44.713 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:44.713 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:44.714 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:44.714 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:29:44.714 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:44.714 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:44.714 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:44.714 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.714 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.714 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6824", 2024-09-14T05:29:44.714 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:44.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.715 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.715 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6825", 2024-09-14T05:29:44.715 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:44.715 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.715 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.715 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:44.715 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.715 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.715 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.715 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6826", 2024-09-14T05:29:44.715 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:44.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.716 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.716 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.716 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6827", 2024-09-14T05:29:44.716 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:44.716 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.716 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.716 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:44.716 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.716 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.716 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.716 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6830", 2024-09-14T05:29:44.716 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:44.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.717 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.717 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6831", 2024-09-14T05:29:44.717 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:44.717 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.717 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.717 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:44.717 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:44.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.717 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:44.717 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6828", 2024-09-14T05:29:44.718 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:44.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.718 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:44.718 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6829", 2024-09-14T05:29:44.718 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:44.718 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.718 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.718 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6825/2381001355", 2024-09-14T05:29:44.718 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6827/2381001355", 2024-09-14T05:29:44.718 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6831/2381001355", 2024-09-14T05:29:44.718 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6829/2381001355", 2024-09-14T05:29:44.718 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:44.719 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:44.719 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:44.719 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.719 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.719 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:29:44.719 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2024-09-14T05:29:44.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.719 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2024-09-14T05:29:44.719 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:44.719 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:44.719 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:44.719 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:44.719 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:44.720 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.548803+0000", 2024-09-14T05:29:44.720 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:44.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.720 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2024-09-14T05:29:44.720 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:44.720 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:44.720 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:44.720 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:44.720 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:44.720 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.075023+0000", 2024-09-14T05:29:44.720 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:44.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.721 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2024-09-14T05:29:44.721 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:44.721 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:44.721 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:44.721 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:44.721 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:44.721 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.117036+0000", 2024-09-14T05:29:44.721 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:44.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.721 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2024-09-14T05:29:44.721 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:44.722 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:44.722 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:44.722 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:44.722 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:44.722 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.332113+0000", 2024-09-14T05:29:44.722 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:44.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.722 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2024-09-14T05:29:44.722 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:44.722 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:44.722 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:44.722 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:44.723 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:44.723 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.330371+0000", 2024-09-14T05:29:44.723 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:44.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.723 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2024-09-14T05:29:44.723 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:44.723 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:44.723 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:44.723 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:44.723 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:44.723 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:02.753946+0000", 2024-09-14T05:29:44.723 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:44.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.724 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2024-09-14T05:29:44.724 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:44.724 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:44.724 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:44.724 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:44.724 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:44.724 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:02.071186+0000", 2024-09-14T05:29:44.724 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:44.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.725 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2024-09-14T05:29:44.725 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:44.725 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:44.725 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:44.725 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:44.725 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:44.725 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.378781+0000", 2024-09-14T05:29:44.725 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:44.725 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.725 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:29:44.725 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2024-09-14T05:29:44.725 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2024-09-14T05:29:44.726 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2024-09-14T05:29:44.726 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2024-09-14T05:29:44.726 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2024-09-14T05:29:44.726 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2024-09-14T05:29:44.726 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2024-09-14T05:29:44.726 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2024-09-14T05:29:44.726 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2024-09-14T05:29:44.726 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2024-09-14T05:29:44.726 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2024-09-14T05:29:44.726 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2024-09-14T05:29:44.726 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2024-09-14T05:29:44.726 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2024-09-14T05:29:44.726 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.727 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2024-09-14T05:29:44.727 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2024-09-14T05:29:44.727 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2024-09-14T05:29:44.727 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2024-09-14T05:29:44.727 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2024-09-14T05:29:44.727 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2024-09-14T05:29:44.727 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2024-09-14T05:29:44.727 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2024-09-14T05:29:44.727 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2024-09-14T05:29:44.727 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2024-09-14T05:29:44.727 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2024-09-14T05:29:44.727 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.727 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:44.730 INFO:tasks.thrashosds.thrasher:j is {'epoch': 44, 'fsid': '1127be44-5f14-4e7d-ad23-f608afe154f5', 'created': '2024-09-14T05:28:54.203072+0000', 'modified': '2024-09-14T05:29:44.110881+0000', 'last_up_change': '2024-09-14T05:29:06.755531+0000', 'last_in_change': '2024-09-14T05:28:56.535048+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T05:29:06.452744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8770809173583984, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T05:29:12.380843+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.8770809173583984, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T05:29:22.335419+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 16, 'snap_epoch': 44, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8770809173583984, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'e7c7817f-4a65-4f66-9af5-d31d0a92cf30', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6818', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6819', 'nonce': 2926109885}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6820', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6821', 'nonce': 2926109885}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6824', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6825', 'nonce': 2926109885}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6822', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6823', 'nonce': 2926109885}]}, 'public_addr': '172.21.15.71:6819/2926109885', 'cluster_addr': '172.21.15.71:6821/2926109885', 'heartbeat_back_addr': '172.21.15.71:6825/2926109885', 'heartbeat_front_addr': '172.21.15.71:6823/2926109885', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e184604e-b973-4b7a-bba2-6f471753a0fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6826', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6827', 'nonce': 345649648}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6828', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6829', 'nonce': 345649648}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6832', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6833', 'nonce': 345649648}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6830', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6831', 'nonce': 345649648}]}, 'public_addr': '172.21.15.71:6827/345649648', 'cluster_addr': '172.21.15.71:6829/345649648', 'heartbeat_back_addr': '172.21.15.71:6833/345649648', 'heartbeat_front_addr': '172.21.15.71:6831/345649648', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd8b70f09-5afb-4296-b8c9-719c172e0e70', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6800', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6801', 'nonce': 2721860691}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6802', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6803', 'nonce': 2721860691}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6806', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6807', 'nonce': 2721860691}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6804', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6805', 'nonce': 2721860691}]}, 'public_addr': '172.21.15.71:6801/2721860691', 'cluster_addr': '172.21.15.71:6803/2721860691', 'heartbeat_back_addr': '172.21.15.71:6807/2721860691', 'heartbeat_front_addr': '172.21.15.71:6805/2721860691', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6808', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6809', 'nonce': 494607463}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6810', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6811', 'nonce': 494607463}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6814', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6815', 'nonce': 494607463}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6812', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6813', 'nonce': 494607463}]}, 'public_addr': '172.21.15.71:6809/494607463', 'cluster_addr': '172.21.15.71:6811/494607463', 'heartbeat_back_addr': '172.21.15.71:6815/494607463', 'heartbeat_front_addr': '172.21.15.71:6813/494607463', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '515cb7d6-16eb-4aff-973a-7f7ed49325ba', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6816', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6817', 'nonce': 2240133417}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6818', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6819', 'nonce': 2240133417}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6822', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6823', 'nonce': 2240133417}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6820', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6821', 'nonce': 2240133417}]}, 'public_addr': '172.21.15.150:6817/2240133417', 'cluster_addr': '172.21.15.150:6819/2240133417', 'heartbeat_back_addr': '172.21.15.150:6823/2240133417', 'heartbeat_front_addr': '172.21.15.150:6821/2240133417', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'c434b53a-04be-45b9-b89a-193654133950', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6808', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6809', 'nonce': 2393825223}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6810', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6811', 'nonce': 2393825223}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6814', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6815', 'nonce': 2393825223}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6812', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6813', 'nonce': 2393825223}]}, 'public_addr': '172.21.15.150:6809/2393825223', 'cluster_addr': '172.21.15.150:6811/2393825223', 'heartbeat_back_addr': '172.21.15.150:6815/2393825223', 'heartbeat_front_addr': '172.21.15.150:6813/2393825223', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '3d2f8405-9286-4b40-8acd-036f67373ac6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0166473388671875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6800', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6801', 'nonce': 3235729544}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6802', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6803', 'nonce': 3235729544}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6806', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6807', 'nonce': 3235729544}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6804', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6805', 'nonce': 3235729544}]}, 'public_addr': '172.21.15.150:6801/3235729544', 'cluster_addr': '172.21.15.150:6803/3235729544', 'heartbeat_back_addr': '172.21.15.150:6807/3235729544', 'heartbeat_front_addr': '172.21.15.150:6805/3235729544', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '6724f080-4dc3-465a-939f-b023985bf998', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6824', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6825', 'nonce': 2381001355}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6826', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6827', 'nonce': 2381001355}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6830', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6831', 'nonce': 2381001355}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6828', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6829', 'nonce': 2381001355}]}, 'public_addr': '172.21.15.150:6825/2381001355', 'cluster_addr': '172.21.15.150:6827/2381001355', 'heartbeat_back_addr': '172.21.15.150:6831/2381001355', 'heartbeat_front_addr': '172.21.15.150:6829/2381001355', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.548803+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.075023+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.117036+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.332113+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.330371+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:02.753946+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:02.071186+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.378781+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T05:29:44.730 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-14T05:29:44.731 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:44.731 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:44.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:44.731 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:44.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.732 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:2130 3.1b e30/24 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:head v 30'21, mlcod=30'19)", 2024-09-14T05:29:44.732 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.782900+0000", 2024-09-14T05:29:44.732 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 13.894307768999999, 2024-09-14T05:29:44.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.034589779000000001, 2024-09-14T05:29:44.732 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.732 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.732 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:44.732 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.732 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.732 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.782900+0000", 2024-09-14T05:29:44.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.782900+0000", 2024-09-14T05:29:44.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.782903+0000", 2024-09-14T05:29:44.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6800000000000002e-06 2024-09-14T05:29:44.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.802899+0000", 2024-09-14T05:29:44.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.019996435999999999 2024-09-14T05:29:44.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.734 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.802924+0000", 2024-09-14T05:29:44.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.463e-05 2024-09-14T05:29:44.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.734 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.802934+0000", 2024-09-14T05:29:44.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8800000000000003e-06 2024-09-14T05:29:44.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.803003+0000", 2024-09-14T05:29:44.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9158000000000004e-05 2024-09-14T05:29:44.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.803029+0000", 2024-09-14T05:29:44.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6395999999999999e-05 2024-09-14T05:29:44.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:44.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.817445+0000", 2024-09-14T05:29:44.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.014415258 2024-09-14T05:29:44.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.817490+0000", 2024-09-14T05:29:44.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5340999999999998e-05 2024-09-14T05:29:44.736 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.736 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.736 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.736 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:2132 3.1b e30/24 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:head v 30'22, mlcod=30'19)", 2024-09-14T05:29:44.736 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.802961+0000", 2024-09-14T05:29:44.736 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 13.874246844, 2024-09-14T05:29:44.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026740369, 2024-09-14T05:29:44.737 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.737 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.737 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:44.737 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.802961+0000", 2024-09-14T05:29:44.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.802961+0000", 2024-09-14T05:29:44.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.738 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.738 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.802962+0000", 2024-09-14T05:29:44.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7999999999999993e-07 2024-09-14T05:29:44.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.738 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.738 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.818597+0000", 2024-09-14T05:29:44.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.015634577 2024-09-14T05:29:44.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.738 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.738 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.818609+0000", 2024-09-14T05:29:44.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2669e-05 2024-09-14T05:29:44.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.818617+0000", 2024-09-14T05:29:44.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1620000000000002e-06 2024-09-14T05:29:44.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.818663+0000", 2024-09-14T05:29:44.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6137999999999998e-05 2024-09-14T05:29:44.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.740 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.740 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.818681+0000", 2024-09-14T05:29:44.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8726e-05 2024-09-14T05:29:44.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.740 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:44.740 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.829654+0000", 2024-09-14T05:29:44.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010972986000000001 2024-09-14T05:29:44.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.740 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.741 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.829702+0000", 2024-09-14T05:29:44.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7131e-05 2024-09-14T05:29:44.741 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.741 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.741 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.741 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18155 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 5830656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:44.741 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.529473+0000", 2024-09-14T05:29:44.741 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.14773541000000001, 2024-09-14T05:29:44.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024617763000000001, 2024-09-14T05:29:44.741 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.741 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.742 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:44.742 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:44.742 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:44.742 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.742 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18155 2024-09-14T05:29:44.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.742 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.742 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529473+0000", 2024-09-14T05:29:44.742 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529473+0000", 2024-09-14T05:29:44.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529473+0000", 2024-09-14T05:29:44.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4900000000000001e-07 2024-09-14T05:29:44.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529475+0000", 2024-09-14T05:29:44.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1019999999999999e-06 2024-09-14T05:29:44.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.744 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529481+0000", 2024-09-14T05:29:44.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0979999999999999e-06 2024-09-14T05:29:44.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.744 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529486+0000", 2024-09-14T05:29:44.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8609999999999997e-06 2024-09-14T05:29:44.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.745 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.553868+0000", 2024-09-14T05:29:44.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024382122999999999 2024-09-14T05:29:44.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.745 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.553950+0000", 2024-09-14T05:29:44.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1373999999999997e-05 2024-09-14T05:29:44.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.745 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.554090+0000", 2024-09-14T05:29:44.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014075599999999999 2024-09-14T05:29:44.746 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.746 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.746 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.746 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18156 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 5842944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:44.746 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.529493+0000", 2024-09-14T05:29:44.746 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.14771485000000001, 2024-09-14T05:29:44.746 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024866210999999999, 2024-09-14T05:29:44.746 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.746 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.746 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:44.746 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:44.746 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:44.747 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.747 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18156 2024-09-14T05:29:44.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.747 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.747 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.747 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529493+0000", 2024-09-14T05:29:44.747 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.747 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.747 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529493+0000", 2024-09-14T05:29:44.747 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.748 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.748 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529494+0000", 2024-09-14T05:29:44.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9599999999999999e-07 2024-09-14T05:29:44.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.748 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.748 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529496+0000", 2024-09-14T05:29:44.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9759999999999998e-06 2024-09-14T05:29:44.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.748 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529502+0000", 2024-09-14T05:29:44.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0580000000000003e-06 2024-09-14T05:29:44.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529506+0000", 2024-09-14T05:29:44.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6020000000000002e-06 2024-09-14T05:29:44.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.554127+0000", 2024-09-14T05:29:44.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024620276999999999 2024-09-14T05:29:44.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.554214+0000", 2024-09-14T05:29:44.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7434000000000003e-05 2024-09-14T05:29:44.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.554359+0000", 2024-09-14T05:29:44.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014536799999999999 2024-09-14T05:29:44.750 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.750 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.750 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.751 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18157 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 5863424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:44.751 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.529512+0000", 2024-09-14T05:29:44.751 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.147695823, 2024-09-14T05:29:44.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025106020999999999, 2024-09-14T05:29:44.751 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.751 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.751 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:44.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:44.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:44.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.751 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18157 2024-09-14T05:29:44.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.751 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529512+0000", 2024-09-14T05:29:44.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529512+0000", 2024-09-14T05:29:44.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529513+0000", 2024-09-14T05:29:44.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4400000000000001e-07 2024-09-14T05:29:44.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529515+0000", 2024-09-14T05:29:44.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9350000000000001e-06 2024-09-14T05:29:44.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529521+0000", 2024-09-14T05:29:44.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2249999999999997e-06 2024-09-14T05:29:44.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529526+0000", 2024-09-14T05:29:44.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8799999999999999e-06 2024-09-14T05:29:44.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.554396+0000", 2024-09-14T05:29:44.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024869789999999999 2024-09-14T05:29:44.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.554476+0000", 2024-09-14T05:29:44.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0796000000000004e-05 2024-09-14T05:29:44.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.755 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.554618+0000", 2024-09-14T05:29:44.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000141951 2024-09-14T05:29:44.755 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.755 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.755 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.755 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18158 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 5883904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:44.755 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.529532+0000", 2024-09-14T05:29:44.756 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.147676053, 2024-09-14T05:29:44.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025332225, 2024-09-14T05:29:44.756 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.756 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.756 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:44.756 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:44.756 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:44.756 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.756 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18158 2024-09-14T05:29:44.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.756 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529532+0000", 2024-09-14T05:29:44.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529532+0000", 2024-09-14T05:29:44.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529532+0000", 2024-09-14T05:29:44.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6199999999999998e-07 2024-09-14T05:29:44.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529534+0000", 2024-09-14T05:29:44.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.911e-06 2024-09-14T05:29:44.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529540+0000", 2024-09-14T05:29:44.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9020000000000004e-06 2024-09-14T05:29:44.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529545+0000", 2024-09-14T05:29:44.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8029999999999996e-06 2024-09-14T05:29:44.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.554655+0000", 2024-09-14T05:29:44.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025110127999999999 2024-09-14T05:29:44.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.554734+0000", 2024-09-14T05:29:44.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9079999999999995e-05 2024-09-14T05:29:44.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.554864+0000", 2024-09-14T05:29:44.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000129939 2024-09-14T05:29:44.760 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.760 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.760 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.760 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18287 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 5865472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:44.760 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.534043+0000", 2024-09-14T05:29:44.760 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.14316522600000001, 2024-09-14T05:29:44.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024600286999999998, 2024-09-14T05:29:44.760 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.760 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.761 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:44.761 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:44.761 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:44.761 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.761 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18287 2024-09-14T05:29:44.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.761 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.761 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.761 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.534043+0000", 2024-09-14T05:29:44.761 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.534043+0000", 2024-09-14T05:29:44.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.534043+0000", 2024-09-14T05:29:44.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2099999999999996e-07 2024-09-14T05:29:44.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.534046+0000", 2024-09-14T05:29:44.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7379999999999999e-06 2024-09-14T05:29:44.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.534054+0000", 2024-09-14T05:29:44.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8099999999999998e-06 2024-09-14T05:29:44.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.534060+0000", 2024-09-14T05:29:44.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7450000000000001e-06 2024-09-14T05:29:44.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.558421+0000", 2024-09-14T05:29:44.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024361552000000002 2024-09-14T05:29:44.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.558505+0000", 2024-09-14T05:29:44.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3658000000000004e-05 2024-09-14T05:29:44.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.558643+0000", 2024-09-14T05:29:44.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013816300000000001 2024-09-14T05:29:44.764 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.764 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.764 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.765 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18288 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 5885952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:44.765 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.534082+0000", 2024-09-14T05:29:44.765 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.14312594300000001, 2024-09-14T05:29:44.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024823218000000001, 2024-09-14T05:29:44.765 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.765 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.765 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:44.765 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:44.765 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:44.765 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.765 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18288 2024-09-14T05:29:44.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.766 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.534082+0000", 2024-09-14T05:29:44.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.534082+0000", 2024-09-14T05:29:44.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.534083+0000", 2024-09-14T05:29:44.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6000000000000003e-07 2024-09-14T05:29:44.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.534085+0000", 2024-09-14T05:29:44.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6979999999999998e-06 2024-09-14T05:29:44.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.534092+0000", 2024-09-14T05:29:44.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9979999999999997e-06 2024-09-14T05:29:44.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.534098+0000", 2024-09-14T05:29:44.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4399999999999996e-06 2024-09-14T05:29:44.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.558682+0000", 2024-09-14T05:29:44.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024584335999999998 2024-09-14T05:29:44.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.558760+0000", 2024-09-14T05:29:44.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7335000000000004e-05 2024-09-14T05:29:44.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.769 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.769 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.558905+0000", 2024-09-14T05:29:44.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014575100000000001 2024-09-14T05:29:44.769 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.769 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.769 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.769 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18405 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 5646336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:44.769 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.535853+0000", 2024-09-14T05:29:44.769 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.14135556599999999, 2024-09-14T05:29:44.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024397868, 2024-09-14T05:29:44.770 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.770 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.770 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:44.770 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:44.770 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:44.770 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.770 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18405 2024-09-14T05:29:44.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.770 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535853+0000", 2024-09-14T05:29:44.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535853+0000", 2024-09-14T05:29:44.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535862+0000", 2024-09-14T05:29:44.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2240000000000001e-06 2024-09-14T05:29:44.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535864+0000", 2024-09-14T05:29:44.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5270000000000001e-06 2024-09-14T05:29:44.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535873+0000", 2024-09-14T05:29:44.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1509999999999993e-06 2024-09-14T05:29:44.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535885+0000", 2024-09-14T05:29:44.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1993e-05 2024-09-14T05:29:44.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.560047+0000", 2024-09-14T05:29:44.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024161175999999999 2024-09-14T05:29:44.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.560122+0000", 2024-09-14T05:29:44.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5222000000000006e-05 2024-09-14T05:29:44.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.774 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.560250+0000", 2024-09-14T05:29:44.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012857499999999999 2024-09-14T05:29:44.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.774 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.774 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18406 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 5666816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:44.775 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.535894+0000", 2024-09-14T05:29:44.775 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.141313885, 2024-09-14T05:29:44.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024596106999999999, 2024-09-14T05:29:44.775 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.775 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.775 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:44.775 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:44.775 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:44.775 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.775 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18406 2024-09-14T05:29:44.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.775 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535894+0000", 2024-09-14T05:29:44.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535894+0000", 2024-09-14T05:29:44.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535895+0000", 2024-09-14T05:29:44.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0999999999999999e-07 2024-09-14T05:29:44.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535897+0000", 2024-09-14T05:29:44.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3379999999999999e-06 2024-09-14T05:29:44.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535904+0000", 2024-09-14T05:29:44.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7229999999999998e-06 2024-09-14T05:29:44.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535909+0000", 2024-09-14T05:29:44.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8010000000000003e-06 2024-09-14T05:29:44.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.560281+0000", 2024-09-14T05:29:44.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024372746000000001 2024-09-14T05:29:44.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.560361+0000", 2024-09-14T05:29:44.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9911000000000001e-05 2024-09-14T05:29:44.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.560490+0000", 2024-09-14T05:29:44.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000129078 2024-09-14T05:29:44.779 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.779 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.779 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.780 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18407 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 5687296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:44.780 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.535915+0000", 2024-09-14T05:29:44.780 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.141292848, 2024-09-14T05:29:44.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024796025999999999, 2024-09-14T05:29:44.780 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.780 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.780 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:44.780 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:44.780 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:44.780 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.780 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18407 2024-09-14T05:29:44.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.780 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535915+0000", 2024-09-14T05:29:44.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535915+0000", 2024-09-14T05:29:44.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535916+0000", 2024-09-14T05:29:44.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4299999999999998e-07 2024-09-14T05:29:44.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535918+0000", 2024-09-14T05:29:44.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0779999999999998e-06 2024-09-14T05:29:44.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535924+0000", 2024-09-14T05:29:44.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9710000000000001e-06 2024-09-14T05:29:44.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535929+0000", 2024-09-14T05:29:44.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1250000000000001e-06 2024-09-14T05:29:44.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.560520+0000", 2024-09-14T05:29:44.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024591273 2024-09-14T05:29:44.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.560590+0000", 2024-09-14T05:29:44.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0308000000000005e-05 2024-09-14T05:29:44.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.560711+0000", 2024-09-14T05:29:44.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000120828 2024-09-14T05:29:44.784 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.784 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.784 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.785 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18408 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 5720064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:44.785 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.535934+0000", 2024-09-14T05:29:44.785 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.14127377299999999, 2024-09-14T05:29:44.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025083411, 2024-09-14T05:29:44.785 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.785 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.785 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:44.785 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:44.785 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:44.785 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.786 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18408 2024-09-14T05:29:44.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.786 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535934+0000", 2024-09-14T05:29:44.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535934+0000", 2024-09-14T05:29:44.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535935+0000", 2024-09-14T05:29:44.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4799999999999999e-07 2024-09-14T05:29:44.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535936+0000", 2024-09-14T05:29:44.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6640000000000001e-06 2024-09-14T05:29:44.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535950+0000", 2024-09-14T05:29:44.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3918e-05 2024-09-14T05:29:44.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535955+0000", 2024-09-14T05:29:44.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9280000000000001e-06 2024-09-14T05:29:44.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.560741+0000", 2024-09-14T05:29:44.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.02478611 2024-09-14T05:29:44.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.789 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.789 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.560807+0000", 2024-09-14T05:29:44.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5580999999999994e-05 2024-09-14T05:29:44.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.789 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.789 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.561018+0000", 2024-09-14T05:29:44.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000210862 2024-09-14T05:29:44.789 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.789 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.790 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.790 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18409 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 5740544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:44.790 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.535969+0000", 2024-09-14T05:29:44.790 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.141239057, 2024-09-14T05:29:44.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025273507000000001, 2024-09-14T05:29:44.790 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.790 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.790 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:44.790 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:44.790 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:44.790 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.790 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18409 2024-09-14T05:29:44.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.791 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535969+0000", 2024-09-14T05:29:44.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535969+0000", 2024-09-14T05:29:44.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535969+0000", 2024-09-14T05:29:44.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2e-07 2024-09-14T05:29:44.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535972+0000", 2024-09-14T05:29:44.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4109999999999999e-06 2024-09-14T05:29:44.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535978+0000", 2024-09-14T05:29:44.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1190000000000002e-06 2024-09-14T05:29:44.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535982+0000", 2024-09-14T05:29:44.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.51e-06 2024-09-14T05:29:44.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.561049+0000", 2024-09-14T05:29:44.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025066606000000002 2024-09-14T05:29:44.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.561118+0000", 2024-09-14T05:29:44.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8938999999999998e-05 2024-09-14T05:29:44.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.561243+0000", 2024-09-14T05:29:44.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012450199999999999 2024-09-14T05:29:44.794 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.794 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.794 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.794 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18410 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 5761024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:44.794 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.535988+0000", 2024-09-14T05:29:44.794 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.14121966799999999, 2024-09-14T05:29:44.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025472058999999998, 2024-09-14T05:29:44.795 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.795 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.795 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:44.795 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:44.795 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:44.795 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.795 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18410 2024-09-14T05:29:44.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.795 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535988+0000", 2024-09-14T05:29:44.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535988+0000", 2024-09-14T05:29:44.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535989+0000", 2024-09-14T05:29:44.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0400000000000002e-07 2024-09-14T05:29:44.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535991+0000", 2024-09-14T05:29:44.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0420000000000001e-06 2024-09-14T05:29:44.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.535997+0000", 2024-09-14T05:29:44.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3180000000000003e-06 2024-09-14T05:29:44.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536002+0000", 2024-09-14T05:29:44.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0350000000000001e-06 2024-09-14T05:29:44.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.561273+0000", 2024-09-14T05:29:44.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025270707999999999 2024-09-14T05:29:44.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.561339+0000", 2024-09-14T05:29:44.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6308000000000003e-05 2024-09-14T05:29:44.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.561460+0000", 2024-09-14T05:29:44.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012124400000000001 2024-09-14T05:29:44.798 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.798 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.798 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.799 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18411 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 5773312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:44.799 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.536009+0000", 2024-09-14T05:29:44.799 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.141198928, 2024-09-14T05:29:44.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025669636999999999, 2024-09-14T05:29:44.799 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.799 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.799 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:44.799 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:44.799 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:44.799 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.799 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18411 2024-09-14T05:29:44.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.800 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536009+0000", 2024-09-14T05:29:44.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536009+0000", 2024-09-14T05:29:44.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536010+0000", 2024-09-14T05:29:44.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1399999999999997e-07 2024-09-14T05:29:44.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536012+0000", 2024-09-14T05:29:44.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1780000000000002e-06 2024-09-14T05:29:44.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536021+0000", 2024-09-14T05:29:44.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.003e-06 2024-09-14T05:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536026+0000", 2024-09-14T05:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2850000000000002e-06 2024-09-14T05:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.561491+0000", 2024-09-14T05:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025464784000000001 2024-09-14T05:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.561557+0000", 2024-09-14T05:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5792999999999998e-05 2024-09-14T05:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.561679+0000", 2024-09-14T05:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012218 2024-09-14T05:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18412 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 5793792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.536033+0000", 2024-09-14T05:29:44.804 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.141175468, 2024-09-14T05:29:44.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025869762000000001, 2024-09-14T05:29:44.804 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.804 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.804 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:44.804 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:44.804 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:44.804 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.804 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18412 2024-09-14T05:29:44.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.804 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536033+0000", 2024-09-14T05:29:44.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536033+0000", 2024-09-14T05:29:44.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536033+0000", 2024-09-14T05:29:44.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4299999999999998e-07 2024-09-14T05:29:44.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536045+0000", 2024-09-14T05:29:44.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1936999999999999e-05 2024-09-14T05:29:44.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536055+0000", 2024-09-14T05:29:44.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6809999999999993e-06 2024-09-14T05:29:44.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536060+0000", 2024-09-14T05:29:44.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1279999999999999e-06 2024-09-14T05:29:44.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.561709+0000", 2024-09-14T05:29:44.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025649511999999999 2024-09-14T05:29:44.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.561776+0000", 2024-09-14T05:29:44.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6507999999999994e-05 2024-09-14T05:29:44.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.561902+0000", 2024-09-14T05:29:44.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012655300000000001 2024-09-14T05:29:44.808 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.808 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.808 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.808 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18413 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 5814272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:44.808 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.536067+0000", 2024-09-14T05:29:44.808 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.141140714, 2024-09-14T05:29:44.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026718025999999999, 2024-09-14T05:29:44.808 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.808 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.808 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:44.808 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:44.808 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:44.809 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.809 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18413 2024-09-14T05:29:44.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.809 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536067+0000", 2024-09-14T05:29:44.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536067+0000", 2024-09-14T05:29:44.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536068+0000", 2024-09-14T05:29:44.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8699999999999995e-07 2024-09-14T05:29:44.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536070+0000", 2024-09-14T05:29:44.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3369999999999998e-06 2024-09-14T05:29:44.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536076+0000", 2024-09-14T05:29:44.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2539999999999998e-06 2024-09-14T05:29:44.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536081+0000", 2024-09-14T05:29:44.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7090000000000001e-06 2024-09-14T05:29:44.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.561932+0000", 2024-09-14T05:29:44.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025850374999999998 2024-09-14T05:29:44.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.562005+0000", 2024-09-14T05:29:44.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3198e-05 2024-09-14T05:29:44.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.562785+0000", 2024-09-14T05:29:44.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00078066599999999996 2024-09-14T05:29:44.812 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.812 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.812 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.812 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18414 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 5847040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:44.813 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.536087+0000", 2024-09-14T05:29:44.813 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.14112070500000001, 2024-09-14T05:29:44.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026271103000000001, 2024-09-14T05:29:44.813 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.813 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.813 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:44.813 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:44.813 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:44.813 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.813 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18414 2024-09-14T05:29:44.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.813 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536087+0000", 2024-09-14T05:29:44.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536087+0000", 2024-09-14T05:29:44.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536088+0000", 2024-09-14T05:29:44.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6800000000000001e-07 2024-09-14T05:29:44.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536092+0000", 2024-09-14T05:29:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5569999999999997e-06 2024-09-14T05:29:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536101+0000", 2024-09-14T05:29:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8349999999999993e-06 2024-09-14T05:29:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536107+0000", 2024-09-14T05:29:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4e-06 2024-09-14T05:29:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.562148+0000", 2024-09-14T05:29:44.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026041229999999999 2024-09-14T05:29:44.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.562225+0000", 2024-09-14T05:29:44.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7305999999999995e-05 2024-09-14T05:29:44.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.562358+0000", 2024-09-14T05:29:44.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013330699999999999 2024-09-14T05:29:44.817 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.817 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.817 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.817 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18415 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 5867520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:44.817 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.536114+0000", 2024-09-14T05:29:44.817 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.141094306, 2024-09-14T05:29:44.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026458223999999999, 2024-09-14T05:29:44.817 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.817 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.817 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:44.818 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:44.818 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:44.818 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.818 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18415 2024-09-14T05:29:44.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.818 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536114+0000", 2024-09-14T05:29:44.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536114+0000", 2024-09-14T05:29:44.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536114+0000", 2024-09-14T05:29:44.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7700000000000005e-07 2024-09-14T05:29:44.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536116+0000", 2024-09-14T05:29:44.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1040000000000001e-06 2024-09-14T05:29:44.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536122+0000", 2024-09-14T05:29:44.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7409999999999998e-06 2024-09-14T05:29:44.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536127+0000", 2024-09-14T05:29:44.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8060000000000002e-06 2024-09-14T05:29:44.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.562384+0000", 2024-09-14T05:29:44.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026257466 2024-09-14T05:29:44.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.562451+0000", 2024-09-14T05:29:44.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7112000000000004e-05 2024-09-14T05:29:44.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.562572+0000", 2024-09-14T05:29:44.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000120518 2024-09-14T05:29:44.821 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.821 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.821 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.822 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18416 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 5888000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:44.822 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.536134+0000", 2024-09-14T05:29:44.822 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.14107403099999999, 2024-09-14T05:29:44.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026645426, 2024-09-14T05:29:44.822 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:44.822 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:44.822 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:44.822 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:44.822 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:44.822 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:44.822 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18416 2024-09-14T05:29:44.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.823 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:44.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:44.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536134+0000", 2024-09-14T05:29:44.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:44.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536134+0000", 2024-09-14T05:29:44.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:44.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:44.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536135+0000", 2024-09-14T05:29:44.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8999999999999997e-07 2024-09-14T05:29:44.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:44.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536137+0000", 2024-09-14T05:29:44.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0310000000000001e-06 2024-09-14T05:29:44.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:44.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536145+0000", 2024-09-14T05:29:44.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7360000000000007e-06 2024-09-14T05:29:44.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:44.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.536151+0000", 2024-09-14T05:29:44.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4559999999999999e-06 2024-09-14T05:29:44.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:44.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.562598+0000", 2024-09-14T05:29:44.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026446844000000001 2024-09-14T05:29:44.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:44.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.562663+0000", 2024-09-14T05:29:44.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5099999999999997e-05 2024-09-14T05:29:44.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:44.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:44.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:44.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.562779+0000", 2024-09-14T05:29:44.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011676900000000001 2024-09-14T05:29:44.826 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.826 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.826 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.826 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:44.826 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:44.826 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:44.827 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:29:44.827 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:44.728+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:44.830 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:44.829+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:44.867 INFO:teuthology.orchestra.run.smithi071.stdout:1554 write 0x4ff5fb2 thru 0x5002e76 (0xcec5 bytes) 2024-09-14T05:29:44.930 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:44.930+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:44.967 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:29:44.968 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:44.971 INFO:teuthology.orchestra.run.smithi071.stdout:1555 read 0x8d2e894 thru 0x8d3948a (0xabf7 bytes) 2024-09-14T05:29:44.974 INFO:teuthology.orchestra.run.smithi071.stdout:1556 write 0x11f5bcc thru 0x1205a19 (0xfe4e bytes) 2024-09-14T05:29:44.980 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:29:45.031 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:45.030+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:45.056 INFO:teuthology.orchestra.run.smithi071.stdout:1557 punch from 0x6a07926 to 0x6a0e064, (0x673e bytes) 2024-09-14T05:29:45.065 INFO:teuthology.orchestra.run.smithi071.stdout:1559 trunc from 0xd9e54a7 to 0x6393d09 2024-09-14T05:29:45.132 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:45.131+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:45.137 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:45.137 INFO:teuthology.orchestra.run.smithi071.stdout:1561 punch from 0x3b2d57a to 0x3b30a80, (0x3506 bytes) 2024-09-14T05:29:45.137 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:45.137 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:45.147 INFO:teuthology.orchestra.run.smithi071.stdout:1564 writesame 0xa8844ef thru 0xa891be5 data_size 0x1(0xd6f7 bytes) 2024-09-14T05:29:45.147 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:45.150 INFO:teuthology.orchestra.run.smithi071.stdout:1566 trunc from 0xa891be6 to 0xcf432d9 2024-09-14T05:29:45.159 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:45.159 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:45.159 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:45.159 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:45.159 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:45.166 INFO:teuthology.orchestra.run.smithi071.stdout:1568 trunc from 0xcf432d9 to 0x3fad95a 2024-09-14T05:29:45.173 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:29:45.256 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:45.256+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:45.268 INFO:teuthology.orchestra.run.smithi071.stdout:1569 write 0x497a441 thru 0x497df83 (0x3b43 bytes) 2024-09-14T05:29:45.278 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:45.278 INFO:teuthology.orchestra.run.smithi071.stdout:1572 write 0x35c3445 thru 0x35ca34d (0x6f09 bytes) 2024-09-14T05:29:45.333 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:45.333+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:45.339 INFO:teuthology.orchestra.run.smithi071.stdout:1573 read 0x8435db thru 0x84afe1 (0x7a07 bytes) 2024-09-14T05:29:45.342 INFO:teuthology.orchestra.run.smithi071.stdout:1577 write 0x41315d5 thru 0x4131cc8 (0x6f4 bytes) 2024-09-14T05:29:45.346 INFO:teuthology.orchestra.run.smithi071.stdout:1578 trunc from 0x497df84 to 0x54691fe 2024-09-14T05:29:45.350 INFO:teuthology.orchestra.run.smithi071.stdout:1580 punch from 0x241427b to 0x241cc7b, (0x8a00 bytes) 2024-09-14T05:29:45.358 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:45.358 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:45.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:45.359 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:45.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.359 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:45.359 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:29:45.359 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.580413395000001, 2024-09-14T05:29:45.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.24562984099999999, 2024-09-14T05:29:45.359 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.359 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.359 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:45.359 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.359 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:45.360 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:45.360 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1 2024-09-14T05:29:45.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.360 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:29:45.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:29:45.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776565+0000", 2024-09-14T05:29:45.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4780000000000001e-06 2024-09-14T05:29:45.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776571+0000", 2024-09-14T05:29:45.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7669999999999998e-06 2024-09-14T05:29:45.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776580+0000", 2024-09-14T05:29:45.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9830000000000002e-06 2024-09-14T05:29:45.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.362 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.362 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776589+0000", 2024-09-14T05:29:45.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7579999999999998e-06 2024-09-14T05:29:45.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.362 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.362 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776662+0000", 2024-09-14T05:29:45.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3332999999999997e-05 2024-09-14T05:29:45.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.362 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:29:45.363 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776670+0000", 2024-09-14T05:29:45.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9850000000000006e-06 2024-09-14T05:29:45.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.363 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.363 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.020867+0000", 2024-09-14T05:29:45.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.244196265 2024-09-14T05:29:45.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.363 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.363 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021022+0000", 2024-09-14T05:29:45.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000155699 2024-09-14T05:29:45.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:45.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021169+0000", 2024-09-14T05:29:45.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014661700000000001 2024-09-14T05:29:45.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:45.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021891+0000", 2024-09-14T05:29:45.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072146700000000005 2024-09-14T05:29:45.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:45.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022124+0000", 2024-09-14T05:29:45.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023354199999999999 2024-09-14T05:29:45.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:45.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022164+0000", 2024-09-14T05:29:45.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9897000000000001e-05 2024-09-14T05:29:45.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022193+0000", 2024-09-14T05:29:45.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9050000000000001e-05 2024-09-14T05:29:45.365 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.366 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.366 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.366 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:1989 3.1f e30/24 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head v 30'29, mlcod=30'27)", 2024-09-14T05:29:45.366 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:29:45.366 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 14.763642408000001, 2024-09-14T05:29:45.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.064242492999999998, 2024-09-14T05:29:45.366 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.366 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.367 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:45.367 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.367 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:29:45.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.367 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:29:45.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593336+0000", 2024-09-14T05:29:45.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3909999999999999e-06 2024-09-14T05:29:45.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616632+0000", 2024-09-14T05:29:45.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.023296598000000002 2024-09-14T05:29:45.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616657+0000", 2024-09-14T05:29:45.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5228e-05 2024-09-14T05:29:45.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616684+0000", 2024-09-14T05:29:45.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6441999999999999e-05 2024-09-14T05:29:45.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.370 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616755+0000", 2024-09-14T05:29:45.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1452000000000006e-05 2024-09-14T05:29:45.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.370 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616780+0000", 2024-09-14T05:29:45.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4436000000000001e-05 2024-09-14T05:29:45.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:45.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657527+0000", 2024-09-14T05:29:45.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.040747323000000002 2024-09-14T05:29:45.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657577+0000", 2024-09-14T05:29:45.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9623000000000002e-05 2024-09-14T05:29:45.371 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.371 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.371 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.372 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18107 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 5834752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.372 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.529233+0000", 2024-09-14T05:29:45.372 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.82774311599999995, 2024-09-14T05:29:45.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0259046, 2024-09-14T05:29:45.372 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.372 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.372 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.372 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.372 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.372 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.373 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18107 2024-09-14T05:29:45.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.373 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529233+0000", 2024-09-14T05:29:45.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529233+0000", 2024-09-14T05:29:45.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529234+0000", 2024-09-14T05:29:45.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2199999999999999e-07 2024-09-14T05:29:45.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529237+0000", 2024-09-14T05:29:45.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5389999999999999e-06 2024-09-14T05:29:45.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529243+0000", 2024-09-14T05:29:45.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0750000000000003e-06 2024-09-14T05:29:45.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529248+0000", 2024-09-14T05:29:45.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7700000000000001e-06 2024-09-14T05:29:45.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.554931+0000", 2024-09-14T05:29:45.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025682778 2024-09-14T05:29:45.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.555026+0000", 2024-09-14T05:29:45.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5199999999999997e-05 2024-09-14T05:29:45.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.555138+0000", 2024-09-14T05:29:45.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000111816 2024-09-14T05:29:45.377 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.377 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.377 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.377 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18108 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 5855232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.377 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.529255+0000", 2024-09-14T05:29:45.377 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.82772167799999996, 2024-09-14T05:29:45.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026090665999999998, 2024-09-14T05:29:45.377 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.377 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.377 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.378 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.378 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.378 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.378 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18108 2024-09-14T05:29:45.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.378 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529255+0000", 2024-09-14T05:29:45.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529255+0000", 2024-09-14T05:29:45.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529255+0000", 2024-09-14T05:29:45.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2399999999999999e-07 2024-09-14T05:29:45.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529257+0000", 2024-09-14T05:29:45.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0959999999999999e-06 2024-09-14T05:29:45.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529263+0000", 2024-09-14T05:29:45.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0329999999999997e-06 2024-09-14T05:29:45.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529268+0000", 2024-09-14T05:29:45.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6870000000000002e-06 2024-09-14T05:29:45.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.555168+0000", 2024-09-14T05:29:45.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025899769999999999 2024-09-14T05:29:45.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.555233+0000", 2024-09-14T05:29:45.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4629999999999996e-05 2024-09-14T05:29:45.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.555346+0000", 2024-09-14T05:29:45.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000113026 2024-09-14T05:29:45.382 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.382 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.382 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.382 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18109 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 5875712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.382 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.529275+0000", 2024-09-14T05:29:45.382 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.827701927, 2024-09-14T05:29:45.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026329918000000001, 2024-09-14T05:29:45.383 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.383 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.383 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.383 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.383 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.383 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.383 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18109 2024-09-14T05:29:45.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.383 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529275+0000", 2024-09-14T05:29:45.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529275+0000", 2024-09-14T05:29:45.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529275+0000", 2024-09-14T05:29:45.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9799999999999999e-07 2024-09-14T05:29:45.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529277+0000", 2024-09-14T05:29:45.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.063e-06 2024-09-14T05:29:45.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529283+0000", 2024-09-14T05:29:45.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.964e-06 2024-09-14T05:29:45.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529287+0000", 2024-09-14T05:29:45.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3730000000000003e-06 2024-09-14T05:29:45.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.555374+0000", 2024-09-14T05:29:45.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026086964000000001 2024-09-14T05:29:45.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.555486+0000", 2024-09-14T05:29:45.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000111623 2024-09-14T05:29:45.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.555605+0000", 2024-09-14T05:29:45.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000118533 2024-09-14T05:29:45.387 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.387 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.387 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.388 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18110 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 5896192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.388 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.529294+0000", 2024-09-14T05:29:45.388 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.82768275700000005, 2024-09-14T05:29:45.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026518798, 2024-09-14T05:29:45.388 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.388 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.388 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.388 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.388 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.389 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.389 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18110 2024-09-14T05:29:45.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.389 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529294+0000", 2024-09-14T05:29:45.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529294+0000", 2024-09-14T05:29:45.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529294+0000", 2024-09-14T05:29:45.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4099999999999999e-07 2024-09-14T05:29:45.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529296+0000", 2024-09-14T05:29:45.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1459999999999999e-06 2024-09-14T05:29:45.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529302+0000", 2024-09-14T05:29:45.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8730000000000004e-06 2024-09-14T05:29:45.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529307+0000", 2024-09-14T05:29:45.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.887e-06 2024-09-14T05:29:45.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.555635+0000", 2024-09-14T05:29:45.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026327518000000001 2024-09-14T05:29:45.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.555700+0000", 2024-09-14T05:29:45.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5214000000000003e-05 2024-09-14T05:29:45.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.555813+0000", 2024-09-14T05:29:45.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000112719 2024-09-14T05:29:45.393 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.393 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.393 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.393 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18228 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 5677056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.393 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.532803+0000", 2024-09-14T05:29:45.394 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.82417325200000002, 2024-09-14T05:29:45.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026086107000000001, 2024-09-14T05:29:45.394 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.394 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.394 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.394 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.394 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.394 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.394 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18228 2024-09-14T05:29:45.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.394 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.532803+0000", 2024-09-14T05:29:45.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.532803+0000", 2024-09-14T05:29:45.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.532804+0000", 2024-09-14T05:29:45.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-14T05:29:45.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.532806+0000", 2024-09-14T05:29:45.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.137e-06 2024-09-14T05:29:45.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.532811+0000", 2024-09-14T05:29:45.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.57e-06 2024-09-14T05:29:45.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.397 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.532816+0000", 2024-09-14T05:29:45.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.882e-06 2024-09-14T05:29:45.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.397 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.556894+0000", 2024-09-14T05:29:45.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024077497 2024-09-14T05:29:45.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.556958+0000", 2024-09-14T05:29:45.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4485000000000003e-05 2024-09-14T05:29:45.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.558889+0000", 2024-09-14T05:29:45.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019310760000000001 2024-09-14T05:29:45.399 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.399 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.399 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.399 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18237 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 5857280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.399 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.533030+0000", 2024-09-14T05:29:45.399 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.82394617999999997, 2024-09-14T05:29:45.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025983968999999999, 2024-09-14T05:29:45.399 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.400 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.400 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.400 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.400 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.400 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.400 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18237 2024-09-14T05:29:45.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.400 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.533030+0000", 2024-09-14T05:29:45.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.533030+0000", 2024-09-14T05:29:45.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.533031+0000", 2024-09-14T05:29:45.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6800000000000001e-07 2024-09-14T05:29:45.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.533033+0000", 2024-09-14T05:29:45.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.041e-06 2024-09-14T05:29:45.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.533039+0000", 2024-09-14T05:29:45.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0229999999999998e-06 2024-09-14T05:29:45.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.533044+0000", 2024-09-14T05:29:45.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0459999999999996e-06 2024-09-14T05:29:45.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.403 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.558814+0000", 2024-09-14T05:29:45.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025770029 2024-09-14T05:29:45.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.403 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.558890+0000", 2024-09-14T05:29:45.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6105000000000004e-05 2024-09-14T05:29:45.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.404 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.404 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.559014+0000", 2024-09-14T05:29:45.404 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012425700000000001 2024-09-14T05:29:45.404 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.404 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.404 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.404 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18238 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 5877760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.404 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.533057+0000", 2024-09-14T05:29:45.404 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.82391994599999996, 2024-09-14T05:29:45.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026177656000000001, 2024-09-14T05:29:45.405 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.405 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.405 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.405 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.405 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.405 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.405 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18238 2024-09-14T05:29:45.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.405 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.533057+0000", 2024-09-14T05:29:45.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.533057+0000", 2024-09-14T05:29:45.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.533057+0000", 2024-09-14T05:29:45.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8500000000000002e-07 2024-09-14T05:29:45.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.533059+0000", 2024-09-14T05:29:45.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0940000000000002e-06 2024-09-14T05:29:45.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.533068+0000", 2024-09-14T05:29:45.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5769999999999994e-06 2024-09-14T05:29:45.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.533073+0000", 2024-09-14T05:29:45.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8919999999999999e-06 2024-09-14T05:29:45.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.559044+0000", 2024-09-14T05:29:45.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025971483 2024-09-14T05:29:45.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.559115+0000", 2024-09-14T05:29:45.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0483999999999998e-05 2024-09-14T05:29:45.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.409 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.409 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.559234+0000", 2024-09-14T05:29:45.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000119641 2024-09-14T05:29:45.409 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.409 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.409 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.409 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:20815 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 10653696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.410 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.889060+0000", 2024-09-14T05:29:45.410 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.46791650000000001, 2024-09-14T05:29:45.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025911152999999999, 2024-09-14T05:29:45.410 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.410 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.410 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.410 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.410 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.410 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.410 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20815 2024-09-14T05:29:45.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.410 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.889060+0000", 2024-09-14T05:29:45.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.889060+0000", 2024-09-14T05:29:45.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.889060+0000", 2024-09-14T05:29:45.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7599999999999998e-07 2024-09-14T05:29:45.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.889062+0000", 2024-09-14T05:29:45.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7710000000000001e-06 2024-09-14T05:29:45.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.889068+0000", 2024-09-14T05:29:45.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.942e-06 2024-09-14T05:29:45.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.889073+0000", 2024-09-14T05:29:45.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.921e-06 2024-09-14T05:29:45.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.914812+0000", 2024-09-14T05:29:45.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025738706 2024-09-14T05:29:45.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.914869+0000", 2024-09-14T05:29:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6755999999999997e-05 2024-09-14T05:29:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.914971+0000", 2024-09-14T05:29:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000102681 2024-09-14T05:29:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:20816 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 10674176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.889079+0000", 2024-09-14T05:29:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.46789773200000001, 2024-09-14T05:29:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026074927000000001, 2024-09-14T05:29:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20816 2024-09-14T05:29:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.889079+0000", 2024-09-14T05:29:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.889079+0000", 2024-09-14T05:29:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.889079+0000", 2024-09-14T05:29:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9000000000000002e-07 2024-09-14T05:29:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.889081+0000", 2024-09-14T05:29:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.761e-06 2024-09-14T05:29:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.889087+0000", 2024-09-14T05:29:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9869999999999996e-06 2024-09-14T05:29:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.889092+0000", 2024-09-14T05:29:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0710000000000002e-06 2024-09-14T05:29:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.915001+0000", 2024-09-14T05:29:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025908646 2024-09-14T05:29:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.915060+0000", 2024-09-14T05:29:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9673000000000003e-05 2024-09-14T05:29:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.915154+0000", 2024-09-14T05:29:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3399000000000005e-05 2024-09-14T05:29:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:20817 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 10686464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.889099+0000", 2024-09-14T05:29:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.46787804900000002, 2024-09-14T05:29:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026240507, 2024-09-14T05:29:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20817 2024-09-14T05:29:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.889099+0000", 2024-09-14T05:29:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.889099+0000", 2024-09-14T05:29:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.889099+0000", 2024-09-14T05:29:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9499999999999998e-07 2024-09-14T05:29:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.889101+0000", 2024-09-14T05:29:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.855e-06 2024-09-14T05:29:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.889107+0000", 2024-09-14T05:29:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3640000000000004e-06 2024-09-14T05:29:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.889112+0000", 2024-09-14T05:29:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0429999999999998e-06 2024-09-14T05:29:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.915182+0000", 2024-09-14T05:29:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026069631999999999 2024-09-14T05:29:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.915243+0000", 2024-09-14T05:29:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0897000000000003e-05 2024-09-14T05:29:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.915339+0000", 2024-09-14T05:29:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.6321000000000002e-05 2024-09-14T05:29:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:20821 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 10727424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.893666+0000", 2024-09-14T05:29:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.46331010500000003, 2024-09-14T05:29:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027326612, 2024-09-14T05:29:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20821 2024-09-14T05:29:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.893666+0000", 2024-09-14T05:29:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.893666+0000", 2024-09-14T05:29:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.893667+0000", 2024-09-14T05:29:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.92e-07 2024-09-14T05:29:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.893671+0000", 2024-09-14T05:29:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4450000000000001e-06 2024-09-14T05:29:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.893679+0000", 2024-09-14T05:29:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9319999999999996e-06 2024-09-14T05:29:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.893689+0000", 2024-09-14T05:29:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0336999999999999e-05 2024-09-14T05:29:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.915565+0000", 2024-09-14T05:29:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.021876321000000001 2024-09-14T05:29:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.915630+0000", 2024-09-14T05:29:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4511999999999995e-05 2024-09-14T05:29:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.920993+0000", 2024-09-14T05:29:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0053633730000000003 2024-09-14T05:29:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21076 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 10678272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.898604+0000", 2024-09-14T05:29:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.45837250499999999, 2024-09-14T05:29:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025936101999999999, 2024-09-14T05:29:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21076 2024-09-14T05:29:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898604+0000", 2024-09-14T05:29:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898604+0000", 2024-09-14T05:29:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898605+0000", 2024-09-14T05:29:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8899999999999999e-07 2024-09-14T05:29:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898608+0000", 2024-09-14T05:29:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.033e-06 2024-09-14T05:29:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898616+0000", 2024-09-14T05:29:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5210000000000002e-06 2024-09-14T05:29:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898622+0000", 2024-09-14T05:29:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0100000000000001e-06 2024-09-14T05:29:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.924410+0000", 2024-09-14T05:29:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025787800999999999 2024-09-14T05:29:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.924473+0000", 2024-09-14T05:29:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2927000000000002e-05 2024-09-14T05:29:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.924540+0000", 2024-09-14T05:29:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7120999999999997e-05 2024-09-14T05:29:45.436 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.436 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.436 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.436 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21077 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 10690560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.436 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.898691+0000", 2024-09-14T05:29:45.436 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.45828575900000001, 2024-09-14T05:29:45.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026075574000000001, 2024-09-14T05:29:45.436 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.436 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.436 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21077 2024-09-14T05:29:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898691+0000", 2024-09-14T05:29:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898691+0000", 2024-09-14T05:29:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898691+0000", 2024-09-14T05:29:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4199999999999995e-07 2024-09-14T05:29:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898695+0000", 2024-09-14T05:29:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2789999999999999e-06 2024-09-14T05:29:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898703+0000", 2024-09-14T05:29:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4509999999999992e-06 2024-09-14T05:29:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898709+0000", 2024-09-14T05:29:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2779999999999999e-06 2024-09-14T05:29:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.924593+0000", 2024-09-14T05:29:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025884045000000001 2024-09-14T05:29:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.924679+0000", 2024-09-14T05:29:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5877000000000005e-05 2024-09-14T05:29:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.924766+0000", 2024-09-14T05:29:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7002000000000005e-05 2024-09-14T05:29:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21078 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 10711040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.898719+0000", 2024-09-14T05:29:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.458257886, 2024-09-14T05:29:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026242636999999999, 2024-09-14T05:29:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21078 2024-09-14T05:29:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898719+0000", 2024-09-14T05:29:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898719+0000", 2024-09-14T05:29:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898719+0000", 2024-09-14T05:29:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4799999999999999e-07 2024-09-14T05:29:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898727+0000", 2024-09-14T05:29:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5079999999999999e-06 2024-09-14T05:29:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898734+0000", 2024-09-14T05:29:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2150000000000004e-06 2024-09-14T05:29:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898740+0000", 2024-09-14T05:29:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0039999999999996e-06 2024-09-14T05:29:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.924798+0000", 2024-09-14T05:29:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026058230000000002 2024-09-14T05:29:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.924876+0000", 2024-09-14T05:29:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8126000000000004e-05 2024-09-14T05:29:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.924961+0000", 2024-09-14T05:29:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5105999999999995e-05 2024-09-14T05:29:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21079 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 10731520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.898781+0000", 2024-09-14T05:29:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.45819546700000002, 2024-09-14T05:29:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026369606, 2024-09-14T05:29:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21079 2024-09-14T05:29:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898781+0000", 2024-09-14T05:29:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898781+0000", 2024-09-14T05:29:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898782+0000", 2024-09-14T05:29:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8800000000000002e-07 2024-09-14T05:29:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898784+0000", 2024-09-14T05:29:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7439999999999999e-06 2024-09-14T05:29:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898793+0000", 2024-09-14T05:29:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2260000000000005e-06 2024-09-14T05:29:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898799+0000", 2024-09-14T05:29:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.353e-06 2024-09-14T05:29:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.924995+0000", 2024-09-14T05:29:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026195897999999999 2024-09-14T05:29:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.925076+0000", 2024-09-14T05:29:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1012000000000003e-05 2024-09-14T05:29:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.925151+0000", 2024-09-14T05:29:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4784999999999996e-05 2024-09-14T05:29:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21080 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 10764288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.898911+0000", 2024-09-14T05:29:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.458065903, 2024-09-14T05:29:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026414006, 2024-09-14T05:29:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21080 2024-09-14T05:29:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898911+0000", 2024-09-14T05:29:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898911+0000", 2024-09-14T05:29:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898911+0000", 2024-09-14T05:29:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5600000000000005e-07 2024-09-14T05:29:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898915+0000", 2024-09-14T05:29:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4989999999999999e-06 2024-09-14T05:29:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898924+0000", 2024-09-14T05:29:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9609999999999994e-06 2024-09-14T05:29:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898936+0000", 2024-09-14T05:29:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1976e-05 2024-09-14T05:29:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.925182+0000", 2024-09-14T05:29:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026246427999999999 2024-09-14T05:29:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.925250+0000", 2024-09-14T05:29:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8040000000000006e-05 2024-09-14T05:29:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.925325+0000", 2024-09-14T05:29:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4445999999999998e-05 2024-09-14T05:29:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21081 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 10784768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.898946+0000", 2024-09-14T05:29:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.45803006899999998, 2024-09-14T05:29:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026556192999999999, 2024-09-14T05:29:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21081 2024-09-14T05:29:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898946+0000", 2024-09-14T05:29:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898946+0000", 2024-09-14T05:29:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898947+0000", 2024-09-14T05:29:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3099999999999998e-07 2024-09-14T05:29:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898949+0000", 2024-09-14T05:29:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3760000000000002e-06 2024-09-14T05:29:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898957+0000", 2024-09-14T05:29:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3989999999999998e-06 2024-09-14T05:29:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898963+0000", 2024-09-14T05:29:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.772e-06 2024-09-14T05:29:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.925362+0000", 2024-09-14T05:29:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026398978 2024-09-14T05:29:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.925435+0000", 2024-09-14T05:29:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3147999999999998e-05 2024-09-14T05:29:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.925503+0000", 2024-09-14T05:29:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7989000000000002e-05 2024-09-14T05:29:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21082 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 10805248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.898970+0000", 2024-09-14T05:29:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.45800701999999999, 2024-09-14T05:29:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026769630999999999, 2024-09-14T05:29:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21082 2024-09-14T05:29:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898970+0000", 2024-09-14T05:29:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898970+0000", 2024-09-14T05:29:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898970+0000", 2024-09-14T05:29:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8099999999999998e-07 2024-09-14T05:29:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898972+0000", 2024-09-14T05:29:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0150000000000002e-06 2024-09-14T05:29:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898979+0000", 2024-09-14T05:29:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6529999999999997e-06 2024-09-14T05:29:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898984+0000", 2024-09-14T05:29:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3430000000000003e-06 2024-09-14T05:29:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.925572+0000", 2024-09-14T05:29:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026587745999999999 2024-09-14T05:29:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.925656+0000", 2024-09-14T05:29:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4488999999999997e-05 2024-09-14T05:29:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.925739+0000", 2024-09-14T05:29:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3003999999999993e-05 2024-09-14T05:29:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.469 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:45.469 INFO:teuthology.orchestra.run.smithi071.stdout:1582 write 0x7d71fc5 thru 0x7d79bf8 (0x7c34 bytes) 2024-09-14T05:29:45.470 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:29:45.471 INFO:teuthology.orchestra.run.smithi071.stdout:1583 write 0x6efb31f thru 0x6efb83f (0x521 bytes) 2024-09-14T05:29:45.472 INFO:teuthology.orchestra.run.smithi071.stdout:1587 writesame 0xd04b06f thru 0xd052557 data_size 0x1(0x74e9 bytes) 2024-09-14T05:29:45.472 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:45.472 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:45.472 INFO:teuthology.orchestra.run.smithi071.stdout:1590 write 0xb7461dd thru 0xb74e116 (0x7f3a bytes) 2024-09-14T05:29:45.472 INFO:teuthology.orchestra.run.smithi071.stdout:1592 trunc from 0xd052558 to 0xb07f4ad 2024-09-14T05:29:45.472 INFO:teuthology.orchestra.run.smithi071.stdout:1593 writesame 0x6c47f6b thru 0x6c51afb data_size 0x87(0x9b91 bytes) 2024-09-14T05:29:45.472 INFO:teuthology.orchestra.run.smithi071.stdout:1594 write 0xea9dda7 thru 0xeaa2b44 (0x4d9e bytes) 2024-09-14T05:29:45.472 INFO:teuthology.orchestra.run.smithi071.stdout:1595 write 0x876cd26 thru 0x8772736 (0x5a11 bytes) 2024-09-14T05:29:45.472 INFO:teuthology.orchestra.run.smithi071.stdout:1597 clone 15 order 19 su 16384 sc 10 2024-09-14T05:29:45.512 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:45.511+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:45.574 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:45.574+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:45.614 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:45.614 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:45.614 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:29:45.614 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:45.623 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:29:45.675 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:45.675+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:45.724 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:45.724 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:45.725 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:45.725 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:45.725 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:45.733 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:29:45.777 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:45.776+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:45.837 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:45.837 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:45.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:45.837 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:2033 3.1e e30/24 3:7e47f6eb:::rbd_data.112d1f36d097.0000000000000002:head v 30'60, mlcod=30'57)", 2024-09-14T05:29:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.693515+0000", 2024-09-14T05:29:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.142444643999999, 2024-09-14T05:29:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044021994000000002, 2024-09-14T05:29:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.693515+0000", 2024-09-14T05:29:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.693515+0000", 2024-09-14T05:29:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.693518+0000", 2024-09-14T05:29:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4540000000000001e-06 2024-09-14T05:29:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.710247+0000", 2024-09-14T05:29:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.016729561 2024-09-14T05:29:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.710271+0000", 2024-09-14T05:29:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3646999999999999e-05 2024-09-14T05:29:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.710297+0000", 2024-09-14T05:29:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6030999999999999e-05 2024-09-14T05:29:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.710364+0000", 2024-09-14T05:29:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6958000000000005e-05 2024-09-14T05:29:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.710383+0000", 2024-09-14T05:29:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9638000000000001e-05 2024-09-14T05:29:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.737475+0000", 2024-09-14T05:29:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027092082999999999 2024-09-14T05:29:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.737537+0000", 2024-09-14T05:29:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1622000000000001e-05 2024-09-14T05:29:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21004 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 10512384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.897014+0000", 2024-09-14T05:29:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.93894595199999997, 2024-09-14T05:29:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026844488999999999, 2024-09-14T05:29:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21004 2024-09-14T05:29:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897014+0000", 2024-09-14T05:29:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897014+0000", 2024-09-14T05:29:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897014+0000", 2024-09-14T05:29:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7100000000000002e-07 2024-09-14T05:29:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897017+0000", 2024-09-14T05:29:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2759999999999999e-06 2024-09-14T05:29:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897023+0000", 2024-09-14T05:29:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3969999999999999e-06 2024-09-14T05:29:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897027+0000", 2024-09-14T05:29:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2490000000000003e-06 2024-09-14T05:29:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.921576+0000", 2024-09-14T05:29:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024548803000000001 2024-09-14T05:29:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.921643+0000", 2024-09-14T05:29:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7117000000000001e-05 2024-09-14T05:29:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.923858+0000", 2024-09-14T05:29:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0022151760000000001 2024-09-14T05:29:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21006 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 10565632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.897051+0000", 2024-09-14T05:29:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.93890873600000002, 2024-09-14T05:29:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025048432999999998, 2024-09-14T05:29:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.848 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.848 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21006 2024-09-14T05:29:45.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.848 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897051+0000", 2024-09-14T05:29:45.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897051+0000", 2024-09-14T05:29:45.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897051+0000", 2024-09-14T05:29:45.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0900000000000002e-07 2024-09-14T05:29:45.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897053+0000", 2024-09-14T05:29:45.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.73e-06 2024-09-14T05:29:45.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897059+0000", 2024-09-14T05:29:45.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7389999999999996e-06 2024-09-14T05:29:45.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897063+0000", 2024-09-14T05:29:45.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5059999999999997e-06 2024-09-14T05:29:45.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.921974+0000", 2024-09-14T05:29:45.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024910427999999998 2024-09-14T05:29:45.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.922034+0000", 2024-09-14T05:29:45.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9898000000000001e-05 2024-09-14T05:29:45.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.922099+0000", 2024-09-14T05:29:45.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5722999999999994e-05 2024-09-14T05:29:45.851 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.851 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.851 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.851 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21007 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 10586112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.852 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.897069+0000", 2024-09-14T05:29:45.852 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.93889109900000001, 2024-09-14T05:29:45.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025190937, 2024-09-14T05:29:45.852 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.852 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.852 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.852 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.852 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.852 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.852 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21007 2024-09-14T05:29:45.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.852 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897069+0000", 2024-09-14T05:29:45.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897069+0000", 2024-09-14T05:29:45.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897069+0000", 2024-09-14T05:29:45.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8099999999999998e-07 2024-09-14T05:29:45.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897071+0000", 2024-09-14T05:29:45.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4950000000000001e-06 2024-09-14T05:29:45.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897076+0000", 2024-09-14T05:29:45.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.502e-06 2024-09-14T05:29:45.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897080+0000", 2024-09-14T05:29:45.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3649999999999997e-06 2024-09-14T05:29:45.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.922125+0000", 2024-09-14T05:29:45.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025044172999999999 2024-09-14T05:29:45.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.922190+0000", 2024-09-14T05:29:45.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5655999999999996e-05 2024-09-14T05:29:45.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.922260+0000", 2024-09-14T05:29:45.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9364999999999997e-05 2024-09-14T05:29:45.856 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.856 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.856 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.856 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21008 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 10606592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.856 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.897086+0000", 2024-09-14T05:29:45.856 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.93887411300000001, 2024-09-14T05:29:45.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025342256000000001, 2024-09-14T05:29:45.856 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.857 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.857 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.857 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.857 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.857 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.857 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21008 2024-09-14T05:29:45.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.857 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897086+0000", 2024-09-14T05:29:45.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897086+0000", 2024-09-14T05:29:45.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897086+0000", 2024-09-14T05:29:45.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5400000000000002e-07 2024-09-14T05:29:45.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897103+0000", 2024-09-14T05:29:45.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6566999999999999e-05 2024-09-14T05:29:45.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897109+0000", 2024-09-14T05:29:45.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7730000000000002e-06 2024-09-14T05:29:45.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897114+0000", 2024-09-14T05:29:45.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6280000000000004e-06 2024-09-14T05:29:45.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.922287+0000", 2024-09-14T05:29:45.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025172561999999999 2024-09-14T05:29:45.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.922359+0000", 2024-09-14T05:29:45.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2492999999999999e-05 2024-09-14T05:29:45.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.922428+0000", 2024-09-14T05:29:45.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8879000000000002e-05 2024-09-14T05:29:45.860 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.860 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.861 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.861 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21009 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 10618880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.861 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.897122+0000", 2024-09-14T05:29:45.861 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.93883822800000005, 2024-09-14T05:29:45.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025470848000000001, 2024-09-14T05:29:45.861 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.861 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.861 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.861 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.861 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.862 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.862 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21009 2024-09-14T05:29:45.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.862 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897122+0000", 2024-09-14T05:29:45.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897122+0000", 2024-09-14T05:29:45.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897122+0000", 2024-09-14T05:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.75e-07 2024-09-14T05:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897124+0000", 2024-09-14T05:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.03e-06 2024-09-14T05:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897130+0000", 2024-09-14T05:29:45.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1619999999999996e-06 2024-09-14T05:29:45.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897135+0000", 2024-09-14T05:29:45.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4909999999999999e-06 2024-09-14T05:29:45.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.922450+0000", 2024-09-14T05:29:45.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025315251 2024-09-14T05:29:45.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.922520+0000", 2024-09-14T05:29:45.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0108e-05 2024-09-14T05:29:45.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.922592+0000", 2024-09-14T05:29:45.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2330999999999996e-05 2024-09-14T05:29:45.865 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.865 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.865 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.866 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21010 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 10639360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.866 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.897141+0000", 2024-09-14T05:29:45.866 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.93881840299999997, 2024-09-14T05:29:45.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025611407999999999, 2024-09-14T05:29:45.866 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.866 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.866 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.866 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.866 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.866 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.866 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21010 2024-09-14T05:29:45.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.867 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897141+0000", 2024-09-14T05:29:45.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897141+0000", 2024-09-14T05:29:45.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897142+0000", 2024-09-14T05:29:45.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0600000000000001e-07 2024-09-14T05:29:45.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897144+0000", 2024-09-14T05:29:45.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.858e-06 2024-09-14T05:29:45.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897149+0000", 2024-09-14T05:29:45.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7200000000000003e-06 2024-09-14T05:29:45.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897154+0000", 2024-09-14T05:29:45.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2899999999999996e-06 2024-09-14T05:29:45.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.922615+0000", 2024-09-14T05:29:45.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025461418 2024-09-14T05:29:45.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.922682+0000", 2024-09-14T05:29:45.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7424999999999999e-05 2024-09-14T05:29:45.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.870 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.870 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.922753+0000", 2024-09-14T05:29:45.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0290999999999995e-05 2024-09-14T05:29:45.870 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.870 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.870 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.870 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21011 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 10659840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.870 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.897161+0000", 2024-09-14T05:29:45.870 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.93879921300000002, 2024-09-14T05:29:45.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025761003000000001, 2024-09-14T05:29:45.870 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.871 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.871 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.871 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.871 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.871 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.871 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21011 2024-09-14T05:29:45.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.871 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.871 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.871 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897161+0000", 2024-09-14T05:29:45.871 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.872 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897161+0000", 2024-09-14T05:29:45.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.872 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897161+0000", 2024-09-14T05:29:45.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4000000000000002e-07 2024-09-14T05:29:45.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.872 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897169+0000", 2024-09-14T05:29:45.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9650000000000008e-06 2024-09-14T05:29:45.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.873 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.873 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897175+0000", 2024-09-14T05:29:45.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9359999999999996e-06 2024-09-14T05:29:45.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.873 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.873 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897185+0000", 2024-09-14T05:29:45.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0083e-05 2024-09-14T05:29:45.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.922778+0000", 2024-09-14T05:29:45.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025593281999999998 2024-09-14T05:29:45.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.922851+0000", 2024-09-14T05:29:45.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3038000000000001e-05 2024-09-14T05:29:45.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.922922+0000", 2024-09-14T05:29:45.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0259000000000006e-05 2024-09-14T05:29:45.875 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.875 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.875 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.875 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21012 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 10680320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.875 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.897192+0000", 2024-09-14T05:29:45.875 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.93876810399999999, 2024-09-14T05:29:45.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025884926999999999, 2024-09-14T05:29:45.875 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.875 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.875 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.875 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.875 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.876 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.876 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21012 2024-09-14T05:29:45.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.876 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897192+0000", 2024-09-14T05:29:45.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897192+0000", 2024-09-14T05:29:45.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897192+0000", 2024-09-14T05:29:45.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3099999999999998e-07 2024-09-14T05:29:45.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897194+0000", 2024-09-14T05:29:45.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.984e-06 2024-09-14T05:29:45.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897199+0000", 2024-09-14T05:29:45.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0379999999999999e-06 2024-09-14T05:29:45.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897203+0000", 2024-09-14T05:29:45.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0990000000000001e-06 2024-09-14T05:29:45.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.922946+0000", 2024-09-14T05:29:45.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025742818000000001 2024-09-14T05:29:45.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.923012+0000", 2024-09-14T05:29:45.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6177e-05 2024-09-14T05:29:45.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.923077+0000", 2024-09-14T05:29:45.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4380000000000004e-05 2024-09-14T05:29:45.879 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.879 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.879 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.879 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21013 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 10692608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.880 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.897208+0000", 2024-09-14T05:29:45.880 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.93875150799999996, 2024-09-14T05:29:45.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026057318999999999, 2024-09-14T05:29:45.880 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.880 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.880 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.880 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.880 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.880 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.880 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21013 2024-09-14T05:29:45.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.880 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897208+0000", 2024-09-14T05:29:45.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897208+0000", 2024-09-14T05:29:45.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897209+0000", 2024-09-14T05:29:45.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3700000000000001e-07 2024-09-14T05:29:45.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897210+0000", 2024-09-14T05:29:45.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6360000000000001e-06 2024-09-14T05:29:45.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897215+0000", 2024-09-14T05:29:45.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0050000000000004e-06 2024-09-14T05:29:45.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897219+0000", 2024-09-14T05:29:45.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2860000000000001e-06 2024-09-14T05:29:45.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.923102+0000", 2024-09-14T05:29:45.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.025882018 2024-09-14T05:29:45.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.923167+0000", 2024-09-14T05:29:45.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5545999999999999e-05 2024-09-14T05:29:45.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.923266+0000", 2024-09-14T05:29:45.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8491000000000003e-05 2024-09-14T05:29:45.884 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.884 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.884 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.884 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21014 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 10713088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.884 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.897225+0000", 2024-09-14T05:29:45.884 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.93873437199999998, 2024-09-14T05:29:45.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026227377999999999, 2024-09-14T05:29:45.884 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.884 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.884 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.884 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.885 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.885 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.885 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21014 2024-09-14T05:29:45.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.885 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897225+0000", 2024-09-14T05:29:45.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897225+0000", 2024-09-14T05:29:45.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897226+0000", 2024-09-14T05:29:45.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6899999999999998e-07 2024-09-14T05:29:45.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897228+0000", 2024-09-14T05:29:45.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.824e-06 2024-09-14T05:29:45.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897233+0000", 2024-09-14T05:29:45.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7869999999999998e-06 2024-09-14T05:29:45.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897238+0000", 2024-09-14T05:29:45.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5560000000000001e-06 2024-09-14T05:29:45.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.923289+0000", 2024-09-14T05:29:45.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026051507000000002 2024-09-14T05:29:45.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.923359+0000", 2024-09-14T05:29:45.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9412000000000006e-05 2024-09-14T05:29:45.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.923453+0000", 2024-09-14T05:29:45.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3923000000000002e-05 2024-09-14T05:29:45.888 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.888 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.888 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.888 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21015 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 10733568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.889 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.897243+0000", 2024-09-14T05:29:45.889 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.93871660000000001, 2024-09-14T05:29:45.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026455392000000001, 2024-09-14T05:29:45.889 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.889 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.889 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.889 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.889 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.889 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.889 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21015 2024-09-14T05:29:45.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.889 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897243+0000", 2024-09-14T05:29:45.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897243+0000", 2024-09-14T05:29:45.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897243+0000", 2024-09-14T05:29:45.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.34e-07 2024-09-14T05:29:45.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897245+0000", 2024-09-14T05:29:45.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.522e-06 2024-09-14T05:29:45.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897251+0000", 2024-09-14T05:29:45.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5929999999999996e-06 2024-09-14T05:29:45.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897255+0000", 2024-09-14T05:29:45.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3370000000000001e-06 2024-09-14T05:29:45.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.923478+0000", 2024-09-14T05:29:45.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026223369 2024-09-14T05:29:45.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.923546+0000", 2024-09-14T05:29:45.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7582000000000004e-05 2024-09-14T05:29:45.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.923699+0000", 2024-09-14T05:29:45.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015265499999999999 2024-09-14T05:29:45.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.893 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.893 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21016 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 10766336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.893 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.897260+0000", 2024-09-14T05:29:45.893 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.93869977500000001, 2024-09-14T05:29:45.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026664641999999999, 2024-09-14T05:29:45.894 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.894 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.894 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.894 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21016 2024-09-14T05:29:45.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.894 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897260+0000", 2024-09-14T05:29:45.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897260+0000", 2024-09-14T05:29:45.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897260+0000", 2024-09-14T05:29:45.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5900000000000003e-07 2024-09-14T05:29:45.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897262+0000", 2024-09-14T05:29:45.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4529999999999999e-06 2024-09-14T05:29:45.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897267+0000", 2024-09-14T05:29:45.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1719999999999998e-06 2024-09-14T05:29:45.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897277+0000", 2024-09-14T05:29:45.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0147e-05 2024-09-14T05:29:45.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.923725+0000", 2024-09-14T05:29:45.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026447515000000001 2024-09-14T05:29:45.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.923799+0000", 2024-09-14T05:29:45.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4232000000000003e-05 2024-09-14T05:29:45.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.923925+0000", 2024-09-14T05:29:45.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000125764 2024-09-14T05:29:45.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.898 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.898 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21017 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 10786816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.898 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.897284+0000", 2024-09-14T05:29:45.898 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.93867567200000002, 2024-09-14T05:29:45.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027246563000000001, 2024-09-14T05:29:45.898 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.898 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.898 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.898 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.899 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.899 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.899 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21017 2024-09-14T05:29:45.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.899 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897284+0000", 2024-09-14T05:29:45.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897284+0000", 2024-09-14T05:29:45.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897285+0000", 2024-09-14T05:29:45.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4700000000000002e-07 2024-09-14T05:29:45.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897287+0000", 2024-09-14T05:29:45.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1289999999999999e-06 2024-09-14T05:29:45.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897292+0000", 2024-09-14T05:29:45.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7950000000000004e-06 2024-09-14T05:29:45.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897297+0000", 2024-09-14T05:29:45.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2180000000000001e-06 2024-09-14T05:29:45.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.924173+0000", 2024-09-14T05:29:45.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0268766 2024-09-14T05:29:45.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.924453+0000", 2024-09-14T05:29:45.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00028013400000000003 2024-09-14T05:29:45.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.924531+0000", 2024-09-14T05:29:45.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7239999999999999e-05 2024-09-14T05:29:45.905 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.905 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.905 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.905 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21018 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 10807296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.905 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.897303+0000", 2024-09-14T05:29:45.905 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.938656924, 2024-09-14T05:29:45.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027409388999999999, 2024-09-14T05:29:45.906 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.906 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.906 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:45.906 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.906 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.906 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.906 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21018 2024-09-14T05:29:45.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.906 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897303+0000", 2024-09-14T05:29:45.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897303+0000", 2024-09-14T05:29:45.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897303+0000", 2024-09-14T05:29:45.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1699999999999999e-07 2024-09-14T05:29:45.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897305+0000", 2024-09-14T05:29:45.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8700000000000001e-06 2024-09-14T05:29:45.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897311+0000", 2024-09-14T05:29:45.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9259999999999997e-06 2024-09-14T05:29:45.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.897315+0000", 2024-09-14T05:29:45.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1300000000000003e-06 2024-09-14T05:29:45.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.924561+0000", 2024-09-14T05:29:45.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027245396000000002 2024-09-14T05:29:45.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:45.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.924633+0000", 2024-09-14T05:29:45.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2317999999999994e-05 2024-09-14T05:29:45.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.924712+0000", 2024-09-14T05:29:45.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9332000000000005e-05 2024-09-14T05:29:45.909 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.910 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.910 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.910 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23920 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.910 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.085988+0000", 2024-09-14T05:29:45.910 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.74997159899999999, 2024-09-14T05:29:45.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024930924, 2024-09-14T05:29:45.910 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.910 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.910 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:29:45.910 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.910 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.910 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.910 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23920 2024-09-14T05:29:45.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.911 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085988+0000", 2024-09-14T05:29:45.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085988+0000", 2024-09-14T05:29:45.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085989+0000", 2024-09-14T05:29:45.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4499999999999997e-07 2024-09-14T05:29:45.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085994+0000", 2024-09-14T05:29:45.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6810000000000003e-06 2024-09-14T05:29:45.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086001+0000", 2024-09-14T05:29:45.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9229999999999996e-06 2024-09-14T05:29:45.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086006+0000", 2024-09-14T05:29:45.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1830000000000002e-06 2024-09-14T05:29:45.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.110912+0000", 2024-09-14T05:29:45.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024905809000000001 2024-09-14T05:29:45.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.110919+0000", 2024-09-14T05:29:45.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.883e-06 2024-09-14T05:29:45.914 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.914 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.914 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.914 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23921 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.914 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.086013+0000", 2024-09-14T05:29:45.914 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.74994634599999999, 2024-09-14T05:29:45.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024935789, 2024-09-14T05:29:45.914 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.914 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.914 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:29:45.915 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.915 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.915 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.915 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23921 2024-09-14T05:29:45.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.915 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086013+0000", 2024-09-14T05:29:45.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086013+0000", 2024-09-14T05:29:45.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086014+0000", 2024-09-14T05:29:45.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3599999999999999e-07 2024-09-14T05:29:45.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086016+0000", 2024-09-14T05:29:45.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9609999999999999e-06 2024-09-14T05:29:45.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086022+0000", 2024-09-14T05:29:45.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5919999999999997e-06 2024-09-14T05:29:45.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086027+0000", 2024-09-14T05:29:45.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0420000000000002e-06 2024-09-14T05:29:45.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.110943+0000", 2024-09-14T05:29:45.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024915705 2024-09-14T05:29:45.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.110949+0000", 2024-09-14T05:29:45.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0530000000000003e-06 2024-09-14T05:29:45.918 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.918 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.918 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.918 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23922 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.918 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.086034+0000", 2024-09-14T05:29:45.918 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.74992531500000004, 2024-09-14T05:29:45.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024942755, 2024-09-14T05:29:45.918 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.918 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.918 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:29:45.919 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.919 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.919 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.919 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23922 2024-09-14T05:29:45.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.919 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086034+0000", 2024-09-14T05:29:45.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086034+0000", 2024-09-14T05:29:45.920 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.920 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.920 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086035+0000", 2024-09-14T05:29:45.920 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.75e-07 2024-09-14T05:29:45.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.920 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.920 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086037+0000", 2024-09-14T05:29:45.920 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.013e-06 2024-09-14T05:29:45.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086043+0000", 2024-09-14T05:29:45.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.049e-06 2024-09-14T05:29:45.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086048+0000", 2024-09-14T05:29:45.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.758e-06 2024-09-14T05:29:45.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.110971+0000", 2024-09-14T05:29:45.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024922988 2024-09-14T05:29:45.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.922 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.922 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.110977+0000", 2024-09-14T05:29:45.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.472e-06 2024-09-14T05:29:45.922 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.922 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.922 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.922 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23923 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.922 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.086055+0000", 2024-09-14T05:29:45.922 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.74990494799999996, 2024-09-14T05:29:45.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024954005000000001, 2024-09-14T05:29:45.922 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.922 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.922 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:29:45.922 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.923 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.923 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.923 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23923 2024-09-14T05:29:45.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.923 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086055+0000", 2024-09-14T05:29:45.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086055+0000", 2024-09-14T05:29:45.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.924 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086055+0000", 2024-09-14T05:29:45.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7199999999999999e-07 2024-09-14T05:29:45.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.924 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086057+0000", 2024-09-14T05:29:45.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1749999999999999e-06 2024-09-14T05:29:45.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.924 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086064+0000", 2024-09-14T05:29:45.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5830000000000003e-06 2024-09-14T05:29:45.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086069+0000", 2024-09-14T05:29:45.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8879999999999996e-06 2024-09-14T05:29:45.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.110999+0000", 2024-09-14T05:29:45.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024929973000000001 2024-09-14T05:29:45.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.111009+0000", 2024-09-14T05:29:45.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9140000000000003e-06 2024-09-14T05:29:45.926 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.926 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.926 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.926 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23924 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:45.926 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.086076+0000", 2024-09-14T05:29:45.926 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.74988411600000004, 2024-09-14T05:29:45.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024961806, 2024-09-14T05:29:45.926 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:45.926 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:45.927 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:29:45.927 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:45.927 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:45.927 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:45.927 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23924 2024-09-14T05:29:45.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.927 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:45.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:45.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086076+0000", 2024-09-14T05:29:45.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:45.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086076+0000", 2024-09-14T05:29:45.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:45.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:45.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086076+0000", 2024-09-14T05:29:45.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1100000000000001e-07 2024-09-14T05:29:45.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:45.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086078+0000", 2024-09-14T05:29:45.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9300000000000002e-06 2024-09-14T05:29:45.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:45.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086085+0000", 2024-09-14T05:29:45.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6150000000000002e-06 2024-09-14T05:29:45.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:45.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086090+0000", 2024-09-14T05:29:45.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.22e-06 2024-09-14T05:29:45.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:45.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.111030+0000", 2024-09-14T05:29:45.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.024939731999999999 2024-09-14T05:29:45.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:45.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:45.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:45.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.111037+0000", 2024-09-14T05:29:45.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8979999999999996e-06 2024-09-14T05:29:45.930 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.930 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.930 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.930 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:45.930 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:45.930 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:45.932 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:29:45.932 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:45.931+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:46.033 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:46.032+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:46.055 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:29:46.080 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:46.080 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:46.080 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:29:46.080 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:46.087 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:29:46.133 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:46.132+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:46.163 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T05:29:46.185 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:29:46.186 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:29:46.196 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone14 from 0xeaa2b45 (overlap 0x3fad95a) to 0x0 2024-09-14T05:29:46.222 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:29:46.222 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:29:46.234 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:46.233+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:46.264 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:29:46.264 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:29:46.271 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:46.271 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:46.272 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:46.272 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:46.272 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:46.285 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:29:46.299 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:29:46.300 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:29:46.330 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:29:46.330 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:29:46.335 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:46.335+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:46.360 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:29:46.360 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:29:46.393 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:29:46.394 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:29:46.425 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:29:46.425 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:29:46.437 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:46.436+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:46.457 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:46.457 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:46.469 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:29:46.537 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:46.537+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:46.602 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:46.602 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:46.613 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:29:46.638 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:46.637+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:46.739 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:46.738+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:46.748 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:46.748 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:46.758 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:29:46.840 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:46.839+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:46.893 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:46.893 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:46.904 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:29:46.940 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:46.940+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:47.006 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:47.006 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:47.015 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:29:47.041 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:47.041+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:47.142 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:47.141+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:47.143 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-14T05:29:47.147 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:47.147 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:47.158 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:29:47.244 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:47.243+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:47.292 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:47.292 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:47.304 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:29:47.347 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:47.347+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:47.429 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:47.430 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:47.441 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:29:47.483 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:47.482+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:47.545 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:47.545+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:47.612 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:47.612 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:47.623 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:29:47.646 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:47.645+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:47.747 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:47.747+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:47.799 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:47.799 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:47.810 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:29:47.848 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:47.847+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:47.919 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-14T05:29:47.923 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:47.923 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:47.932 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:29:47.976 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:47.975+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:48.036 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:48.036 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:48.045 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:29:48.050 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:48.049+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:48.119 INFO:teuthology.orchestra.run.smithi071.stdout:1598 read 0x95807ee thru 0x95812ce (0xae1 bytes) 2024-09-14T05:29:48.120 INFO:teuthology.orchestra.run.smithi071.stdout:1599 writesame 0x54211a7 thru 0x542650a data_size 0x12(0x5364 bytes) 2024-09-14T05:29:48.129 INFO:teuthology.orchestra.run.smithi071.stdout:1600 read 0x98d2ef8 thru 0x98dbd40 (0x8e49 bytes) 2024-09-14T05:29:48.129 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:48.129 INFO:teuthology.orchestra.run.smithi071.stdout:1602 read 0x669176 thru 0x673126 (0x9fb1 bytes) 2024-09-14T05:29:48.131 INFO:teuthology.orchestra.run.smithi071.stdout:1603 write 0x9191c04 thru 0x919d208 (0xb605 bytes) 2024-09-14T05:29:48.140 INFO:teuthology.orchestra.run.smithi071.stdout:1604 writesame 0x851c2de thru 0x851d67f data_size 0x7(0x13a2 bytes) 2024-09-14T05:29:48.144 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:48.144 INFO:teuthology.orchestra.run.smithi071.stdout:1607 punch from 0xcfd79a2 to 0xcfdf062, (0x76c0 bytes) 2024-09-14T05:29:48.144 INFO:teuthology.orchestra.run.smithi071.stdout:1608 write 0xa253828 thru 0xa25f58c (0xbd65 bytes) 2024-09-14T05:29:48.151 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:48.150+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:48.153 INFO:teuthology.orchestra.run.smithi071.stdout:1609 read 0x2cfefb0 thru 0x2d091af (0xa200 bytes) 2024-09-14T05:29:48.155 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:48.155 INFO:teuthology.orchestra.run.smithi071.stdout:1611 writesame 0x38fe75a thru 0x390b6ff data_size 0x7(0xcfa6 bytes) 2024-09-14T05:29:48.203 INFO:teuthology.orchestra.run.smithi071.stdout:1612 punch from 0xa059956 to 0xa05ae4d, (0x14f7 bytes) 2024-09-14T05:29:48.203 INFO:teuthology.orchestra.run.smithi071.stdout:1614 punch from 0x7aa1d46 to 0x7aac86c, (0xab26 bytes) 2024-09-14T05:29:48.203 INFO:teuthology.orchestra.run.smithi071.stdout:1615 read 0x282dcd1 thru 0x283c7ab (0xeadb bytes) 2024-09-14T05:29:48.204 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:48.204 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:48.205 INFO:teuthology.orchestra.run.smithi071.stdout:1616 read 0xf85f59 thru 0xf9106a (0xb112 bytes) 2024-09-14T05:29:48.207 INFO:teuthology.orchestra.run.smithi071.stdout:1617 writesame 0x401eabb thru 0x4028d9f data_size 0x11(0xa2e5 bytes) 2024-09-14T05:29:48.216 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:29:48.250 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:48.251 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:48.251+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:48.255 INFO:teuthology.orchestra.run.smithi071.stdout:1620 trunc from 0xeaa2b45 to 0xa6229fa 2024-09-14T05:29:48.271 INFO:teuthology.orchestra.run.smithi071.stdout:1621 write 0x5323b4c thru 0x5327a8c (0x3f41 bytes) 2024-09-14T05:29:48.284 INFO:teuthology.orchestra.run.smithi071.stdout:1622 trunc from 0xa6229fa to 0x30fd51a 2024-09-14T05:29:48.353 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:48.351+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:48.381 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:48.381 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:48.394 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:29:48.410 INFO:teuthology.orchestra.run.smithi071.stdout:1623 read 0x3b262d thru 0x3bd61c (0xaff0 bytes) 2024-09-14T05:29:48.413 INFO:teuthology.orchestra.run.smithi071.stdout:1624 write 0x1725ea7 thru 0x1729795 (0x38ef bytes) 2024-09-14T05:29:48.440 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:29:48.446 INFO:teuthology.orchestra.run.smithi071.stdout:1626 trunc from 0x30fd51a to 0x346a5a0 2024-09-14T05:29:48.454 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:48.452+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:48.465 INFO:teuthology.orchestra.run.smithi071.stdout:1628 writesame 0x9b213a6 thru 0x9b2e728 data_size 0x1(0xd383 bytes) 2024-09-14T05:29:48.465 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:48.465 INFO:teuthology.orchestra.run.smithi071.stdout:1632 read 0x7fd9721 thru 0x7fe38ee (0xa1ce bytes) 2024-09-14T05:29:48.465 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:48.465 INFO:teuthology.orchestra.run.smithi071.stdout:1635 writesame 0x4507f01 thru 0x450d6a4 data_size 0x4(0x57a4 bytes) 2024-09-14T05:29:48.470 INFO:teuthology.orchestra.run.smithi071.stdout:1636 write 0x281d2aa thru 0x2826723 (0x947a bytes) 2024-09-14T05:29:48.529 INFO:teuthology.orchestra.run.smithi071.stdout:1637 punch from 0x477224f to 0x4775c9f, (0x3a50 bytes) 2024-09-14T05:29:48.529 INFO:teuthology.orchestra.run.smithi071.stdout:1638 write 0x1c7a363 thru 0x1c8a1c5 (0xfe63 bytes) 2024-09-14T05:29:48.554 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:48.554+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:48.572 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:48.572 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:48.585 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:29:48.589 INFO:teuthology.orchestra.run.smithi071.stdout:1639 punch from 0x4dce4f to 0x4e1c3d, (0x4dee bytes) 2024-09-14T05:29:48.596 INFO:teuthology.orchestra.run.smithi071.stdout:1640 write 0xade3853 thru 0xade942b (0x5bd9 bytes) 2024-09-14T05:29:48.605 INFO:teuthology.orchestra.run.smithi071.stdout:1641 writesame 0x8463459 thru 0x846d8f0 data_size 0x5c(0xa498 bytes) 2024-09-14T05:29:48.613 INFO:teuthology.orchestra.run.smithi071.stdout:1642 read 0x9f4c90d thru 0x9f5b063 (0xe757 bytes) 2024-09-14T05:29:48.614 INFO:teuthology.orchestra.run.smithi071.stdout:1644 write 0x5f4a343 thru 0x5f4e412 (0x40d0 bytes) 2024-09-14T05:29:48.614 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:29:48.614 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:29:48.618 INFO:teuthology.orchestra.run.smithi071.stdout:1646 read 0x2ed2c69 thru 0x2edba12 (0x8daa bytes) 2024-09-14T05:29:48.619 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:48.619 INFO:teuthology.orchestra.run.smithi071.stdout:1648 write 0x9435585 thru 0x9441e4b (0xc8c7 bytes) 2024-09-14T05:29:48.633 INFO:teuthology.orchestra.run.smithi071.stdout:1649 write 0xd19a4af thru 0xd1a76ca (0xd21c bytes) 2024-09-14T05:29:48.634 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:29:48.634 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:29:48.642 INFO:teuthology.orchestra.run.smithi071.stdout:1651 writesame 0x8819a29 thru 0x88275ec data_size 0x91(0xdbc4 bytes) 2024-09-14T05:29:48.648 INFO:teuthology.orchestra.run.smithi071.stdout:1652 write 0xa64e50 thru 0xa6d8e4 (0x8a95 bytes) 2024-09-14T05:29:48.656 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:48.654+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:48.665 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:29:48.665 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:29:48.681 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:48.682 INFO:teuthology.orchestra.run.smithi071.stdout:1654 punch from 0x6c66eb1 to 0x6c750db, (0xe22a bytes) 2024-09-14T05:29:48.682 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:48.682 INFO:teuthology.orchestra.run.smithi071.stdout:1656 read 0x2618049 thru 0x261e8f9 (0x68b1 bytes) 2024-09-14T05:29:48.683 INFO:teuthology.orchestra.run.smithi071.stdout:1657 write 0xad62ac0 thru 0xad70034 (0xd575 bytes) 2024-09-14T05:29:48.687 INFO:teuthology.orchestra.run.smithi071.stdout:1658 writesame 0x8b47046 thru 0x8b48220 data_size 0x1(0x11db bytes) 2024-09-14T05:29:48.690 INFO:teuthology.orchestra.run.smithi071.stdout:1659 read 0x2588eed thru 0x2592d31 (0x9e45 bytes) 2024-09-14T05:29:48.692 INFO:teuthology.orchestra.run.smithi071.stdout:1660 writesame 0xbad8430 thru 0xbae1dba data_size 0x1(0x998b bytes) 2024-09-14T05:29:48.698 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:29:48.698 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:29:48.704 INFO:teuthology.orchestra.run.smithi071.stdout:1662 read 0x2eb4a79 thru 0x2eb92ef (0x4877 bytes) 2024-09-14T05:29:48.705 INFO:teuthology.orchestra.run.smithi071.stdout:1663 punch from 0x62e7307 to 0x62f51f0, (0xdee9 bytes) 2024-09-14T05:29:48.705 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:48.705 INFO:teuthology.orchestra.run.smithi071.stdout:1665 read 0x8225fbb thru 0x822e4d3 (0x8519 bytes) 2024-09-14T05:29:48.705 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:48.705 INFO:teuthology.orchestra.run.smithi071.stdout:1667 punch from 0x87eb2db to 0x87f6b84, (0xb8a9 bytes) 2024-09-14T05:29:48.705 INFO:teuthology.orchestra.run.smithi071.stdout:1668 write 0x14d1b33 thru 0x14d7fa3 (0x6471 bytes) 2024-09-14T05:29:48.723 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:48.724 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:48.724 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:48.724 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:48.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.724 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:2130 3.1b e30/24 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:head v 30'21, mlcod=30'19)", 2024-09-14T05:29:48.725 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.782900+0000", 2024-09-14T05:29:48.725 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.938420945000001, 2024-09-14T05:29:48.725 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.034589779000000001, 2024-09-14T05:29:48.725 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.725 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.725 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:48.726 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.726 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.726 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.782900+0000", 2024-09-14T05:29:48.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.782900+0000", 2024-09-14T05:29:48.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.782903+0000", 2024-09-14T05:29:48.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6800000000000002e-06 2024-09-14T05:29:48.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.729 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.729 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.802899+0000", 2024-09-14T05:29:48.729 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.019996435999999999 2024-09-14T05:29:48.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.729 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.802924+0000", 2024-09-14T05:29:48.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.463e-05 2024-09-14T05:29:48.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.802934+0000", 2024-09-14T05:29:48.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8800000000000003e-06 2024-09-14T05:29:48.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.803003+0000", 2024-09-14T05:29:48.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9158000000000004e-05 2024-09-14T05:29:48.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.803029+0000", 2024-09-14T05:29:48.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6395999999999999e-05 2024-09-14T05:29:48.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:48.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.817445+0000", 2024-09-14T05:29:48.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.014415258 2024-09-14T05:29:48.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.732 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.817490+0000", 2024-09-14T05:29:48.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5340999999999998e-05 2024-09-14T05:29:48.732 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.732 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.732 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.732 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:2132 3.1b e30/24 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:head v 30'22, mlcod=30'19)", 2024-09-14T05:29:48.732 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.802961+0000", 2024-09-14T05:29:48.732 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 17.918360020000001, 2024-09-14T05:29:48.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026740369, 2024-09-14T05:29:48.732 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.732 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.733 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:48.733 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.802961+0000", 2024-09-14T05:29:48.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.802961+0000", 2024-09-14T05:29:48.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.734 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.802962+0000", 2024-09-14T05:29:48.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7999999999999993e-07 2024-09-14T05:29:48.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.734 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.818597+0000", 2024-09-14T05:29:48.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.015634577 2024-09-14T05:29:48.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.734 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.818609+0000", 2024-09-14T05:29:48.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2669e-05 2024-09-14T05:29:48.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.818617+0000", 2024-09-14T05:29:48.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1620000000000002e-06 2024-09-14T05:29:48.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.818663+0000", 2024-09-14T05:29:48.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6137999999999998e-05 2024-09-14T05:29:48.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.818681+0000", 2024-09-14T05:29:48.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8726e-05 2024-09-14T05:29:48.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:48.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.829654+0000", 2024-09-14T05:29:48.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.010972986000000001 2024-09-14T05:29:48.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.829702+0000", 2024-09-14T05:29:48.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7131e-05 2024-09-14T05:29:48.737 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.737 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.737 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.737 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:20614 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 10805248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:48.737 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.887390+0000", 2024-09-14T05:29:48.737 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.8339315780000001, 2024-09-14T05:29:48.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026905603, 2024-09-14T05:29:48.737 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.737 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.737 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:48.737 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:48.738 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:48.738 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:48.738 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20614 2024-09-14T05:29:48.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.738 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.738 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.738 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.887390+0000", 2024-09-14T05:29:48.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.738 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.738 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.887390+0000", 2024-09-14T05:29:48.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.887390+0000", 2024-09-14T05:29:48.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5499999999999998e-07 2024-09-14T05:29:48.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.887392+0000", 2024-09-14T05:29:48.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9549999999999999e-06 2024-09-14T05:29:48.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.740 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.887399+0000", 2024-09-14T05:29:48.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5289999999999997e-06 2024-09-14T05:29:48.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.740 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.740 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.887403+0000", 2024-09-14T05:29:48.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7790000000000003e-06 2024-09-14T05:29:48.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.740 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.740 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.909241+0000", 2024-09-14T05:29:48.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.021837918000000001 2024-09-14T05:29:48.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.741 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.741 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.909310+0000", 2024-09-14T05:29:48.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8450999999999999e-05 2024-09-14T05:29:48.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.741 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.741 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.914295+0000", 2024-09-14T05:29:48.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0049855159999999997 2024-09-14T05:29:48.741 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.741 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.741 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.742 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:22586 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2611200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:48.742 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.987656+0000", 2024-09-14T05:29:48.742 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.733665577, 2024-09-14T05:29:48.742 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026838026000000001, 2024-09-14T05:29:48.742 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.742 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.742 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:48.742 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:48.742 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:48.742 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:48.742 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 22586 2024-09-14T05:29:48.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.743 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.987656+0000", 2024-09-14T05:29:48.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.744 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.987656+0000", 2024-09-14T05:29:48.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.744 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.987656+0000", 2024-09-14T05:29:48.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:29:48.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.987658+0000", 2024-09-14T05:29:48.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.751e-06 2024-09-14T05:29:48.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.745 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.987664+0000", 2024-09-14T05:29:48.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7749999999999998e-06 2024-09-14T05:29:48.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.745 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.987668+0000", 2024-09-14T05:29:48.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9550000000000002e-06 2024-09-14T05:29:48.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.746 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.746 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.014327+0000", 2024-09-14T05:29:48.746 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026659414999999999 2024-09-14T05:29:48.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.746 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.746 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.014402+0000", 2024-09-14T05:29:48.746 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4711999999999999e-05 2024-09-14T05:29:48.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.746 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.746 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.014494+0000", 2024-09-14T05:29:48.747 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2057000000000003e-05 2024-09-14T05:29:48.747 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.747 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.747 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.747 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23893 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 12781568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:48.747 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.085796+0000", 2024-09-14T05:29:48.748 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.6355251580000001, 2024-09-14T05:29:48.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026899266000000002, 2024-09-14T05:29:48.748 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.748 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.748 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:48.748 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:48.749 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:48.749 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:48.749 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23893 2024-09-14T05:29:48.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.749 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085796+0000", 2024-09-14T05:29:48.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085796+0000", 2024-09-14T05:29:48.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085796+0000", 2024-09-14T05:29:48.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.96e-07 2024-09-14T05:29:48.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085798+0000", 2024-09-14T05:29:48.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.903e-06 2024-09-14T05:29:48.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.751 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.751 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085807+0000", 2024-09-14T05:29:48.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1109999999999997e-06 2024-09-14T05:29:48.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085813+0000", 2024-09-14T05:29:48.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2549999999999997e-06 2024-09-14T05:29:48.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.112563+0000", 2024-09-14T05:29:48.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.02675015 2024-09-14T05:29:48.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.112631+0000", 2024-09-14T05:29:48.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7980999999999998e-05 2024-09-14T05:29:48.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.112695+0000", 2024-09-14T05:29:48.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4469999999999998e-05 2024-09-14T05:29:48.754 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.754 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.755 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.755 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23894 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 12802048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:48.755 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.085820+0000", 2024-09-14T05:29:48.755 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.6355016529999999, 2024-09-14T05:29:48.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027071665000000002, 2024-09-14T05:29:48.756 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.756 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.756 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:48.756 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:48.756 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:48.757 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:48.757 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23894 2024-09-14T05:29:48.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.757 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085820+0000", 2024-09-14T05:29:48.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085820+0000", 2024-09-14T05:29:48.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085820+0000", 2024-09-14T05:29:48.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4400000000000001e-07 2024-09-14T05:29:48.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085822+0000", 2024-09-14T05:29:48.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9089999999999998e-06 2024-09-14T05:29:48.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085831+0000", 2024-09-14T05:29:48.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0909999999999999e-06 2024-09-14T05:29:48.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.761 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.761 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085836+0000", 2024-09-14T05:29:48.761 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2460000000000003e-06 2024-09-14T05:29:48.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.761 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.112725+0000", 2024-09-14T05:29:48.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026888351000000001 2024-09-14T05:29:48.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.112788+0000", 2024-09-14T05:29:48.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3396000000000001e-05 2024-09-14T05:29:48.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.112891+0000", 2024-09-14T05:29:48.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010322799999999999 2024-09-14T05:29:48.763 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.763 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.763 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.764 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23895 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 12822528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:48.764 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.085846+0000", 2024-09-14T05:29:48.764 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.6354748529999998, 2024-09-14T05:29:48.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027236515999999999, 2024-09-14T05:29:48.764 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.765 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.765 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:48.765 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:48.765 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:48.765 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:48.765 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23895 2024-09-14T05:29:48.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.765 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085846+0000", 2024-09-14T05:29:48.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085846+0000", 2024-09-14T05:29:48.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085847+0000", 2024-09-14T05:29:48.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.75e-07 2024-09-14T05:29:48.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085859+0000", 2024-09-14T05:29:48.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1911e-05 2024-09-14T05:29:48.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085869+0000", 2024-09-14T05:29:48.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0427e-05 2024-09-14T05:29:48.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085875+0000", 2024-09-14T05:29:48.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4110000000000004e-06 2024-09-14T05:29:48.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.112920+0000", 2024-09-14T05:29:48.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027045446000000001 2024-09-14T05:29:48.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.112987+0000", 2024-09-14T05:29:48.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7047999999999999e-05 2024-09-14T05:29:48.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.769 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.769 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.113083+0000", 2024-09-14T05:29:48.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5798000000000007e-05 2024-09-14T05:29:48.769 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.769 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.769 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.769 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23896 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 12843008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:48.769 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.085886+0000", 2024-09-14T05:29:48.769 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.635434981, 2024-09-14T05:29:48.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027380866, 2024-09-14T05:29:48.770 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.770 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.770 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:48.770 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:48.770 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:48.770 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:48.770 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23896 2024-09-14T05:29:48.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.770 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.770 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.770 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085886+0000", 2024-09-14T05:29:48.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085886+0000", 2024-09-14T05:29:48.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085887+0000", 2024-09-14T05:29:48.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8299999999999997e-07 2024-09-14T05:29:48.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085889+0000", 2024-09-14T05:29:48.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8840000000000001e-06 2024-09-14T05:29:48.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085899+0000", 2024-09-14T05:29:48.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0326000000000001e-05 2024-09-14T05:29:48.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085904+0000", 2024-09-14T05:29:48.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3959999999999997e-06 2024-09-14T05:29:48.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.113111+0000", 2024-09-14T05:29:48.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027206563999999999 2024-09-14T05:29:48.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.113176+0000", 2024-09-14T05:29:48.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4707000000000002e-05 2024-09-14T05:29:48.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.774 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.113267+0000", 2024-09-14T05:29:48.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1506000000000001e-05 2024-09-14T05:29:48.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.774 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.774 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23897 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 12855296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:48.774 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.085925+0000", 2024-09-14T05:29:48.775 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.6353963650000001, 2024-09-14T05:29:48.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027529789999999998, 2024-09-14T05:29:48.775 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.775 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.775 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:48.775 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:48.775 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:48.775 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:48.775 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23897 2024-09-14T05:29:48.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.775 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085925+0000", 2024-09-14T05:29:48.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085925+0000", 2024-09-14T05:29:48.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085925+0000", 2024-09-14T05:29:48.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4300000000000003e-07 2024-09-14T05:29:48.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085928+0000", 2024-09-14T05:29:48.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4540000000000001e-06 2024-09-14T05:29:48.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085938+0000", 2024-09-14T05:29:48.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0203e-05 2024-09-14T05:29:48.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085944+0000", 2024-09-14T05:29:48.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4829999999999999e-06 2024-09-14T05:29:48.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.113299+0000", 2024-09-14T05:29:48.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027354976 2024-09-14T05:29:48.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.113362+0000", 2024-09-14T05:29:48.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3769000000000006e-05 2024-09-14T05:29:48.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.113455+0000", 2024-09-14T05:29:48.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2361999999999994e-05 2024-09-14T05:29:48.779 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.779 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.779 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.779 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23898 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 12875776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:48.779 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.085951+0000", 2024-09-14T05:29:48.779 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.6353705239999998, 2024-09-14T05:29:48.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027690184999999999, 2024-09-14T05:29:48.780 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.780 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.780 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:48.780 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:48.780 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:48.780 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:48.780 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23898 2024-09-14T05:29:48.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.780 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.780 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085951+0000", 2024-09-14T05:29:48.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085951+0000", 2024-09-14T05:29:48.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085951+0000", 2024-09-14T05:29:48.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.63e-07 2024-09-14T05:29:48.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085953+0000", 2024-09-14T05:29:48.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9690000000000001e-06 2024-09-14T05:29:48.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085962+0000", 2024-09-14T05:29:48.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1859999999999998e-06 2024-09-14T05:29:48.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085967+0000", 2024-09-14T05:29:48.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1529999999999996e-06 2024-09-14T05:29:48.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.113483+0000", 2024-09-14T05:29:48.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027515128 2024-09-14T05:29:48.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.113549+0000", 2024-09-14T05:29:48.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6476000000000005e-05 2024-09-14T05:29:48.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.113641+0000", 2024-09-14T05:29:48.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1810000000000004e-05 2024-09-14T05:29:48.784 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.784 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.784 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.784 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23899 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 12896256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:48.784 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.085975+0000", 2024-09-14T05:29:48.784 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.635346025, 2024-09-14T05:29:48.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027857347000000001, 2024-09-14T05:29:48.784 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.784 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.784 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:48.785 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:48.785 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:48.785 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:48.785 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23899 2024-09-14T05:29:48.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.785 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085975+0000", 2024-09-14T05:29:48.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085975+0000", 2024-09-14T05:29:48.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085976+0000", 2024-09-14T05:29:48.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1600000000000002e-07 2024-09-14T05:29:48.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085978+0000", 2024-09-14T05:29:48.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.035e-06 2024-09-14T05:29:48.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085987+0000", 2024-09-14T05:29:48.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8999999999999995e-06 2024-09-14T05:29:48.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085991+0000", 2024-09-14T05:29:48.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9219999999999996e-06 2024-09-14T05:29:48.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.113670+0000", 2024-09-14T05:29:48.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027678991999999999 2024-09-14T05:29:48.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.113738+0000", 2024-09-14T05:29:48.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7457000000000001e-05 2024-09-14T05:29:48.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.113833+0000", 2024-09-14T05:29:48.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4624999999999996e-05 2024-09-14T05:29:48.788 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.788 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.788 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.789 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23900 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 12929024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:48.789 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.085999+0000", 2024-09-14T05:29:48.789 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.6353219709999998, 2024-09-14T05:29:48.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.028042719000000001, 2024-09-14T05:29:48.789 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.789 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.789 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:48.789 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:48.789 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:48.789 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:48.790 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23900 2024-09-14T05:29:48.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.790 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085999+0000", 2024-09-14T05:29:48.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.085999+0000", 2024-09-14T05:29:48.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086000+0000", 2024-09-14T05:29:48.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4900000000000001e-07 2024-09-14T05:29:48.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086001+0000", 2024-09-14T05:29:48.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7630000000000001e-06 2024-09-14T05:29:48.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086011+0000", 2024-09-14T05:29:48.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3039999999999994e-06 2024-09-14T05:29:48.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086016+0000", 2024-09-14T05:29:48.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0340000000000004e-06 2024-09-14T05:29:48.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.113860+0000", 2024-09-14T05:29:48.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027843948 2024-09-14T05:29:48.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.113931+0000", 2024-09-14T05:29:48.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1699999999999995e-05 2024-09-14T05:29:48.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.114042+0000", 2024-09-14T05:29:48.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011052099999999999 2024-09-14T05:29:48.793 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.793 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.793 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.794 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23901 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 12949504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:48.794 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.086023+0000", 2024-09-14T05:29:48.794 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.635298347, 2024-09-14T05:29:48.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.028214295, 2024-09-14T05:29:48.794 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.794 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.794 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:48.794 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:48.794 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:48.794 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:48.794 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23901 2024-09-14T05:29:48.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.795 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086023+0000", 2024-09-14T05:29:48.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086023+0000", 2024-09-14T05:29:48.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086023+0000", 2024-09-14T05:29:48.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2399999999999999e-07 2024-09-14T05:29:48.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086025+0000", 2024-09-14T05:29:48.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9379999999999999e-06 2024-09-14T05:29:48.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086034+0000", 2024-09-14T05:29:48.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8610000000000004e-06 2024-09-14T05:29:48.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086047+0000", 2024-09-14T05:29:48.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.322e-05 2024-09-14T05:29:48.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.114071+0000", 2024-09-14T05:29:48.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.028023329999999999 2024-09-14T05:29:48.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.114137+0000", 2024-09-14T05:29:48.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6647000000000001e-05 2024-09-14T05:29:48.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.114237+0000", 2024-09-14T05:29:48.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9875000000000002e-05 2024-09-14T05:29:48.798 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.798 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.798 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.798 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23902 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 12969984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:48.798 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.086055+0000", 2024-09-14T05:29:48.798 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.635266434, 2024-09-14T05:29:48.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.028450738999999999, 2024-09-14T05:29:48.799 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.799 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.799 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:48.799 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:48.799 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:48.799 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:48.799 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23902 2024-09-14T05:29:48.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.799 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086055+0000", 2024-09-14T05:29:48.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086055+0000", 2024-09-14T05:29:48.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.800 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.800 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086055+0000", 2024-09-14T05:29:48.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0900000000000002e-07 2024-09-14T05:29:48.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086057+0000", 2024-09-14T05:29:48.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.075e-06 2024-09-14T05:29:48.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086067+0000", 2024-09-14T05:29:48.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5840000000000001e-06 2024-09-14T05:29:48.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086072+0000", 2024-09-14T05:29:48.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3040000000000004e-06 2024-09-14T05:29:48.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.114265+0000", 2024-09-14T05:29:48.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.028193187000000001 2024-09-14T05:29:48.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.114399+0000", 2024-09-14T05:29:48.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000133363 2024-09-14T05:29:48.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.114506+0000", 2024-09-14T05:29:48.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000106717 2024-09-14T05:29:48.803 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.803 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.803 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.803 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23903 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 12982272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:48.803 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.086079+0000", 2024-09-14T05:29:48.803 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.6352420680000002, 2024-09-14T05:29:48.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.028626242999999999, 2024-09-14T05:29:48.804 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.804 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.804 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:48.804 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:48.804 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:48.804 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:48.804 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23903 2024-09-14T05:29:48.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.804 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086079+0000", 2024-09-14T05:29:48.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086079+0000", 2024-09-14T05:29:48.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086080+0000", 2024-09-14T05:29:48.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3000000000000001e-07 2024-09-14T05:29:48.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086082+0000", 2024-09-14T05:29:48.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9259999999999999e-06 2024-09-14T05:29:48.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086090+0000", 2024-09-14T05:29:48.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8980000000000002e-06 2024-09-14T05:29:48.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086096+0000", 2024-09-14T05:29:48.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2989999999999996e-06 2024-09-14T05:29:48.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.114535+0000", 2024-09-14T05:29:48.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.028438952999999999 2024-09-14T05:29:48.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.114612+0000", 2024-09-14T05:29:48.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7589999999999994e-05 2024-09-14T05:29:48.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.114705+0000", 2024-09-14T05:29:48.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3146999999999994e-05 2024-09-14T05:29:48.808 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.808 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.808 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.808 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23904 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 13002752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:48.808 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.086102+0000", 2024-09-14T05:29:48.808 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.6352189660000001, 2024-09-14T05:29:48.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.028794, 2024-09-14T05:29:48.808 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.808 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.808 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:48.809 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:48.809 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:48.809 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:48.809 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23904 2024-09-14T05:29:48.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.809 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086102+0000", 2024-09-14T05:29:48.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086102+0000", 2024-09-14T05:29:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086103+0000", 2024-09-14T05:29:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0900000000000002e-07 2024-09-14T05:29:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086105+0000", 2024-09-14T05:29:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0310000000000001e-06 2024-09-14T05:29:48.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086114+0000", 2024-09-14T05:29:48.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8400000000000001e-06 2024-09-14T05:29:48.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086120+0000", 2024-09-14T05:29:48.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9680000000000003e-06 2024-09-14T05:29:48.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.114733+0000", 2024-09-14T05:29:48.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.028613651 2024-09-14T05:29:48.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.114797+0000", 2024-09-14T05:29:48.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3422000000000004e-05 2024-09-14T05:29:48.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.114896+0000", 2024-09-14T05:29:48.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9579000000000003e-05 2024-09-14T05:29:48.815 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.816 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.816 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.816 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23905 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 13023232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:48.817 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.086129+0000", 2024-09-14T05:29:48.817 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.635192183, 2024-09-14T05:29:48.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029112118999999999, 2024-09-14T05:29:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23905 2024-09-14T05:29:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086129+0000", 2024-09-14T05:29:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086129+0000", 2024-09-14T05:29:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086130+0000", 2024-09-14T05:29:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1200000000000003e-07 2024-09-14T05:29:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086132+0000", 2024-09-14T05:29:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9690000000000001e-06 2024-09-14T05:29:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086138+0000", 2024-09-14T05:29:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8539999999999999e-06 2024-09-14T05:29:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086146+0000", 2024-09-14T05:29:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6660000000000008e-06 2024-09-14T05:29:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.114925+0000", 2024-09-14T05:29:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.028778860999999999 2024-09-14T05:29:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.114991+0000", 2024-09-14T05:29:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6162999999999997e-05 2024-09-14T05:29:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.115241+0000", 2024-09-14T05:29:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025009400000000002 2024-09-14T05:29:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23906 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 13056000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.086153+0000", 2024-09-14T05:29:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.6351679240000001, 2024-09-14T05:29:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029320481999999998, 2024-09-14T05:29:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23906 2024-09-14T05:29:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086153+0000", 2024-09-14T05:29:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086153+0000", 2024-09-14T05:29:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086154+0000", 2024-09-14T05:29:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.08e-07 2024-09-14T05:29:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086159+0000", 2024-09-14T05:29:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3990000000000003e-06 2024-09-14T05:29:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086168+0000", 2024-09-14T05:29:48.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.0040000000000005e-06 2024-09-14T05:29:48.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086174+0000", 2024-09-14T05:29:48.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.066e-06 2024-09-14T05:29:48.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.115270+0000", 2024-09-14T05:29:48.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029096101999999999 2024-09-14T05:29:48.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.115340+0000", 2024-09-14T05:29:48.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9219000000000003e-05 2024-09-14T05:29:48.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.115474+0000", 2024-09-14T05:29:48.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000134284 2024-09-14T05:29:48.829 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.829 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.829 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.829 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23907 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 13076480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:48.829 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.086184+0000", 2024-09-14T05:29:48.829 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.6351371019999998, 2024-09-14T05:29:48.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029492628999999999, 2024-09-14T05:29:48.829 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.829 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.829 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:48.829 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:48.830 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:48.830 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:48.830 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23907 2024-09-14T05:29:48.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.830 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086184+0000", 2024-09-14T05:29:48.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086184+0000", 2024-09-14T05:29:48.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086185+0000", 2024-09-14T05:29:48.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6899999999999998e-07 2024-09-14T05:29:48.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086186+0000", 2024-09-14T05:29:48.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.875e-06 2024-09-14T05:29:48.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086195+0000", 2024-09-14T05:29:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5250000000000005e-06 2024-09-14T05:29:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086201+0000", 2024-09-14T05:29:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.604e-06 2024-09-14T05:29:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.115503+0000", 2024-09-14T05:29:48.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029301884 2024-09-14T05:29:48.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.115574+0000", 2024-09-14T05:29:48.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1465999999999996e-05 2024-09-14T05:29:48.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.115677+0000", 2024-09-14T05:29:48.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000102806 2024-09-14T05:29:48.834 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.834 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.834 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.834 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:23908 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 13096960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:48.834 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:45.086210+0000", 2024-09-14T05:29:48.834 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 3.6351114730000003, 2024-09-14T05:29:48.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029658842000000001, 2024-09-14T05:29:48.834 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:48.834 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:48.834 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:48.834 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:48.834 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:48.835 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:48.835 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 23908 2024-09-14T05:29:48.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.835 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:48.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:48.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086210+0000", 2024-09-14T05:29:48.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:48.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086210+0000", 2024-09-14T05:29:48.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:48.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:48.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086210+0000", 2024-09-14T05:29:48.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1899999999999998e-07 2024-09-14T05:29:48.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:48.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086212+0000", 2024-09-14T05:29:48.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1770000000000001e-06 2024-09-14T05:29:48.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:48.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086221+0000", 2024-09-14T05:29:48.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6780000000000006e-06 2024-09-14T05:29:48.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:48.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.086226+0000", 2024-09-14T05:29:48.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2989999999999996e-06 2024-09-14T05:29:48.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:48.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.115705+0000", 2024-09-14T05:29:48.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.02947874 2024-09-14T05:29:48.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:48.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.115771+0000", 2024-09-14T05:29:48.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5438000000000006e-05 2024-09-14T05:29:48.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:48.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:48.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:48.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:45.115869+0000", 2024-09-14T05:29:48.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8091000000000007e-05 2024-09-14T05:29:48.840 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.840 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.840 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.840 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:48.840 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:48.840 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:48.842 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:29:48.842 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:29:48.843 INFO:teuthology.orchestra.run.smithi071.stdout:1669 read 0x4b5ac1e thru 0x4b68cb0 (0xe093 bytes) 2024-09-14T05:29:48.843 INFO:teuthology.orchestra.run.smithi071.stdout:1670 read 0xb22fd68 thru 0xb2360dd (0x6376 bytes) 2024-09-14T05:29:48.844 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:48.844 INFO:teuthology.orchestra.run.smithi071.stdout:1674 punch from 0x599010f to 0x599e7c5, (0xe6b6 bytes) 2024-09-14T05:29:48.846 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:29:48.846 INFO:teuthology.orchestra.run.smithi071.stdout:1675 write 0xd56c711 thru 0xd574fb6 (0x88a6 bytes) 2024-09-14T05:29:48.847 INFO:teuthology.orchestra.run.smithi071.stdout:1677 write 0x67236ef thru 0x672375a (0x6c bytes) 2024-09-14T05:29:48.847 INFO:teuthology.orchestra.run.smithi071.stdout:1678 read 0x2aea560 thru 0x2af5258 (0xacf9 bytes) 2024-09-14T05:29:48.847 INFO:teuthology.orchestra.run.smithi071.stdout:1679 read 0x3988325 thru 0x3996257 (0xdf33 bytes) 2024-09-14T05:29:48.847 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:29:48.847 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:29:48.848 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:48.848 INFO:teuthology.orchestra.run.smithi071.stdout:1681 read 0xc15af22 thru 0xc15c2d5 (0x13b4 bytes) 2024-09-14T05:29:48.848 INFO:teuthology.orchestra.run.smithi071.stdout:1682 punch from 0x41981e4 to 0x419bceb, (0x3b07 bytes) 2024-09-14T05:29:48.848 INFO:teuthology.orchestra.run.smithi071.stdout:1684 write 0xa03ce5e thru 0xa0484fa (0xb69d bytes) 2024-09-14T05:29:48.848 INFO:teuthology.orchestra.run.smithi071.stdout:1685 read 0xd3e7602 thru 0xd3f22b1 (0xacb0 bytes) 2024-09-14T05:29:48.849 INFO:teuthology.orchestra.run.smithi071.stdout:1686 read 0x28880fb thru 0x288879d (0x6a3 bytes) 2024-09-14T05:29:48.849 INFO:teuthology.orchestra.run.smithi071.stdout:1688 punch from 0x117ab07 to 0x1182b3e, (0x8037 bytes) 2024-09-14T05:29:48.849 INFO:teuthology.orchestra.run.smithi071.stdout:1689 writesame 0x5bc6262 thru 0x5bcf815 data_size 0x86(0x95b4 bytes) 2024-09-14T05:29:48.849 INFO:teuthology.orchestra.run.smithi071.stdout:1690 trunc from 0xd574fb7 to 0x5a6cac9 2024-09-14T05:29:48.849 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:29:48.849 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:29:48.850 INFO:teuthology.orchestra.run.smithi071.stdout:1692 write 0xfb7343 thru 0xfc1d7d (0xaa3b bytes) 2024-09-14T05:29:48.850 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:29:48.851 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:29:48.851 INFO:teuthology.orchestra.run.smithi071.stdout:1693 write 0x8fe2233 thru 0x8ff21ea (0xffb8 bytes) 2024-09-14T05:29:48.851 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:48.840+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:48.852 INFO:teuthology.orchestra.run.smithi071.stdout:1695 write 0xd9f4211 thru 0xd9fcced (0x8add bytes) 2024-09-14T05:29:48.855 INFO:teuthology.orchestra.run.smithi071.stdout:1696 write 0x76853c8 thru 0x7686594 (0x11cd bytes) 2024-09-14T05:29:48.864 INFO:teuthology.orchestra.run.smithi071.stdout:1697 writesame 0xe0ca96d thru 0xe0cdf77 data_size 0x1(0x360b bytes) 2024-09-14T05:29:48.864 INFO:teuthology.orchestra.run.smithi071.stdout:truncating to largest ever: 0xe8a90c7 2024-09-14T05:29:48.864 INFO:teuthology.orchestra.run.smithi071.stdout:1698 trunc from 0xe0cdf78 to 0xe8a90c7 2024-09-14T05:29:48.869 INFO:teuthology.orchestra.run.smithi071.stdout:1699 write 0x9b0c217 thru 0x9b1b053 (0xee3d bytes) 2024-09-14T05:29:48.874 INFO:teuthology.orchestra.run.smithi071.stdout:1700 write 0x840198e thru 0x84111e5 (0xf858 bytes) 2024-09-14T05:29:48.884 INFO:teuthology.orchestra.run.smithi071.stdout:1701 write 0xba2d3f9 thru 0xba3579a (0x83a2 bytes) 2024-09-14T05:29:48.889 INFO:teuthology.orchestra.run.smithi071.stdout:1702 write 0x252a9be thru 0x2536dee (0xc431 bytes) 2024-09-14T05:29:48.934 INFO:teuthology.orchestra.run.smithi071.stdout:1704 read 0x4025d3e thru 0x40338d2 (0xdb95 bytes) 2024-09-14T05:29:48.934 INFO:teuthology.orchestra.run.smithi071.stdout:1705 write 0x9395df4 thru 0x939c6c7 (0x68d4 bytes) 2024-09-14T05:29:48.941 INFO:teuthology.orchestra.run.smithi071.stdout:1706 trunc from 0xe8a90c7 to 0xe5ee4e6 2024-09-14T05:29:48.943 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:48.941+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:48.945 INFO:teuthology.orchestra.run.smithi071.stdout:1707 clone 16 order 19 su 4096 sc 11 2024-09-14T05:29:48.974 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:29:48.974 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:48.983 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:29:49.044 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:49.043+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:49.125 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:49.125 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:49.125 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:49.126 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:49.126 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:49.134 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:29:49.144 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:49.143+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:49.165 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T05:29:49.209 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone15 from 0xe5ee4e6 (overlap 0x30fd51a) to 0x187ea8d 2024-09-14T05:29:49.256 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:49.243+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:49.279 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:49.280 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:49.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:49.280 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:49.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.280 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:49.281 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:29:49.281 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.500133744000003, 2024-09-14T05:29:49.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.24562984099999999, 2024-09-14T05:29:49.281 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.281 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.281 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:49.282 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.282 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:49.282 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:49.282 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1 2024-09-14T05:29:49.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.282 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:29:49.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:29:49.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776565+0000", 2024-09-14T05:29:49.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4780000000000001e-06 2024-09-14T05:29:49.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776571+0000", 2024-09-14T05:29:49.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7669999999999998e-06 2024-09-14T05:29:49.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776580+0000", 2024-09-14T05:29:49.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9830000000000002e-06 2024-09-14T05:29:49.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776589+0000", 2024-09-14T05:29:49.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7579999999999998e-06 2024-09-14T05:29:49.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776662+0000", 2024-09-14T05:29:49.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3332999999999997e-05 2024-09-14T05:29:49.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:29:49.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776670+0000", 2024-09-14T05:29:49.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9850000000000006e-06 2024-09-14T05:29:49.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.020867+0000", 2024-09-14T05:29:49.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.244196265 2024-09-14T05:29:49.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.288 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.288 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021022+0000", 2024-09-14T05:29:49.288 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000155699 2024-09-14T05:29:49.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.288 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:49.288 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021169+0000", 2024-09-14T05:29:49.288 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014661700000000001 2024-09-14T05:29:49.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.288 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:49.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021891+0000", 2024-09-14T05:29:49.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072146700000000005 2024-09-14T05:29:49.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:49.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022124+0000", 2024-09-14T05:29:49.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023354199999999999 2024-09-14T05:29:49.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:49.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022164+0000", 2024-09-14T05:29:49.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9897000000000001e-05 2024-09-14T05:29:49.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.290 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.290 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022193+0000", 2024-09-14T05:29:49.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9050000000000001e-05 2024-09-14T05:29:49.290 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.290 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.290 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.290 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:1989 3.1f e30/24 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head v 30'29, mlcod=30'27)", 2024-09-14T05:29:49.290 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:29:49.290 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.683362757000001, 2024-09-14T05:29:49.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.064242492999999998, 2024-09-14T05:29:49.290 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.291 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.291 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:49.291 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.291 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:29:49.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.291 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:29:49.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.292 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593336+0000", 2024-09-14T05:29:49.292 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3909999999999999e-06 2024-09-14T05:29:49.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.292 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616632+0000", 2024-09-14T05:29:49.292 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.023296598000000002 2024-09-14T05:29:49.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.292 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616657+0000", 2024-09-14T05:29:49.292 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5228e-05 2024-09-14T05:29:49.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.293 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.293 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616684+0000", 2024-09-14T05:29:49.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6441999999999999e-05 2024-09-14T05:29:49.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.294 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.294 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616755+0000", 2024-09-14T05:29:49.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1452000000000006e-05 2024-09-14T05:29:49.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.294 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616780+0000", 2024-09-14T05:29:49.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4436000000000001e-05 2024-09-14T05:29:49.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:49.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657527+0000", 2024-09-14T05:29:49.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.040747323000000002 2024-09-14T05:29:49.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657577+0000", 2024-09-14T05:29:49.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9623000000000002e-05 2024-09-14T05:29:49.296 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.296 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.296 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.296 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:18110 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 5896192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:49.296 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.529294+0000", 2024-09-14T05:29:49.296 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.7474031060000002, 2024-09-14T05:29:49.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026518798, 2024-09-14T05:29:49.296 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.297 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.297 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.297 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.297 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.297 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.297 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 18110 2024-09-14T05:29:49.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.297 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.297 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.297 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529294+0000", 2024-09-14T05:29:49.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.298 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.298 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529294+0000", 2024-09-14T05:29:49.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.298 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529294+0000", 2024-09-14T05:29:49.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4099999999999999e-07 2024-09-14T05:29:49.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.299 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529296+0000", 2024-09-14T05:29:49.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1459999999999999e-06 2024-09-14T05:29:49.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529302+0000", 2024-09-14T05:29:49.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8730000000000004e-06 2024-09-14T05:29:49.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.529307+0000", 2024-09-14T05:29:49.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.887e-06 2024-09-14T05:29:49.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.555635+0000", 2024-09-14T05:29:49.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026327518000000001 2024-09-14T05:29:49.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.555700+0000", 2024-09-14T05:29:49.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5214000000000003e-05 2024-09-14T05:29:49.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.555813+0000", 2024-09-14T05:29:49.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000112719 2024-09-14T05:29:49.302 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.302 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.302 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.303 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:20821 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 10727424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:49.303 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.893666+0000", 2024-09-14T05:29:49.303 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.383030454, 2024-09-14T05:29:49.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027326612, 2024-09-14T05:29:49.303 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.303 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.303 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.303 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.303 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.303 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.303 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 20821 2024-09-14T05:29:49.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.303 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.304 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.304 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.893666+0000", 2024-09-14T05:29:49.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.304 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.304 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.893666+0000", 2024-09-14T05:29:49.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.304 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.893667+0000", 2024-09-14T05:29:49.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.92e-07 2024-09-14T05:29:49.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.893671+0000", 2024-09-14T05:29:49.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4450000000000001e-06 2024-09-14T05:29:49.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.893679+0000", 2024-09-14T05:29:49.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9319999999999996e-06 2024-09-14T05:29:49.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.893689+0000", 2024-09-14T05:29:49.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0336999999999999e-05 2024-09-14T05:29:49.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.915565+0000", 2024-09-14T05:29:49.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.021876321000000001 2024-09-14T05:29:49.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.915630+0000", 2024-09-14T05:29:49.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4511999999999995e-05 2024-09-14T05:29:49.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.307 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.920993+0000", 2024-09-14T05:29:49.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0053633730000000003 2024-09-14T05:29:49.307 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.307 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.307 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.307 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21081 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 10784768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:49.307 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.898946+0000", 2024-09-14T05:29:49.307 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3777504179999998, 2024-09-14T05:29:49.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026556192999999999, 2024-09-14T05:29:49.308 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.308 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.308 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.308 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.308 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.308 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.308 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21081 2024-09-14T05:29:49.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.308 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.308 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.308 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898946+0000", 2024-09-14T05:29:49.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898946+0000", 2024-09-14T05:29:49.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898947+0000", 2024-09-14T05:29:49.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3099999999999998e-07 2024-09-14T05:29:49.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898949+0000", 2024-09-14T05:29:49.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3760000000000002e-06 2024-09-14T05:29:49.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898957+0000", 2024-09-14T05:29:49.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3989999999999998e-06 2024-09-14T05:29:49.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898963+0000", 2024-09-14T05:29:49.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.772e-06 2024-09-14T05:29:49.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.925362+0000", 2024-09-14T05:29:49.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026398978 2024-09-14T05:29:49.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.925435+0000", 2024-09-14T05:29:49.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3147999999999998e-05 2024-09-14T05:29:49.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.925503+0000", 2024-09-14T05:29:49.312 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7989000000000002e-05 2024-09-14T05:29:49.312 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.312 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.312 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.313 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:21082 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 10805248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-14T05:29:49.313 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:44.898970+0000", 2024-09-14T05:29:49.313 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3777273689999996, 2024-09-14T05:29:49.313 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026769630999999999, 2024-09-14T05:29:49.314 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.314 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.314 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.314 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.314 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.314 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.314 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 21082 2024-09-14T05:29:49.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.314 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.314 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898970+0000", 2024-09-14T05:29:49.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898970+0000", 2024-09-14T05:29:49.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898970+0000", 2024-09-14T05:29:49.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8099999999999998e-07 2024-09-14T05:29:49.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898972+0000", 2024-09-14T05:29:49.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0150000000000002e-06 2024-09-14T05:29:49.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.316 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898979+0000", 2024-09-14T05:29:49.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6529999999999997e-06 2024-09-14T05:29:49.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.316 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.898984+0000", 2024-09-14T05:29:49.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3430000000000003e-06 2024-09-14T05:29:49.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.317 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.317 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.925572+0000", 2024-09-14T05:29:49.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026587745999999999 2024-09-14T05:29:49.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.317 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.317 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.925656+0000", 2024-09-14T05:29:49.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4488999999999997e-05 2024-09-14T05:29:49.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.317 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.317 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:44.925739+0000", 2024-09-14T05:29:49.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3003999999999993e-05 2024-09-14T05:29:49.318 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.318 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.318 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.318 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:63805 3.1a 3:5bd8d5cd:::rbd_data.112d46b1ca94.000000000000004f:11 [read 45056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T05:29:49.318 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:48.787764+0000", 2024-09-14T05:29:49.318 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.48893304199999998, 2024-09-14T05:29:49.318 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026585241999999999, 2024-09-14T05:29:49.318 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.318 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.318 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.318 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.318 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.319 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.319 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 63805 2024-09-14T05:29:49.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.319 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787764+0000", 2024-09-14T05:29:49.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787764+0000", 2024-09-14T05:29:49.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787764+0000", 2024-09-14T05:29:49.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0499999999999999e-07 2024-09-14T05:29:49.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787766+0000", 2024-09-14T05:29:49.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9099999999999999e-06 2024-09-14T05:29:49.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787772+0000", 2024-09-14T05:29:49.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3450000000000002e-06 2024-09-14T05:29:49.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787777+0000", 2024-09-14T05:29:49.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9540000000000003e-06 2024-09-14T05:29:49.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.814229+0000", 2024-09-14T05:29:49.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026451327 2024-09-14T05:29:49.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.814274+0000", 2024-09-14T05:29:49.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5454999999999998e-05 2024-09-14T05:29:49.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.814349+0000", 2024-09-14T05:29:49.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4945999999999997e-05 2024-09-14T05:29:49.322 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.322 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.323 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.323 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:63806 3.1a 3:5bd8d5cd:::rbd_data.112d46b1ca94.000000000000004f:11 [read 61440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T05:29:49.323 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:48.787783+0000", 2024-09-14T05:29:49.323 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.48891349499999998, 2024-09-14T05:29:49.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026792795000000001, 2024-09-14T05:29:49.323 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.323 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.323 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.323 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.323 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.323 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.324 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 63806 2024-09-14T05:29:49.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.324 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787783+0000", 2024-09-14T05:29:49.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787783+0000", 2024-09-14T05:29:49.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787784+0000", 2024-09-14T05:29:49.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7299999999999996e-07 2024-09-14T05:29:49.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787786+0000", 2024-09-14T05:29:49.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.826e-06 2024-09-14T05:29:49.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787792+0000", 2024-09-14T05:29:49.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4289999999999998e-06 2024-09-14T05:29:49.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.326 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787798+0000", 2024-09-14T05:29:49.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.071e-06 2024-09-14T05:29:49.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.326 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.814417+0000", 2024-09-14T05:29:49.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026618741000000001 2024-09-14T05:29:49.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.326 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.814504+0000", 2024-09-14T05:29:49.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6688999999999996e-05 2024-09-14T05:29:49.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.814576+0000", 2024-09-14T05:29:49.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2465999999999994e-05 2024-09-14T05:29:49.327 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.327 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.327 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.327 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:63807 3.1a 3:5bd8d5cd:::rbd_data.112d46b1ca94.000000000000004f:11 [read 77824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T05:29:49.328 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:48.787805+0000", 2024-09-14T05:29:49.328 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.48889239400000001, 2024-09-14T05:29:49.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.02699671, 2024-09-14T05:29:49.328 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.328 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.328 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.328 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.328 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.328 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.328 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 63807 2024-09-14T05:29:49.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.328 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787805+0000", 2024-09-14T05:29:49.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787805+0000", 2024-09-14T05:29:49.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787805+0000", 2024-09-14T05:29:49.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8799999999999998e-07 2024-09-14T05:29:49.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787807+0000", 2024-09-14T05:29:49.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.694e-06 2024-09-14T05:29:49.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787813+0000", 2024-09-14T05:29:49.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0260000000000004e-06 2024-09-14T05:29:49.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787818+0000", 2024-09-14T05:29:49.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9110000000000001e-06 2024-09-14T05:29:49.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.814627+0000", 2024-09-14T05:29:49.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026809769000000001 2024-09-14T05:29:49.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.814703+0000", 2024-09-14T05:29:49.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5735000000000006e-05 2024-09-14T05:29:49.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.332 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.332 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.814801+0000", 2024-09-14T05:29:49.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8187e-05 2024-09-14T05:29:49.332 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.332 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.332 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.332 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:63808 3.1a 3:5bd8d5cd:::rbd_data.112d46b1ca94.000000000000004f:11 [read 94208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T05:29:49.332 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:48.787832+0000", 2024-09-14T05:29:49.332 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.48886448900000001, 2024-09-14T05:29:49.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027107976999999998, 2024-09-14T05:29:49.333 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.333 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.333 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.333 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.333 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.333 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.333 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 63808 2024-09-14T05:29:49.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.333 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.333 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.333 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787832+0000", 2024-09-14T05:29:49.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787832+0000", 2024-09-14T05:29:49.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787833+0000", 2024-09-14T05:29:49.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1500000000000005e-07 2024-09-14T05:29:49.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787835+0000", 2024-09-14T05:29:49.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3010000000000001e-06 2024-09-14T05:29:49.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787842+0000", 2024-09-14T05:29:49.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4779999999999996e-06 2024-09-14T05:29:49.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787847+0000", 2024-09-14T05:29:49.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3900000000000001e-06 2024-09-14T05:29:49.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.814832+0000", 2024-09-14T05:29:49.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026984773 2024-09-14T05:29:49.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.814873+0000", 2024-09-14T05:29:49.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0741999999999997e-05 2024-09-14T05:29:49.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.814940+0000", 2024-09-14T05:29:49.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7778000000000001e-05 2024-09-14T05:29:49.336 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.337 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.337 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.337 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:63809 3.1a 3:5bd8d5cd:::rbd_data.112d46b1ca94.000000000000004f:11 [read 110592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T05:29:49.337 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:48.787854+0000", 2024-09-14T05:29:49.337 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.48884291299999999, 2024-09-14T05:29:49.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027278816000000001, 2024-09-14T05:29:49.337 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.337 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.337 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.337 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.337 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.338 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.338 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 63809 2024-09-14T05:29:49.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.338 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787854+0000", 2024-09-14T05:29:49.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787854+0000", 2024-09-14T05:29:49.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787854+0000", 2024-09-14T05:29:49.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5999999999999999e-07 2024-09-14T05:29:49.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787860+0000", 2024-09-14T05:29:49.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3489999999999999e-06 2024-09-14T05:29:49.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787866+0000", 2024-09-14T05:29:49.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6109999999999999e-06 2024-09-14T05:29:49.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.340 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.340 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787872+0000", 2024-09-14T05:29:49.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2290000000000002e-06 2024-09-14T05:29:49.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.340 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.340 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.814983+0000", 2024-09-14T05:29:49.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027110947999999999 2024-09-14T05:29:49.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.341 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815062+0000", 2024-09-14T05:29:49.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9688e-05 2024-09-14T05:29:49.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.341 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815133+0000", 2024-09-14T05:29:49.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0531000000000006e-05 2024-09-14T05:29:49.341 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.341 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.341 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.342 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:63811 3.1a 3:5bd8d5cd:::rbd_data.112d46b1ca94.000000000000004f:11 [read 126976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T05:29:49.342 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:48.787905+0000", 2024-09-14T05:29:49.342 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.48879199000000001, 2024-09-14T05:29:49.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027373029, 2024-09-14T05:29:49.342 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.342 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.342 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.342 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.342 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.342 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.342 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 63811 2024-09-14T05:29:49.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.342 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787905+0000", 2024-09-14T05:29:49.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787905+0000", 2024-09-14T05:29:49.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787905+0000", 2024-09-14T05:29:49.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0499999999999999e-07 2024-09-14T05:29:49.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787907+0000", 2024-09-14T05:29:49.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.978e-06 2024-09-14T05:29:49.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787914+0000", 2024-09-14T05:29:49.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3980000000000004e-06 2024-09-14T05:29:49.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787918+0000", 2024-09-14T05:29:49.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7779999999999998e-06 2024-09-14T05:29:49.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.345 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815172+0000", 2024-09-14T05:29:49.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027253622000000002 2024-09-14T05:29:49.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.345 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815216+0000", 2024-09-14T05:29:49.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3618000000000001e-05 2024-09-14T05:29:49.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.346 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815278+0000", 2024-09-14T05:29:49.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2230000000000006e-05 2024-09-14T05:29:49.346 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.346 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.346 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.346 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:63813 3.1a 3:5bd8d5cd:::rbd_data.112d46b1ca94.000000000000004f:11 [read 143360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T05:29:49.346 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:48.787925+0000", 2024-09-14T05:29:49.347 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.48877238899999997, 2024-09-14T05:29:49.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027533328999999999, 2024-09-14T05:29:49.347 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.347 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.347 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.347 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.347 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.347 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.347 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 63813 2024-09-14T05:29:49.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.347 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.347 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.347 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787925+0000", 2024-09-14T05:29:49.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.348 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787925+0000", 2024-09-14T05:29:49.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.348 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787925+0000", 2024-09-14T05:29:49.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4299999999999999e-07 2024-09-14T05:29:49.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787927+0000", 2024-09-14T05:29:49.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7850000000000001e-06 2024-09-14T05:29:49.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787932+0000", 2024-09-14T05:29:49.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2499999999999997e-06 2024-09-14T05:29:49.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787969+0000", 2024-09-14T05:29:49.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6770000000000002e-05 2024-09-14T05:29:49.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815306+0000", 2024-09-14T05:29:49.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027337070000000002 2024-09-14T05:29:49.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815366+0000", 2024-09-14T05:29:49.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9973000000000003e-05 2024-09-14T05:29:49.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815458+0000", 2024-09-14T05:29:49.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2138000000000005e-05 2024-09-14T05:29:49.351 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.351 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.351 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.351 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:63815 3.1a 3:5bd8d5cd:::rbd_data.112d46b1ca94.000000000000004f:11 [read 159744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T05:29:49.351 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:48.787995+0000", 2024-09-14T05:29:49.351 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.48870147000000003, 2024-09-14T05:29:49.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027603552999999999, 2024-09-14T05:29:49.351 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.351 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.351 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.351 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.352 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.352 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.352 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 63815 2024-09-14T05:29:49.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.352 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787995+0000", 2024-09-14T05:29:49.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787995+0000", 2024-09-14T05:29:49.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787996+0000", 2024-09-14T05:29:49.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2300000000000001e-07 2024-09-14T05:29:49.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.787999+0000", 2024-09-14T05:29:49.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5579999999999999e-06 2024-09-14T05:29:49.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.788171+0000", 2024-09-14T05:29:49.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000172764 2024-09-14T05:29:49.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.788195+0000", 2024-09-14T05:29:49.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3362000000000001e-05 2024-09-14T05:29:49.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815490+0000", 2024-09-14T05:29:49.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027295211 2024-09-14T05:29:49.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815551+0000", 2024-09-14T05:29:49.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1388000000000002e-05 2024-09-14T05:29:49.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815599+0000", 2024-09-14T05:29:49.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7647e-05 2024-09-14T05:29:49.355 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.355 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.355 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.356 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:63816 3.1a 3:5bd8d5cd:::rbd_data.112d46b1ca94.000000000000004f:11 [read 176128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T05:29:49.356 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:48.788238+0000", 2024-09-14T05:29:49.356 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.48845841299999998, 2024-09-14T05:29:49.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027477590999999999, 2024-09-14T05:29:49.356 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.356 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.356 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.356 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.356 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.356 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.356 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 63816 2024-09-14T05:29:49.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.356 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.788238+0000", 2024-09-14T05:29:49.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.788238+0000", 2024-09-14T05:29:49.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.788239+0000", 2024-09-14T05:29:49.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4199999999999995e-07 2024-09-14T05:29:49.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.788242+0000", 2024-09-14T05:29:49.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.151e-06 2024-09-14T05:29:49.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.788315+0000", 2024-09-14T05:29:49.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3069000000000001e-05 2024-09-14T05:29:49.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.788413+0000", 2024-09-14T05:29:49.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7452000000000001e-05 2024-09-14T05:29:49.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815627+0000", 2024-09-14T05:29:49.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.027213623999999999 2024-09-14T05:29:49.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815678+0000", 2024-09-14T05:29:49.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1638999999999998e-05 2024-09-14T05:29:49.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815716+0000", 2024-09-14T05:29:49.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7914000000000003e-05 2024-09-14T05:29:49.362 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.362 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.362 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.362 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:63818 3.1a 3:5bd8d5cd:::rbd_data.112d46b1ca94.000000000000004f:11 [read 192512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T05:29:49.362 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:48.788867+0000", 2024-09-14T05:29:49.362 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.487830176, 2024-09-14T05:29:49.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026997525000000001, 2024-09-14T05:29:49.362 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.362 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.362 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.363 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.363 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.363 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.363 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 63818 2024-09-14T05:29:49.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.363 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.363 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.363 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.788867+0000", 2024-09-14T05:29:49.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.363 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.363 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.788867+0000", 2024-09-14T05:29:49.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.788867+0000", 2024-09-14T05:29:49.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3399999999999999e-07 2024-09-14T05:29:49.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.788912+0000", 2024-09-14T05:29:49.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4694999999999998e-05 2024-09-14T05:29:49.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789007+0000", 2024-09-14T05:29:49.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5235999999999994e-05 2024-09-14T05:29:49.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789088+0000", 2024-09-14T05:29:49.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1097e-05 2024-09-14T05:29:49.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815751+0000", 2024-09-14T05:29:49.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026662867999999999 2024-09-14T05:29:49.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815803+0000", 2024-09-14T05:29:49.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1492000000000001e-05 2024-09-14T05:29:49.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815864+0000", 2024-09-14T05:29:49.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1502999999999997e-05 2024-09-14T05:29:49.366 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.366 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.367 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.367 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:63820 3.1a 3:5bd8d5cd:::rbd_data.112d46b1ca94.000000000000004f:11 [read 208896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T05:29:49.367 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:48.789354+0000", 2024-09-14T05:29:49.367 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.48734317999999999, 2024-09-14T05:29:49.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026669017, 2024-09-14T05:29:49.367 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.367 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.367 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.367 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.367 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.367 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.368 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 63820 2024-09-14T05:29:49.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.368 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789354+0000", 2024-09-14T05:29:49.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789354+0000", 2024-09-14T05:29:49.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789355+0000", 2024-09-14T05:29:49.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7300000000000005e-07 2024-09-14T05:29:49.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789372+0000", 2024-09-14T05:29:49.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7070000000000001e-05 2024-09-14T05:29:49.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789457+0000", 2024-09-14T05:29:49.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5913000000000002e-05 2024-09-14T05:29:49.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.370 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789464+0000", 2024-09-14T05:29:49.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9330000000000004e-06 2024-09-14T05:29:49.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.370 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815914+0000", 2024-09-14T05:29:49.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026449329000000001 2024-09-14T05:29:49.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.815960+0000", 2024-09-14T05:29:49.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6566000000000001e-05 2024-09-14T05:29:49.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.816023+0000", 2024-09-14T05:29:49.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2433000000000003e-05 2024-09-14T05:29:49.371 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.371 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.371 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.371 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:63822 3.1a 3:5bd8d5cd:::rbd_data.112d46b1ca94.000000000000004f:11 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T05:29:49.372 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:48.789475+0000", 2024-09-14T05:29:49.372 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.487221719, 2024-09-14T05:29:49.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026661463, 2024-09-14T05:29:49.372 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.372 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.372 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.372 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.372 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.372 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.372 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 63822 2024-09-14T05:29:49.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.372 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789475+0000", 2024-09-14T05:29:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789475+0000", 2024-09-14T05:29:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789476+0000", 2024-09-14T05:29:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2399999999999998e-07 2024-09-14T05:29:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789479+0000", 2024-09-14T05:29:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7930000000000002e-06 2024-09-14T05:29:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789486+0000", 2024-09-14T05:29:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8269999999999998e-06 2024-09-14T05:29:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789492+0000", 2024-09-14T05:29:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4589999999999997e-06 2024-09-14T05:29:49.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.816050+0000", 2024-09-14T05:29:49.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026558592999999998 2024-09-14T05:29:49.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.816092+0000", 2024-09-14T05:29:49.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1275e-05 2024-09-14T05:29:49.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.816137+0000", 2024-09-14T05:29:49.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4892000000000003e-05 2024-09-14T05:29:49.376 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.376 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.376 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.376 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:63823 3.1a 3:5bd8d5cd:::rbd_data.112d46b1ca94.000000000000004f:11 [read 241664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T05:29:49.376 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:48.789500+0000", 2024-09-14T05:29:49.376 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.48719728499999998, 2024-09-14T05:29:49.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026776239, 2024-09-14T05:29:49.376 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.376 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.377 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.377 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.377 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.377 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.377 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 63823 2024-09-14T05:29:49.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.378 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789500+0000", 2024-09-14T05:29:49.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789500+0000", 2024-09-14T05:29:49.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789500+0000", 2024-09-14T05:29:49.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8599999999999998e-07 2024-09-14T05:29:49.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789502+0000", 2024-09-14T05:29:49.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3190000000000002e-06 2024-09-14T05:29:49.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789509+0000", 2024-09-14T05:29:49.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7000000000000002e-06 2024-09-14T05:29:49.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789514+0000", 2024-09-14T05:29:49.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2340000000000002e-06 2024-09-14T05:29:49.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.816167+0000", 2024-09-14T05:29:49.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026652432 2024-09-14T05:29:49.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.816233+0000", 2024-09-14T05:29:49.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5937000000000002e-05 2024-09-14T05:29:49.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.816276+0000", 2024-09-14T05:29:49.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3130999999999998e-05 2024-09-14T05:29:49.381 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.381 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.381 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.382 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:63825 3.1a 3:5bd8d5cd:::rbd_data.112d46b1ca94.000000000000004f:11 [read 258048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T05:29:49.382 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:48.789521+0000", 2024-09-14T05:29:49.382 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.4871762, 2024-09-14T05:29:49.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026883566000000001, 2024-09-14T05:29:49.382 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.382 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.382 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.382 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.382 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.384 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.384 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 63825 2024-09-14T05:29:49.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.384 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789521+0000", 2024-09-14T05:29:49.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789521+0000", 2024-09-14T05:29:49.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789521+0000", 2024-09-14T05:29:49.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8500000000000002e-07 2024-09-14T05:29:49.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789523+0000", 2024-09-14T05:29:49.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.824e-06 2024-09-14T05:29:49.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789529+0000", 2024-09-14T05:29:49.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3779999999999998e-06 2024-09-14T05:29:49.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.789534+0000", 2024-09-14T05:29:49.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0900000000000004e-06 2024-09-14T05:29:49.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.816319+0000", 2024-09-14T05:29:49.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.026784589000000001 2024-09-14T05:29:49.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.816362+0000", 2024-09-14T05:29:49.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2874999999999999e-05 2024-09-14T05:29:49.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:49.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:48.816404+0000", 2024-09-14T05:29:49.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2324999999999999e-05 2024-09-14T05:29:49.387 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.387 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.387 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.387 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.387 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.387 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:49.389 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:29:49.390 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:49.388+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:49.490 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:49.488+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:49.591 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:49.590+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:49.607 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:49.607 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:49.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.607 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73740 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.608 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559628+0000", 2024-09-14T05:29:49.608 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040608972, 2024-09-14T05:29:49.608 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.040629432, 2024-09-14T05:29:49.608 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.608 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.608 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.608 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.608 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.609 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.609 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73740 2024-09-14T05:29:49.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.609 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559628+0000", 2024-09-14T05:29:49.609 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.610 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.610 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.610 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559628+0000", 2024-09-14T05:29:49.610 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.610 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.610 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.611 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559628+0000", 2024-09-14T05:29:49.611 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9699999999999996e-07 2024-09-14T05:29:49.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.611 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.611 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559632+0000", 2024-09-14T05:29:49.611 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1659999999999998e-06 2024-09-14T05:29:49.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.612 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.612 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559639+0000", 2024-09-14T05:29:49.612 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9840000000000004e-06 2024-09-14T05:29:49.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.612 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.612 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559644+0000", 2024-09-14T05:29:49.613 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0730000000000004e-06 2024-09-14T05:29:49.613 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.613 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.613 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.613 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.613 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.613 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74052 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.614 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:29:49.614 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.038944096999999997, 2024-09-14T05:29:49.614 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.039023548999999998, 2024-09-14T05:29:49.614 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.614 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.614 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.614 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.614 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.615 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.615 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74052 2024-09-14T05:29:49.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.615 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.616 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:29:49.616 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.616 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.616 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:29:49.616 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.616 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.616 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:29:49.617 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1099999999999996e-07 2024-09-14T05:29:49.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.617 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.617 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561295+0000", 2024-09-14T05:29:49.617 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7770000000000001e-06 2024-09-14T05:29:49.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.617 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.617 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561303+0000", 2024-09-14T05:29:49.617 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6860000000000006e-06 2024-09-14T05:29:49.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.618 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.618 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561308+0000", 2024-09-14T05:29:49.618 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.023e-06 2024-09-14T05:29:49.618 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.618 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.618 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.618 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73743 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.618 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559651+0000", 2024-09-14T05:29:49.618 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040586285, 2024-09-14T05:29:49.619 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.040715715, 2024-09-14T05:29:49.619 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.619 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.619 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.619 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.619 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.619 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.619 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73743 2024-09-14T05:29:49.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.619 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.619 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.619 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559651+0000", 2024-09-14T05:29:49.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.620 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.620 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559651+0000", 2024-09-14T05:29:49.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559651+0000", 2024-09-14T05:29:49.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0400000000000002e-07 2024-09-14T05:29:49.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.622 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559660+0000", 2024-09-14T05:29:49.622 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1889999999999996e-06 2024-09-14T05:29:49.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.622 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.622 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559667+0000", 2024-09-14T05:29:49.623 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8109999999999997e-06 2024-09-14T05:29:49.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.623 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.623 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.623 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559672+0000", 2024-09-14T05:29:49.624 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8840000000000002e-06 2024-09-14T05:29:49.624 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.624 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.624 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.625 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74054 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.625 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:29:49.625 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.038922070000000003, 2024-09-14T05:29:49.625 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.039100028000000002, 2024-09-14T05:29:49.625 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.625 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.625 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.626 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.626 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.626 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.626 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74054 2024-09-14T05:29:49.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.626 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:29:49.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:29:49.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.628 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.628 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:29:49.628 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9699999999999996e-07 2024-09-14T05:29:49.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.628 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.628 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561317+0000", 2024-09-14T05:29:49.628 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9740000000000001e-06 2024-09-14T05:29:49.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.629 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.629 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561324+0000", 2024-09-14T05:29:49.629 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4590000000000003e-06 2024-09-14T05:29:49.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.630 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.630 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561329+0000", 2024-09-14T05:29:49.630 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5969999999999999e-06 2024-09-14T05:29:49.630 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.630 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.631 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.631 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73757 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.631 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:29:49.631 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040558421999999997, 2024-09-14T05:29:49.632 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.040781993000000002, 2024-09-14T05:29:49.632 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.632 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.632 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.632 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.632 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.632 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.632 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73757 2024-09-14T05:29:49.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.633 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:29:49.633 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.634 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:29:49.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.634 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.634 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559679+0000", 2024-09-14T05:29:49.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4000000000000002e-07 2024-09-14T05:29:49.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.634 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.634 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559681+0000", 2024-09-14T05:29:49.635 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1780000000000002e-06 2024-09-14T05:29:49.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.635 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.635 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559687+0000", 2024-09-14T05:29:49.635 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0270000000000001e-06 2024-09-14T05:29:49.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.635 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.635 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559692+0000", 2024-09-14T05:29:49.636 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1030000000000001e-06 2024-09-14T05:29:49.636 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.636 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.636 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.637 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74056 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.637 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:29:49.637 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.038727366999999999, 2024-09-14T05:29:49.637 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.038998964999999997, 2024-09-14T05:29:49.637 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.637 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.637 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.637 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.637 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.637 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.638 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74056 2024-09-14T05:29:49.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.638 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.638 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.638 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:29:49.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.638 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:29:49.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561510+0000", 2024-09-14T05:29:49.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2399999999999998e-07 2024-09-14T05:29:49.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561513+0000", 2024-09-14T05:29:49.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7219999999999999e-06 2024-09-14T05:29:49.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.640 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561524+0000", 2024-09-14T05:29:49.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1357e-05 2024-09-14T05:29:49.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.640 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561530+0000", 2024-09-14T05:29:49.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2890000000000003e-06 2024-09-14T05:29:49.641 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.641 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.641 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.641 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73760 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.642 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:29:49.642 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040538316999999997, 2024-09-14T05:29:49.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.040857301999999998, 2024-09-14T05:29:49.642 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.642 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.642 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.642 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.642 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.642 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.642 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73760 2024-09-14T05:29:49.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.643 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.643 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.643 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:29:49.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.643 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.643 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:29:49.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:29:49.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:29:49.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559701+0000", 2024-09-14T05:29:49.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1909999999999999e-06 2024-09-14T05:29:49.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559708+0000", 2024-09-14T05:29:49.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7279999999999998e-06 2024-09-14T05:29:49.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559713+0000", 2024-09-14T05:29:49.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8389999999999997e-06 2024-09-14T05:29:49.645 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.646 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.646 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.646 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74058 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.646 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:29:49.647 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.038248694, 2024-09-14T05:29:49.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.038613890999999997, 2024-09-14T05:29:49.647 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.647 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.647 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.647 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.647 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.647 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.648 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74058 2024-09-14T05:29:49.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.648 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.648 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.648 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:29:49.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.649 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.649 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:29:49.649 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.649 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561989+0000", 2024-09-14T05:29:49.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3300000000000001e-07 2024-09-14T05:29:49.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561993+0000", 2024-09-14T05:29:49.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1790000000000001e-06 2024-09-14T05:29:49.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.651 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.651 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562007+0000", 2024-09-14T05:29:49.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3434e-05 2024-09-14T05:29:49.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.652 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562014+0000", 2024-09-14T05:29:49.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.537e-06 2024-09-14T05:29:49.653 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.653 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.653 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.653 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74152 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.653 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569214+0000", 2024-09-14T05:29:49.654 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.031022757000000001, 2024-09-14T05:29:49.654 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.031436432, 2024-09-14T05:29:49.654 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.654 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.654 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.655 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.655 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.655 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.655 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74152 2024-09-14T05:29:49.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.655 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.655 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.655 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.655 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569214+0000", 2024-09-14T05:29:49.655 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.655 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.656 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.656 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569214+0000", 2024-09-14T05:29:49.656 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.656 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.657 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569214+0000", 2024-09-14T05:29:49.657 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9799999999999999e-07 2024-09-14T05:29:49.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.657 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.657 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.657 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569217+0000", 2024-09-14T05:29:49.657 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0619999999999999e-06 2024-09-14T05:29:49.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.658 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.658 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.658 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569226+0000", 2024-09-14T05:29:49.658 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7219999999999994e-06 2024-09-14T05:29:49.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.658 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.659 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.659 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569231+0000", 2024-09-14T05:29:49.659 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.164e-06 2024-09-14T05:29:49.659 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.659 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.659 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.660 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.660 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73773 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.660 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:29:49.660 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040518147999999997, 2024-09-14T05:29:49.660 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.040982206, 2024-09-14T05:29:49.660 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.660 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.660 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.660 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.661 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.661 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.661 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73773 2024-09-14T05:29:49.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.661 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.661 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.661 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:29:49.661 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.661 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.662 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:29:49.662 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.662 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.662 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:29:49.662 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8599999999999999e-07 2024-09-14T05:29:49.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.662 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.663 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559721+0000", 2024-09-14T05:29:49.663 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0540000000000002e-06 2024-09-14T05:29:49.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.663 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.663 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559729+0000", 2024-09-14T05:29:49.663 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1359999999999997e-06 2024-09-14T05:29:49.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.663 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.663 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559734+0000", 2024-09-14T05:29:49.663 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2140000000000004e-06 2024-09-14T05:29:49.663 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.664 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.664 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.664 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74059 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.664 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:29:49.664 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.038190702999999999, 2024-09-14T05:29:49.664 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0387027, 2024-09-14T05:29:49.664 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.664 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.664 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.664 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.664 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.665 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.665 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74059 2024-09-14T05:29:49.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.665 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.665 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.665 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:29:49.665 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.665 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.665 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:29:49.666 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.666 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.666 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562047+0000", 2024-09-14T05:29:49.666 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6899999999999997e-07 2024-09-14T05:29:49.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.667 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.667 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562050+0000", 2024-09-14T05:29:49.667 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.943e-06 2024-09-14T05:29:49.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.667 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.667 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562058+0000", 2024-09-14T05:29:49.667 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0120000000000005e-06 2024-09-14T05:29:49.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.668 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.668 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562063+0000", 2024-09-14T05:29:49.668 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:29:49.668 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.668 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.668 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.668 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.669 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73777 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.669 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:29:49.669 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040495746999999999, 2024-09-14T05:29:49.669 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.041055270999999997, 2024-09-14T05:29:49.669 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.669 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.669 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.669 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.669 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.669 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.669 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73777 2024-09-14T05:29:49.669 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.669 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.670 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:29:49.670 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.670 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:29:49.670 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.670 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559742+0000", 2024-09-14T05:29:49.671 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4099999999999999e-07 2024-09-14T05:29:49.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.671 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.671 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.671 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559744+0000", 2024-09-14T05:29:49.671 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0040000000000002e-06 2024-09-14T05:29:49.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.672 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.672 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559749+0000", 2024-09-14T05:29:49.672 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9739999999999999e-06 2024-09-14T05:29:49.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.672 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.672 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559754+0000", 2024-09-14T05:29:49.672 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9540000000000003e-06 2024-09-14T05:29:49.673 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.673 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.673 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.673 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.673 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74060 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.673 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:29:49.673 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.038165325999999999, 2024-09-14T05:29:49.674 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.038770951999999997, 2024-09-14T05:29:49.674 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.674 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.674 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.674 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.674 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.674 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.675 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74060 2024-09-14T05:29:49.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.675 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.675 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.675 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:29:49.675 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.675 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.676 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:29:49.676 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.676 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.676 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.676 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:29:49.677 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5200000000000002e-07 2024-09-14T05:29:49.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.677 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.677 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562075+0000", 2024-09-14T05:29:49.678 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5200000000000002e-06 2024-09-14T05:29:49.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.678 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.678 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.678 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562082+0000", 2024-09-14T05:29:49.679 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8929999999999999e-06 2024-09-14T05:29:49.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.679 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.679 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562088+0000", 2024-09-14T05:29:49.679 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5570000000000003e-06 2024-09-14T05:29:49.679 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.679 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.679 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73790 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:29:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040475812, 2024-09-14T05:29:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.041133752000000003, 2024-09-14T05:29:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73790 2024-09-14T05:29:49.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.681 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.681 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.681 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:29:49.681 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.681 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.681 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:29:49.682 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.682 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.682 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.682 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:29:49.682 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7599999999999998e-07 2024-09-14T05:29:49.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.682 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.682 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.682 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559763+0000", 2024-09-14T05:29:49.682 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0880000000000002e-06 2024-09-14T05:29:49.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.682 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.683 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.683 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559769+0000", 2024-09-14T05:29:49.683 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6740000000000002e-06 2024-09-14T05:29:49.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.683 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.683 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.683 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559774+0000", 2024-09-14T05:29:49.683 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6110000000000004e-06 2024-09-14T05:29:49.683 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.683 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.683 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.683 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.683 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74061 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.684 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.564939+0000", 2024-09-14T05:29:49.684 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.035298266000000002, 2024-09-14T05:29:49.684 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.036000967000000002, 2024-09-14T05:29:49.684 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.684 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.684 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.684 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.684 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.684 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.684 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74061 2024-09-14T05:29:49.684 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.684 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.684 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.685 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.685 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.564939+0000", 2024-09-14T05:29:49.685 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.685 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.685 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.685 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.564939+0000", 2024-09-14T05:29:49.685 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.685 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.685 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.685 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.564940+0000", 2024-09-14T05:29:49.685 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.009e-06 2024-09-14T05:29:49.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.686 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.686 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.686 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.564944+0000", 2024-09-14T05:29:49.686 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3000000000000003e-06 2024-09-14T05:29:49.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.686 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.686 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.686 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.564964+0000", 2024-09-14T05:29:49.686 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0186e-05 2024-09-14T05:29:49.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.686 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.686 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.687 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.564971+0000", 2024-09-14T05:29:49.687 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3919999999999997e-06 2024-09-14T05:29:49.687 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.687 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.687 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.687 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.687 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74231 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.687 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569291+0000", 2024-09-14T05:29:49.687 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.030946069, 2024-09-14T05:29:49.687 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.031695602000000003, 2024-09-14T05:29:49.687 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.687 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.688 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.688 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.688 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.688 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.688 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74231 2024-09-14T05:29:49.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.688 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.688 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.688 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569291+0000", 2024-09-14T05:29:49.689 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.689 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.689 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569291+0000", 2024-09-14T05:29:49.689 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.689 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.689 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569291+0000", 2024-09-14T05:29:49.689 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1300000000000001e-07 2024-09-14T05:29:49.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.690 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.690 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569293+0000", 2024-09-14T05:29:49.690 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7439999999999999e-06 2024-09-14T05:29:49.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.690 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.690 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.690 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569299+0000", 2024-09-14T05:29:49.690 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3130000000000004e-06 2024-09-14T05:29:49.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.690 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.690 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.690 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569304+0000", 2024-09-14T05:29:49.690 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8509999999999998e-06 2024-09-14T05:29:49.691 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.691 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.691 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.691 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.691 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.691 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73793 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.691 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:29:49.691 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040456607999999998, 2024-09-14T05:29:49.691 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.041253639000000002, 2024-09-14T05:29:49.691 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.691 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.691 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.691 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.691 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.692 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.692 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73793 2024-09-14T05:29:49.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.692 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.692 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.692 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.692 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:29:49.692 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.692 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.692 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.692 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:29:49.693 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.693 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.693 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.693 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559781+0000", 2024-09-14T05:29:49.693 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2899999999999999e-07 2024-09-14T05:29:49.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.693 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.693 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.693 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559783+0000", 2024-09-14T05:29:49.693 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.075e-06 2024-09-14T05:29:49.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.693 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.694 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.694 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559788+0000", 2024-09-14T05:29:49.694 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5369999999999997e-06 2024-09-14T05:29:49.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.694 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.694 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.694 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559793+0000", 2024-09-14T05:29:49.694 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:29:49.694 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.694 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.694 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.695 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.695 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74063 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.695 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.567228+0000", 2024-09-14T05:29:49.695 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.033008845000000002, 2024-09-14T05:29:49.695 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.033855771999999999, 2024-09-14T05:29:49.695 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.696 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.696 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.696 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.696 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.696 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.696 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74063 2024-09-14T05:29:49.697 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.697 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.697 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.697 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.697 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567228+0000", 2024-09-14T05:29:49.697 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.697 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.697 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.697 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.697 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567228+0000", 2024-09-14T05:29:49.697 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.697 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.697 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.697 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.698 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567229+0000", 2024-09-14T05:29:49.698 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5600000000000004e-07 2024-09-14T05:29:49.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.698 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.698 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.698 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567233+0000", 2024-09-14T05:29:49.698 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7500000000000001e-06 2024-09-14T05:29:49.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.698 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.698 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.698 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567241+0000", 2024-09-14T05:29:49.698 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7669999999999992e-06 2024-09-14T05:29:49.699 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.699 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.699 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.699 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567248+0000", 2024-09-14T05:29:49.699 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6370000000000002e-06 2024-09-14T05:29:49.699 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.699 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.699 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.699 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.699 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.699 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74233 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.699 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569310+0000", 2024-09-14T05:29:49.699 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.030926457000000001, 2024-09-14T05:29:49.699 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.031821328000000003, 2024-09-14T05:29:49.700 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.700 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.700 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.700 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.700 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.700 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.700 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74233 2024-09-14T05:29:49.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.700 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.700 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.700 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569310+0000", 2024-09-14T05:29:49.700 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.701 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.701 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.701 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569310+0000", 2024-09-14T05:29:49.701 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.701 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.701 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.701 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569311+0000", 2024-09-14T05:29:49.701 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4099999999999999e-07 2024-09-14T05:29:49.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.701 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.702 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.702 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569313+0000", 2024-09-14T05:29:49.702 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8169999999999999e-06 2024-09-14T05:29:49.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.702 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.702 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569319+0000", 2024-09-14T05:29:49.702 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4799999999999998e-06 2024-09-14T05:29:49.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.702 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.702 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569324+0000", 2024-09-14T05:29:49.702 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2020000000000003e-06 2024-09-14T05:29:49.703 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.703 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.703 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.703 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.703 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.703 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73806 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.703 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:29:49.703 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040437981999999997, 2024-09-14T05:29:49.703 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.041380570999999998, 2024-09-14T05:29:49.703 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.703 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.703 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.703 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.703 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.704 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.704 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73806 2024-09-14T05:29:49.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.704 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.704 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.704 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:29:49.704 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.704 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.704 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:29:49.704 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.705 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.705 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:29:49.705 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:29:49.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.705 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.705 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559801+0000", 2024-09-14T05:29:49.705 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.995e-06 2024-09-14T05:29:49.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.705 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.706 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559807+0000", 2024-09-14T05:29:49.706 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5450000000000003e-06 2024-09-14T05:29:49.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.706 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.706 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559811+0000", 2024-09-14T05:29:49.706 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5090000000000004e-06 2024-09-14T05:29:49.706 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.706 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.706 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.706 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74065 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.706 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.567258+0000", 2024-09-14T05:29:49.707 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.032978397, 2024-09-14T05:29:49.707 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.033966521999999999, 2024-09-14T05:29:49.707 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.707 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.707 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.707 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.707 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.707 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.707 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74065 2024-09-14T05:29:49.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.707 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.707 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.707 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567258+0000", 2024-09-14T05:29:49.708 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.708 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.708 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567258+0000", 2024-09-14T05:29:49.708 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.708 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.708 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567259+0000", 2024-09-14T05:29:49.708 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.58e-07 2024-09-14T05:29:49.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567261+0000", 2024-09-14T05:29:49.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2199999999999999e-06 2024-09-14T05:29:49.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567268+0000", 2024-09-14T05:29:49.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2309999999999999e-06 2024-09-14T05:29:49.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567274+0000", 2024-09-14T05:29:49.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.553e-06 2024-09-14T05:29:49.710 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.710 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.710 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.710 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74246 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.710 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569331+0000", 2024-09-14T05:29:49.710 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.030906078, 2024-09-14T05:29:49.710 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.031942283000000002, 2024-09-14T05:29:49.710 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.710 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.710 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.710 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.710 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.711 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.711 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74246 2024-09-14T05:29:49.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.711 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.711 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.711 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569331+0000", 2024-09-14T05:29:49.711 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.711 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.711 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569331+0000", 2024-09-14T05:29:49.711 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.712 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.712 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569331+0000", 2024-09-14T05:29:49.712 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.15e-07 2024-09-14T05:29:49.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.712 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.712 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569333+0000", 2024-09-14T05:29:49.712 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9259999999999999e-06 2024-09-14T05:29:49.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.712 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.713 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569340+0000", 2024-09-14T05:29:49.713 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5239999999999997e-06 2024-09-14T05:29:49.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.713 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.713 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569345+0000", 2024-09-14T05:29:49.713 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5300000000000004e-06 2024-09-14T05:29:49.713 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.713 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.713 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.714 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73809 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.714 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:29:49.714 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040414476999999997, 2024-09-14T05:29:49.714 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.041488108000000003, 2024-09-14T05:29:49.714 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.714 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.714 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.714 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.714 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.714 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.714 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73809 2024-09-14T05:29:49.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.714 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.715 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.715 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:29:49.715 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.715 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.715 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.716 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:29:49.716 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.716 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.716 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.717 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559823+0000", 2024-09-14T05:29:49.717 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.08e-07 2024-09-14T05:29:49.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.717 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.717 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559825+0000", 2024-09-14T05:29:49.717 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.26e-06 2024-09-14T05:29:49.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.718 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.718 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559831+0000", 2024-09-14T05:29:49.718 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6690000000000003e-06 2024-09-14T05:29:49.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.718 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.718 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559835+0000", 2024-09-14T05:29:49.718 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5120000000000002e-06 2024-09-14T05:29:49.718 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.718 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.718 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.719 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74066 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.719 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.567281+0000", 2024-09-14T05:29:49.719 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.032955552999999999, 2024-09-14T05:29:49.719 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.034071800999999999, 2024-09-14T05:29:49.719 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.719 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.719 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.719 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.719 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.719 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.719 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74066 2024-09-14T05:29:49.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.719 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.720 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.720 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567281+0000", 2024-09-14T05:29:49.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.720 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.720 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567281+0000", 2024-09-14T05:29:49.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.720 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.720 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567282+0000", 2024-09-14T05:29:49.721 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6800000000000001e-07 2024-09-14T05:29:49.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.721 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.721 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567284+0000", 2024-09-14T05:29:49.721 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9999999999999999e-06 2024-09-14T05:29:49.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.721 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.721 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567290+0000", 2024-09-14T05:29:49.721 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.37e-06 2024-09-14T05:29:49.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.722 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.722 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567296+0000", 2024-09-14T05:29:49.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5890000000000002e-06 2024-09-14T05:29:49.722 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.722 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.722 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.722 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74250 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.722 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569351+0000", 2024-09-14T05:29:49.722 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.030885590000000001, 2024-09-14T05:29:49.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.032048153000000003, 2024-09-14T05:29:49.723 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.723 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.723 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.723 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.723 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.723 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.723 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74250 2024-09-14T05:29:49.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.723 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.723 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.723 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569351+0000", 2024-09-14T05:29:49.723 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.724 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.724 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569351+0000", 2024-09-14T05:29:49.724 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.724 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.724 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569352+0000", 2024-09-14T05:29:49.724 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0999999999999999e-07 2024-09-14T05:29:49.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.725 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.725 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569354+0000", 2024-09-14T05:29:49.725 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9700000000000002e-06 2024-09-14T05:29:49.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.725 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.725 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.725 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569360+0000", 2024-09-14T05:29:49.725 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0000000000000002e-06 2024-09-14T05:29:49.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.725 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.725 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.725 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569364+0000", 2024-09-14T05:29:49.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8679999999999998e-06 2024-09-14T05:29:49.726 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.726 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.726 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.726 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73822 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.726 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:29:49.726 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040386824000000002, 2024-09-14T05:29:49.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.041596867000000003, 2024-09-14T05:29:49.726 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.726 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.726 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.726 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.727 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.727 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.727 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73822 2024-09-14T05:29:49.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.727 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:29:49.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:29:49.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.729 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.729 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559851+0000", 2024-09-14T05:29:49.729 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4000000000000002e-07 2024-09-14T05:29:49.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559853+0000", 2024-09-14T05:29:49.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5380000000000001e-06 2024-09-14T05:29:49.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559859+0000", 2024-09-14T05:29:49.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0619999999999997e-06 2024-09-14T05:29:49.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.732 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.732 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559864+0000", 2024-09-14T05:29:49.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.977e-06 2024-09-14T05:29:49.732 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.732 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.732 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.733 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74067 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.733 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.567303+0000", 2024-09-14T05:29:49.733 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.032933526999999997, 2024-09-14T05:29:49.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.034190519000000003, 2024-09-14T05:29:49.733 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.734 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.734 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.734 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.734 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.734 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.734 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74067 2024-09-14T05:29:49.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.734 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567303+0000", 2024-09-14T05:29:49.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567303+0000", 2024-09-14T05:29:49.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567304+0000", 2024-09-14T05:29:49.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.34e-07 2024-09-14T05:29:49.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567306+0000", 2024-09-14T05:29:49.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3369999999999998e-06 2024-09-14T05:29:49.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567313+0000", 2024-09-14T05:29:49.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6379999999999998e-06 2024-09-14T05:29:49.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567318+0000", 2024-09-14T05:29:49.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0119999999999996e-06 2024-09-14T05:29:49.737 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.737 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.737 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.737 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74262 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.737 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569375+0000", 2024-09-14T05:29:49.737 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.030861533, 2024-09-14T05:29:49.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.032165276999999999, 2024-09-14T05:29:49.738 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.738 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.738 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.738 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.738 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.738 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.738 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74262 2024-09-14T05:29:49.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.738 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.738 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.738 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569375+0000", 2024-09-14T05:29:49.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569375+0000", 2024-09-14T05:29:49.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569376+0000", 2024-09-14T05:29:49.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.46e-07 2024-09-14T05:29:49.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569378+0000", 2024-09-14T05:29:49.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1840000000000002e-06 2024-09-14T05:29:49.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.740 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.740 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569385+0000", 2024-09-14T05:29:49.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.956e-06 2024-09-14T05:29:49.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.740 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.740 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569390+0000", 2024-09-14T05:29:49.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3669999999999996e-06 2024-09-14T05:29:49.740 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.740 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.740 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.741 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73825 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.741 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:29:49.741 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040363459999999997, 2024-09-14T05:29:49.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.041717563999999999, 2024-09-14T05:29:49.741 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.741 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.741 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.741 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.741 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.741 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.741 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73825 2024-09-14T05:29:49.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.742 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.742 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:29:49.742 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.742 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:29:49.742 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559874+0000", 2024-09-14T05:29:49.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5400000000000002e-07 2024-09-14T05:29:49.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559876+0000", 2024-09-14T05:29:49.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.131e-06 2024-09-14T05:29:49.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559888+0000", 2024-09-14T05:29:49.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2216e-05 2024-09-14T05:29:49.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.744 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559893+0000", 2024-09-14T05:29:49.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6539999999999998e-06 2024-09-14T05:29:49.744 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.744 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.744 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.745 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74069 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.745 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.567325+0000", 2024-09-14T05:29:49.745 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.032912262999999997, 2024-09-14T05:29:49.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.034312122, 2024-09-14T05:29:49.746 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.746 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.746 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.746 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.746 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.746 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.746 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74069 2024-09-14T05:29:49.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.746 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.747 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.747 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567325+0000", 2024-09-14T05:29:49.747 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.747 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.747 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567325+0000", 2024-09-14T05:29:49.747 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.748 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.748 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567325+0000", 2024-09-14T05:29:49.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0900000000000002e-07 2024-09-14T05:29:49.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.748 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567327+0000", 2024-09-14T05:29:49.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.854e-06 2024-09-14T05:29:49.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567333+0000", 2024-09-14T05:29:49.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4760000000000003e-06 2024-09-14T05:29:49.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567338+0000", 2024-09-14T05:29:49.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0239999999999997e-06 2024-09-14T05:29:49.750 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.750 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.750 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.750 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74265 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.750 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569397+0000", 2024-09-14T05:29:49.750 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.030839623, 2024-09-14T05:29:49.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.032286411000000001, 2024-09-14T05:29:49.750 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.750 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.750 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.751 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74265 2024-09-14T05:29:49.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.751 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.751 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.751 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569397+0000", 2024-09-14T05:29:49.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.751 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569397+0000", 2024-09-14T05:29:49.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569398+0000", 2024-09-14T05:29:49.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3399999999999999e-07 2024-09-14T05:29:49.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569400+0000", 2024-09-14T05:29:49.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8479999999999999e-06 2024-09-14T05:29:49.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569406+0000", 2024-09-14T05:29:49.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4679999999999997e-06 2024-09-14T05:29:49.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569411+0000", 2024-09-14T05:29:49.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9429999999999999e-06 2024-09-14T05:29:49.753 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.753 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.753 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.754 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74071 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.754 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.567345+0000", 2024-09-14T05:29:49.754 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.032891974999999997, 2024-09-14T05:29:49.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.034383790999999997, 2024-09-14T05:29:49.754 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.754 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.754 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.754 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.754 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.754 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.754 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74071 2024-09-14T05:29:49.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.755 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.755 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567345+0000", 2024-09-14T05:29:49.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.755 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567345+0000", 2024-09-14T05:29:49.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.756 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.756 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567345+0000", 2024-09-14T05:29:49.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.34e-07 2024-09-14T05:29:49.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.756 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.756 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567347+0000", 2024-09-14T05:29:49.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8050000000000001e-06 2024-09-14T05:29:49.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567353+0000", 2024-09-14T05:29:49.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3990000000000001e-06 2024-09-14T05:29:49.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567358+0000", 2024-09-14T05:29:49.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5909999999999998e-06 2024-09-14T05:29:49.757 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.757 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.757 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.758 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74279 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.758 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571812+0000", 2024-09-14T05:29:49.758 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.028424949000000001, 2024-09-14T05:29:49.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029963724000000001, 2024-09-14T05:29:49.758 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.759 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.759 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.759 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.759 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.759 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.759 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74279 2024-09-14T05:29:49.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.759 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571812+0000", 2024-09-14T05:29:49.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571812+0000", 2024-09-14T05:29:49.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571813+0000", 2024-09-14T05:29:49.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9599999999999998e-07 2024-09-14T05:29:49.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571817+0000", 2024-09-14T05:29:49.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4120000000000003e-06 2024-09-14T05:29:49.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.761 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.761 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571828+0000", 2024-09-14T05:29:49.761 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0832999999999999e-05 2024-09-14T05:29:49.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.761 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.761 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571836+0000", 2024-09-14T05:29:49.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1480000000000006e-06 2024-09-14T05:29:49.762 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.762 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.762 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.763 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74072 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.763 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.567365+0000", 2024-09-14T05:29:49.763 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.032872037999999999, 2024-09-14T05:29:49.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.034459920999999998, 2024-09-14T05:29:49.763 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.763 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.764 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.764 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.764 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.764 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.764 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74072 2024-09-14T05:29:49.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.765 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567365+0000", 2024-09-14T05:29:49.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567365+0000", 2024-09-14T05:29:49.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567365+0000", 2024-09-14T05:29:49.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7999999999999996e-07 2024-09-14T05:29:49.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567368+0000", 2024-09-14T05:29:49.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3020000000000002e-06 2024-09-14T05:29:49.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567374+0000", 2024-09-14T05:29:49.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2949999999999999e-06 2024-09-14T05:29:49.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.769 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567379+0000", 2024-09-14T05:29:49.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5770000000000001e-06 2024-09-14T05:29:49.769 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.769 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.769 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.769 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74283 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.770 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571849+0000", 2024-09-14T05:29:49.770 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.028387652999999999, 2024-09-14T05:29:49.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030035818999999998, 2024-09-14T05:29:49.770 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.770 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.770 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.770 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.770 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.770 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.770 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74283 2024-09-14T05:29:49.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.771 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571849+0000", 2024-09-14T05:29:49.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571849+0000", 2024-09-14T05:29:49.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571850+0000", 2024-09-14T05:29:49.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0200000000000002e-07 2024-09-14T05:29:49.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571852+0000", 2024-09-14T05:29:49.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2879999999999999e-06 2024-09-14T05:29:49.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571859+0000", 2024-09-14T05:29:49.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3860000000000001e-06 2024-09-14T05:29:49.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571865+0000", 2024-09-14T05:29:49.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7180000000000001e-06 2024-09-14T05:29:49.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.774 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.775 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74073 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.775 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.567386+0000", 2024-09-14T05:29:49.775 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.032850737999999997, 2024-09-14T05:29:49.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.034545685, 2024-09-14T05:29:49.775 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.776 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.776 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.776 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.776 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.776 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.776 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74073 2024-09-14T05:29:49.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.777 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567386+0000", 2024-09-14T05:29:49.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567386+0000", 2024-09-14T05:29:49.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567387+0000", 2024-09-14T05:29:49.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6400000000000003e-07 2024-09-14T05:29:49.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.780 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567389+0000", 2024-09-14T05:29:49.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.978e-06 2024-09-14T05:29:49.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567395+0000", 2024-09-14T05:29:49.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3899999999999998e-06 2024-09-14T05:29:49.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567400+0000", 2024-09-14T05:29:49.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5659999999999997e-06 2024-09-14T05:29:49.781 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.781 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.781 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.782 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74294 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.782 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571872+0000", 2024-09-14T05:29:49.782 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.028364759, 2024-09-14T05:29:49.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030108016000000001, 2024-09-14T05:29:49.782 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.782 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.782 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.782 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.782 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.782 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.782 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74294 2024-09-14T05:29:49.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.783 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571872+0000", 2024-09-14T05:29:49.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571872+0000", 2024-09-14T05:29:49.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571873+0000", 2024-09-14T05:29:49.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3599999999999999e-07 2024-09-14T05:29:49.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571874+0000", 2024-09-14T05:29:49.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.84e-06 2024-09-14T05:29:49.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571887+0000", 2024-09-14T05:29:49.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2323e-05 2024-09-14T05:29:49.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571893+0000", 2024-09-14T05:29:49.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.161e-06 2024-09-14T05:29:49.785 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.785 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.785 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.785 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74074 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.785 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.567407+0000", 2024-09-14T05:29:49.785 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.032829349000000001, 2024-09-14T05:29:49.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.034620206000000001, 2024-09-14T05:29:49.786 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.786 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.786 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.786 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.786 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.786 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.786 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74074 2024-09-14T05:29:49.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.786 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567407+0000", 2024-09-14T05:29:49.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567407+0000", 2024-09-14T05:29:49.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567408+0000", 2024-09-14T05:29:49.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5200000000000002e-07 2024-09-14T05:29:49.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567410+0000", 2024-09-14T05:29:49.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9300000000000002e-06 2024-09-14T05:29:49.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567416+0000", 2024-09-14T05:29:49.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1809999999999998e-06 2024-09-14T05:29:49.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567421+0000", 2024-09-14T05:29:49.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4380000000000003e-06 2024-09-14T05:29:49.792 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.792 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.792 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.792 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74297 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.793 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571900+0000", 2024-09-14T05:29:49.793 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.028336597000000002, 2024-09-14T05:29:49.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030173266000000001, 2024-09-14T05:29:49.793 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.793 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.793 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.793 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.793 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.793 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.793 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74297 2024-09-14T05:29:49.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.794 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571900+0000", 2024-09-14T05:29:49.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571900+0000", 2024-09-14T05:29:49.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571901+0000", 2024-09-14T05:29:49.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9399999999999995e-07 2024-09-14T05:29:49.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571903+0000", 2024-09-14T05:29:49.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0760000000000001e-06 2024-09-14T05:29:49.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571909+0000", 2024-09-14T05:29:49.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5259999999999999e-06 2024-09-14T05:29:49.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571915+0000", 2024-09-14T05:29:49.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9030000000000001e-06 2024-09-14T05:29:49.796 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.796 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.796 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.796 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74075 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.796 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.567428+0000", 2024-09-14T05:29:49.796 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.032809261999999999, 2024-09-14T05:29:49.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.034692694000000003, 2024-09-14T05:29:49.797 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.797 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.797 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.797 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.797 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.797 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.797 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74075 2024-09-14T05:29:49.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.797 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567428+0000", 2024-09-14T05:29:49.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567428+0000", 2024-09-14T05:29:49.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567428+0000", 2024-09-14T05:29:49.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9299999999999999e-07 2024-09-14T05:29:49.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567430+0000", 2024-09-14T05:29:49.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1169999999999998e-06 2024-09-14T05:29:49.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567436+0000", 2024-09-14T05:29:49.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3459999999999999e-06 2024-09-14T05:29:49.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567442+0000", 2024-09-14T05:29:49.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5489999999999997e-06 2024-09-14T05:29:49.800 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.800 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.800 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.800 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74310 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.800 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571922+0000", 2024-09-14T05:29:49.800 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.028314819000000001, 2024-09-14T05:29:49.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030242787, 2024-09-14T05:29:49.800 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.800 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.800 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.800 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.801 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.801 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.801 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74310 2024-09-14T05:29:49.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.801 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571922+0000", 2024-09-14T05:29:49.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571922+0000", 2024-09-14T05:29:49.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571922+0000", 2024-09-14T05:29:49.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.65e-07 2024-09-14T05:29:49.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571925+0000", 2024-09-14T05:29:49.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2709999999999999e-06 2024-09-14T05:29:49.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571931+0000", 2024-09-14T05:29:49.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4339999999999997e-06 2024-09-14T05:29:49.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571936+0000", 2024-09-14T05:29:49.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1950000000000002e-06 2024-09-14T05:29:49.803 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.803 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.803 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.803 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74076 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.804 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.567448+0000", 2024-09-14T05:29:49.804 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.032788689000000003, 2024-09-14T05:29:49.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.034761136999999998, 2024-09-14T05:29:49.804 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.804 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.804 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.804 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.804 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.804 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.804 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74076 2024-09-14T05:29:49.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.804 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567448+0000", 2024-09-14T05:29:49.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567448+0000", 2024-09-14T05:29:49.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567449+0000", 2024-09-14T05:29:49.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1699999999999999e-07 2024-09-14T05:29:49.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567450+0000", 2024-09-14T05:29:49.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8610000000000001e-06 2024-09-14T05:29:49.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567457+0000", 2024-09-14T05:29:49.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3539999999999996e-06 2024-09-14T05:29:49.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567462+0000", 2024-09-14T05:29:49.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1109999999999999e-06 2024-09-14T05:29:49.807 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.807 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.807 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.807 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74313 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.807 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571943+0000", 2024-09-14T05:29:49.807 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.028293944000000001, 2024-09-14T05:29:49.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030312602000000001, 2024-09-14T05:29:49.807 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.807 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.807 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.807 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.808 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.808 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.808 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74313 2024-09-14T05:29:49.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.808 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.808 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.808 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571943+0000", 2024-09-14T05:29:49.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.808 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.808 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571943+0000", 2024-09-14T05:29:49.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571943+0000", 2024-09-14T05:29:49.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.63e-07 2024-09-14T05:29:49.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571945+0000", 2024-09-14T05:29:49.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9599999999999999e-06 2024-09-14T05:29:49.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571953+0000", 2024-09-14T05:29:49.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8869999999999992e-06 2024-09-14T05:29:49.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571959+0000", 2024-09-14T05:29:49.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.429e-06 2024-09-14T05:29:49.810 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.810 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.810 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.811 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74097 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.811 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.567482+0000", 2024-09-14T05:29:49.811 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.032754867, 2024-09-14T05:29:49.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.034817656000000002, 2024-09-14T05:29:49.811 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.811 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.811 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.811 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.811 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.811 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.811 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74097 2024-09-14T05:29:49.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.811 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567482+0000", 2024-09-14T05:29:49.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567482+0000", 2024-09-14T05:29:49.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567483+0000", 2024-09-14T05:29:49.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8800000000000002e-07 2024-09-14T05:29:49.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567485+0000", 2024-09-14T05:29:49.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.401e-06 2024-09-14T05:29:49.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567492+0000", 2024-09-14T05:29:49.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5179999999999998e-06 2024-09-14T05:29:49.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.567498+0000", 2024-09-14T05:29:49.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7679999999999997e-06 2024-09-14T05:29:49.814 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.814 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.814 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.814 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74327 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.814 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571966+0000", 2024-09-14T05:29:49.814 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.028270792999999999, 2024-09-14T05:29:49.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030380409000000001, 2024-09-14T05:29:49.814 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.814 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.815 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.815 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.815 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.815 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.815 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74327 2024-09-14T05:29:49.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.815 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571966+0000", 2024-09-14T05:29:49.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571966+0000", 2024-09-14T05:29:49.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.817 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571966+0000", 2024-09-14T05:29:49.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3300000000000003e-07 2024-09-14T05:29:49.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.817 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571969+0000", 2024-09-14T05:29:49.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1409999999999999e-06 2024-09-14T05:29:49.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571976+0000", 2024-09-14T05:29:49.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2960000000000001e-06 2024-09-14T05:29:49.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571982+0000", 2024-09-14T05:29:49.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8209999999999998e-06 2024-09-14T05:29:49.819 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.819 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.819 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.819 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74101 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.819 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568845+0000", 2024-09-14T05:29:49.819 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.031391760999999997, 2024-09-14T05:29:49.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.033546765999999999, 2024-09-14T05:29:49.819 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.819 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.820 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.820 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.820 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.820 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.820 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74101 2024-09-14T05:29:49.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.820 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568845+0000", 2024-09-14T05:29:49.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568845+0000", 2024-09-14T05:29:49.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568846+0000", 2024-09-14T05:29:49.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2699999999999998e-07 2024-09-14T05:29:49.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568850+0000", 2024-09-14T05:29:49.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.963e-06 2024-09-14T05:29:49.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568865+0000", 2024-09-14T05:29:49.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5268e-05 2024-09-14T05:29:49.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568873+0000", 2024-09-14T05:29:49.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5839999999999997e-06 2024-09-14T05:29:49.822 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.823 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.823 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.823 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74329 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.823 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571988+0000", 2024-09-14T05:29:49.823 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.028248678999999999, 2024-09-14T05:29:49.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030448589000000002, 2024-09-14T05:29:49.823 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.823 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.823 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.823 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.823 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.823 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.824 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74329 2024-09-14T05:29:49.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.824 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571988+0000", 2024-09-14T05:29:49.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571988+0000", 2024-09-14T05:29:49.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571989+0000", 2024-09-14T05:29:49.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9900000000000001e-07 2024-09-14T05:29:49.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571990+0000", 2024-09-14T05:29:49.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9240000000000001e-06 2024-09-14T05:29:49.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571997+0000", 2024-09-14T05:29:49.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3849999999999999e-06 2024-09-14T05:29:49.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572002+0000", 2024-09-14T05:29:49.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2100000000000001e-06 2024-09-14T05:29:49.826 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.826 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.826 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.826 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74343 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.826 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572008+0000", 2024-09-14T05:29:49.826 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.028228386000000001, 2024-09-14T05:29:49.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030475584, 2024-09-14T05:29:49.827 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.827 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.827 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.827 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.827 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.827 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.827 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74343 2024-09-14T05:29:49.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.827 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572008+0000", 2024-09-14T05:29:49.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572008+0000", 2024-09-14T05:29:49.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572009+0000", 2024-09-14T05:29:49.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6399999999999998e-07 2024-09-14T05:29:49.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572011+0000", 2024-09-14T05:29:49.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8929999999999999e-06 2024-09-14T05:29:49.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572017+0000", 2024-09-14T05:29:49.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2999999999999998e-06 2024-09-14T05:29:49.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572022+0000", 2024-09-14T05:29:49.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.215e-06 2024-09-14T05:29:49.829 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.830 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.830 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.830 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74346 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.830 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572029+0000", 2024-09-14T05:29:49.830 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.028208130000000001, 2024-09-14T05:29:49.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030502497999999999, 2024-09-14T05:29:49.830 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.830 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.830 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.830 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.830 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.831 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.831 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74346 2024-09-14T05:29:49.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.831 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572029+0000", 2024-09-14T05:29:49.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572029+0000", 2024-09-14T05:29:49.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572029+0000", 2024-09-14T05:29:49.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0499999999999999e-07 2024-09-14T05:29:49.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572031+0000", 2024-09-14T05:29:49.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8330000000000001e-06 2024-09-14T05:29:49.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572037+0000", 2024-09-14T05:29:49.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3289999999999999e-06 2024-09-14T05:29:49.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572042+0000", 2024-09-14T05:29:49.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1229999999999999e-06 2024-09-14T05:29:49.833 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.833 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.833 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.833 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73664 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 208896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.834 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559329+0000", 2024-09-14T05:29:49.834 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040907788, 2024-09-14T05:29:49.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.043253433000000001, 2024-09-14T05:29:49.834 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.834 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.834 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:49.834 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.834 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.834 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.834 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73664 2024-09-14T05:29:49.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.834 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559329+0000", 2024-09-14T05:29:49.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559329+0000", 2024-09-14T05:29:49.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559330+0000", 2024-09-14T05:29:49.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9299999999999998e-07 2024-09-14T05:29:49.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559332+0000", 2024-09-14T05:29:49.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2850000000000001e-06 2024-09-14T05:29:49.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559338+0000", 2024-09-14T05:29:49.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3289999999999999e-06 2024-09-14T05:29:49.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559343+0000", 2024-09-14T05:29:49.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0139999999999998e-06 2024-09-14T05:29:49.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:49.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.602466+0000", 2024-09-14T05:29:49.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.043122471000000002 2024-09-14T05:29:49.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:49.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.602516+0000", 2024-09-14T05:29:49.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0640999999999998e-05 2024-09-14T05:29:49.838 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.839 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.839 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.839 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74359 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.839 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572049+0000", 2024-09-14T05:29:49.839 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.028187521, 2024-09-14T05:29:49.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030587684, 2024-09-14T05:29:49.839 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.839 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.839 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.839 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.839 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.839 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.840 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74359 2024-09-14T05:29:49.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.840 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572049+0000", 2024-09-14T05:29:49.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572049+0000", 2024-09-14T05:29:49.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572050+0000", 2024-09-14T05:29:49.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0699999999999998e-07 2024-09-14T05:29:49.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572052+0000", 2024-09-14T05:29:49.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.841e-06 2024-09-14T05:29:49.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572058+0000", 2024-09-14T05:29:49.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.297e-06 2024-09-14T05:29:49.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572063+0000", 2024-09-14T05:29:49.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1050000000000003e-06 2024-09-14T05:29:49.842 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.842 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.842 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.842 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73665 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.842 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559349+0000", 2024-09-14T05:29:49.843 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040887350000000003, 2024-09-14T05:29:49.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.04332109, 2024-09-14T05:29:49.843 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.843 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.843 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.843 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.843 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.843 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.843 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73665 2024-09-14T05:29:49.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.843 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559349+0000", 2024-09-14T05:29:49.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559349+0000", 2024-09-14T05:29:49.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559350+0000", 2024-09-14T05:29:49.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8299999999999998e-07 2024-09-14T05:29:49.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559352+0000", 2024-09-14T05:29:49.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7489999999999999e-06 2024-09-14T05:29:49.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559357+0000", 2024-09-14T05:29:49.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7919999999999998e-06 2024-09-14T05:29:49.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559362+0000", 2024-09-14T05:29:49.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5970000000000002e-06 2024-09-14T05:29:49.846 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.846 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.846 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.846 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74361 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.846 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572069+0000", 2024-09-14T05:29:49.846 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.028167665000000001, 2024-09-14T05:29:49.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030639765999999999, 2024-09-14T05:29:49.846 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.846 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.846 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.846 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.847 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.847 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.847 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74361 2024-09-14T05:29:49.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.847 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572069+0000", 2024-09-14T05:29:49.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572069+0000", 2024-09-14T05:29:49.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572070+0000", 2024-09-14T05:29:49.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7899999999999999e-07 2024-09-14T05:29:49.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572076+0000", 2024-09-14T05:29:49.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0639999999999999e-06 2024-09-14T05:29:49.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572082+0000", 2024-09-14T05:29:49.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3550000000000001e-06 2024-09-14T05:29:49.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572087+0000", 2024-09-14T05:29:49.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3539999999999999e-06 2024-09-14T05:29:49.850 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.850 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.850 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.851 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73666 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.851 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559368+0000", 2024-09-14T05:29:49.851 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040869250000000003, 2024-09-14T05:29:49.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.043388707999999998, 2024-09-14T05:29:49.851 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.852 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.852 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.852 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.852 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.852 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.853 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73666 2024-09-14T05:29:49.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.853 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559368+0000", 2024-09-14T05:29:49.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559368+0000", 2024-09-14T05:29:49.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559368+0000", 2024-09-14T05:29:49.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7099999999999997e-07 2024-09-14T05:29:49.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559370+0000", 2024-09-14T05:29:49.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.734e-06 2024-09-14T05:29:49.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559375+0000", 2024-09-14T05:29:49.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.665e-06 2024-09-14T05:29:49.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559382+0000", 2024-09-14T05:29:49.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2990000000000002e-06 2024-09-14T05:29:49.858 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.858 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.858 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.858 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74374 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.859 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572248+0000", 2024-09-14T05:29:49.859 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.027988516000000001, 2024-09-14T05:29:49.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030554122999999999, 2024-09-14T05:29:49.859 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.859 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.860 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.860 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.860 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.860 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.860 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74374 2024-09-14T05:29:49.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.861 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572248+0000", 2024-09-14T05:29:49.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572248+0000", 2024-09-14T05:29:49.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572249+0000", 2024-09-14T05:29:49.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1699999999999997e-07 2024-09-14T05:29:49.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572253+0000", 2024-09-14T05:29:49.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6160000000000002e-06 2024-09-14T05:29:49.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572263+0000", 2024-09-14T05:29:49.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9639999999999998e-06 2024-09-14T05:29:49.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572270+0000", 2024-09-14T05:29:49.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9999999999999999e-06 2024-09-14T05:29:49.865 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.865 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.866 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.866 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73667 3.e 3.2e88b86e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.866 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559387+0000", 2024-09-14T05:29:49.867 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040849819000000002, 2024-09-14T05:29:49.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.043461099000000003, 2024-09-14T05:29:49.867 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.867 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.867 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.867 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.867 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.868 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.868 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73667 2024-09-14T05:29:49.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.868 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559387+0000", 2024-09-14T05:29:49.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.870 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.870 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559387+0000", 2024-09-14T05:29:49.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.870 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.871 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559387+0000", 2024-09-14T05:29:49.871 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6800000000000001e-07 2024-09-14T05:29:49.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.871 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.871 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559389+0000", 2024-09-14T05:29:49.871 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.759e-06 2024-09-14T05:29:49.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.872 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559395+0000", 2024-09-14T05:29:49.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6699999999999999e-06 2024-09-14T05:29:49.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.872 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559400+0000", 2024-09-14T05:29:49.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0359999999999997e-06 2024-09-14T05:29:49.872 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.872 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.872 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.873 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74377 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.873 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572281+0000", 2024-09-14T05:29:49.873 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.027956083, 2024-09-14T05:29:49.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030620390000000001, 2024-09-14T05:29:49.873 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.873 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.873 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.873 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.873 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.873 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.873 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74377 2024-09-14T05:29:49.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.874 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572281+0000", 2024-09-14T05:29:49.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572281+0000", 2024-09-14T05:29:49.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.875 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572281+0000", 2024-09-14T05:29:49.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3600000000000004e-07 2024-09-14T05:29:49.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.875 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.875 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572283+0000", 2024-09-14T05:29:49.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1689999999999999e-06 2024-09-14T05:29:49.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.875 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.875 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572290+0000", 2024-09-14T05:29:49.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.708e-06 2024-09-14T05:29:49.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572296+0000", 2024-09-14T05:29:49.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5110000000000003e-06 2024-09-14T05:29:49.876 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.876 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.876 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.876 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73685 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.876 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559405+0000", 2024-09-14T05:29:49.876 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040831635999999998, 2024-09-14T05:29:49.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.043543723999999999, 2024-09-14T05:29:49.876 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.877 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.877 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.877 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.877 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.877 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.877 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73685 2024-09-14T05:29:49.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.877 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559405+0000", 2024-09-14T05:29:49.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559405+0000", 2024-09-14T05:29:49.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559406+0000", 2024-09-14T05:29:49.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2300000000000002e-07 2024-09-14T05:29:49.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559407+0000", 2024-09-14T05:29:49.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7069999999999999e-06 2024-09-14T05:29:49.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559413+0000", 2024-09-14T05:29:49.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6790000000000002e-06 2024-09-14T05:29:49.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559418+0000", 2024-09-14T05:29:49.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.668e-06 2024-09-14T05:29:49.879 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.879 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.880 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.880 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74390 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.880 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.573591+0000", 2024-09-14T05:29:49.880 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.026645894999999999, 2024-09-14T05:29:49.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029403991000000001, 2024-09-14T05:29:49.880 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.880 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.880 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.880 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.881 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.881 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.881 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74390 2024-09-14T05:29:49.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.881 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.573591+0000", 2024-09-14T05:29:49.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.573591+0000", 2024-09-14T05:29:49.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.573592+0000", 2024-09-14T05:29:49.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1800000000000005e-07 2024-09-14T05:29:49.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.573596+0000", 2024-09-14T05:29:49.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9600000000000002e-06 2024-09-14T05:29:49.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.573606+0000", 2024-09-14T05:29:49.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0149000000000001e-05 2024-09-14T05:29:49.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.573632+0000", 2024-09-14T05:29:49.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6171999999999999e-05 2024-09-14T05:29:49.883 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.883 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.883 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.884 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73688 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.884 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559423+0000", 2024-09-14T05:29:49.884 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040813617000000003, 2024-09-14T05:29:49.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.043618170999999997, 2024-09-14T05:29:49.884 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.884 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.884 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.884 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.884 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.884 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.884 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73688 2024-09-14T05:29:49.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.885 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559423+0000", 2024-09-14T05:29:49.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559423+0000", 2024-09-14T05:29:49.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559424+0000", 2024-09-14T05:29:49.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3000000000000002e-07 2024-09-14T05:29:49.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559425+0000", 2024-09-14T05:29:49.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.618e-06 2024-09-14T05:29:49.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559431+0000", 2024-09-14T05:29:49.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5480000000000001e-06 2024-09-14T05:29:49.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559435+0000", 2024-09-14T05:29:49.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.69e-06 2024-09-14T05:29:49.887 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.887 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.887 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.888 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74393 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.888 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.573644+0000", 2024-09-14T05:29:49.888 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.026592378999999999, 2024-09-14T05:29:49.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029433165000000001, 2024-09-14T05:29:49.888 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.889 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.889 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.889 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.889 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.889 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.889 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74393 2024-09-14T05:29:49.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.889 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.573644+0000", 2024-09-14T05:29:49.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.573644+0000", 2024-09-14T05:29:49.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.573645+0000", 2024-09-14T05:29:49.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2799999999999996e-07 2024-09-14T05:29:49.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.573648+0000", 2024-09-14T05:29:49.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9399999999999998e-06 2024-09-14T05:29:49.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.573656+0000", 2024-09-14T05:29:49.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6399999999999997e-06 2024-09-14T05:29:49.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.573662+0000", 2024-09-14T05:29:49.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1539999999999999e-06 2024-09-14T05:29:49.891 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.892 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.892 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.892 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73707 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.892 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559520+0000", 2024-09-14T05:29:49.892 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040717063999999997, 2024-09-14T05:29:49.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.043593267999999998, 2024-09-14T05:29:49.892 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.892 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.892 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.892 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.892 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.893 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.893 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73707 2024-09-14T05:29:49.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.893 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559520+0000", 2024-09-14T05:29:49.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559520+0000", 2024-09-14T05:29:49.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559520+0000", 2024-09-14T05:29:49.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1399999999999997e-07 2024-09-14T05:29:49.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559523+0000", 2024-09-14T05:29:49.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5299999999999999e-06 2024-09-14T05:29:49.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559530+0000", 2024-09-14T05:29:49.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4780000000000002e-06 2024-09-14T05:29:49.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559536+0000", 2024-09-14T05:29:49.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5670000000000002e-06 2024-09-14T05:29:49.895 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.895 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.895 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.896 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74023 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.896 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:29:49.896 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.039177484999999998, 2024-09-14T05:29:49.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.042100414000000003, 2024-09-14T05:29:49.896 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.896 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.896 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.896 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.896 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.897 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.897 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74023 2024-09-14T05:29:49.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.897 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:29:49.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:29:49.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561060+0000", 2024-09-14T05:29:49.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0699999999999998e-07 2024-09-14T05:29:49.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561072+0000", 2024-09-14T05:29:49.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2261e-05 2024-09-14T05:29:49.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561079+0000", 2024-09-14T05:29:49.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8399999999999997e-06 2024-09-14T05:29:49.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561084+0000", 2024-09-14T05:29:49.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7380000000000002e-06 2024-09-14T05:29:49.899 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.899 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.899 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.900 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73711 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.900 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559543+0000", 2024-09-14T05:29:49.900 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040693359999999998, 2024-09-14T05:29:49.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.043661577999999999, 2024-09-14T05:29:49.900 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.900 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.900 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.900 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.900 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.901 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.901 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73711 2024-09-14T05:29:49.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.901 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559543+0000", 2024-09-14T05:29:49.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559543+0000", 2024-09-14T05:29:49.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559544+0000", 2024-09-14T05:29:49.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2899999999999999e-07 2024-09-14T05:29:49.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559547+0000", 2024-09-14T05:29:49.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.23e-06 2024-09-14T05:29:49.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.903 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559553+0000", 2024-09-14T05:29:49.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8459999999999996e-06 2024-09-14T05:29:49.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.903 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.903 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559558+0000", 2024-09-14T05:29:49.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9119999999999997e-06 2024-09-14T05:29:49.903 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.903 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.903 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.903 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74026 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.904 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:29:49.904 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.039146023000000002, 2024-09-14T05:29:49.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.042161819000000003, 2024-09-14T05:29:49.904 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.904 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.904 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.904 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.904 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.904 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.904 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74026 2024-09-14T05:29:49.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.904 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:29:49.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:29:49.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:29:49.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:29:49.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561093+0000", 2024-09-14T05:29:49.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-14T05:29:49.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561099+0000", 2024-09-14T05:29:49.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0399999999999998e-06 2024-09-14T05:29:49.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561104+0000", 2024-09-14T05:29:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8180000000000003e-06 2024-09-14T05:29:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73724 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559564+0000", 2024-09-14T05:29:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040673115000000003, 2024-09-14T05:29:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.043737019000000002, 2024-09-14T05:29:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.907 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73724 2024-09-14T05:29:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559564+0000", 2024-09-14T05:29:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559564+0000", 2024-09-14T05:29:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559564+0000", 2024-09-14T05:29:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.46e-07 2024-09-14T05:29:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559566+0000", 2024-09-14T05:29:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.81e-06 2024-09-14T05:29:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559572+0000", 2024-09-14T05:29:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1099999999999999e-06 2024-09-14T05:29:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559577+0000", 2024-09-14T05:29:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8840000000000002e-06 2024-09-14T05:29:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74039 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:29:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.039019425000000003, 2024-09-14T05:29:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.042131334999999999, 2024-09-14T05:29:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74039 2024-09-14T05:29:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:29:49.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:29:49.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561218+0000", 2024-09-14T05:29:49.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4000000000000001e-07 2024-09-14T05:29:49.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561221+0000", 2024-09-14T05:29:49.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9730000000000002e-06 2024-09-14T05:29:49.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561229+0000", 2024-09-14T05:29:49.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1759999999999993e-06 2024-09-14T05:29:49.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.914 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561235+0000", 2024-09-14T05:29:49.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0109999999999997e-06 2024-09-14T05:29:49.914 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.915 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.915 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.915 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74145 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.915 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569098+0000", 2024-09-14T05:29:49.915 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.031139204, 2024-09-14T05:29:49.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.034296786000000003, 2024-09-14T05:29:49.915 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.915 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.915 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.916 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.916 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.916 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.916 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74145 2024-09-14T05:29:49.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.916 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569098+0000", 2024-09-14T05:29:49.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569098+0000", 2024-09-14T05:29:49.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569098+0000", 2024-09-14T05:29:49.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4200000000000001e-07 2024-09-14T05:29:49.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569113+0000", 2024-09-14T05:29:49.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4965e-05 2024-09-14T05:29:49.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569121+0000", 2024-09-14T05:29:49.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5290000000000003e-06 2024-09-14T05:29:49.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569126+0000", 2024-09-14T05:29:49.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3789999999999997e-06 2024-09-14T05:29:49.919 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.919 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.919 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.919 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73727 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.919 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559583+0000", 2024-09-14T05:29:49.919 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.040654175000000001, 2024-09-14T05:29:49.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.043945703000000003, 2024-09-14T05:29:49.919 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.919 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.920 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.920 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.920 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.920 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.920 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73727 2024-09-14T05:29:49.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.921 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559583+0000", 2024-09-14T05:29:49.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.922 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.922 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559583+0000", 2024-09-14T05:29:49.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559583+0000", 2024-09-14T05:29:49.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.96e-07 2024-09-14T05:29:49.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559585+0000", 2024-09-14T05:29:49.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8870000000000001e-06 2024-09-14T05:29:49.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.924 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559609+0000", 2024-09-14T05:29:49.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3782e-05 2024-09-14T05:29:49.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559617+0000", 2024-09-14T05:29:49.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5769999999999994e-06 2024-09-14T05:29:49.926 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.926 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.926 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.927 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74042 3.16 3.f4eb68d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:49.927 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:29:49.927 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.038970972, 2024-09-14T05:29:49.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.042311449000000001, 2024-09-14T05:29:49.927 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:49.927 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:49.927 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "queued for pg", 2024-09-14T05:29:49.927 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:49.927 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:49.928 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:49.928 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74042 2024-09-14T05:29:49.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.928 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:49.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:49.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:29:49.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:49.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:29:49.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:49.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:49.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:29:49.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4099999999999998e-07 2024-09-14T05:29:49.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:49.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561269+0000", 2024-09-14T05:29:49.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7930000000000002e-06 2024-09-14T05:29:49.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:49.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561278+0000", 2024-09-14T05:29:49.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5150000000000006e-06 2024-09-14T05:29:49.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:49.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:49.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:49.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561284+0000", 2024-09-14T05:29:49.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2029999999999997e-06 2024-09-14T05:29:49.930 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.930 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:49.930 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.930 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:49.930 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:29:49.930 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 70 2024-09-14T05:29:49.930 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:49.931 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T05:29:49.931 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T05:29:49.931 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T05:29:49.932 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T05:29:49.933 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:49.931+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:49.934 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:29:49.934 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-14T05:29:49.935 INFO:teuthology.orchestra.run.smithi071.stdout:1708 read 0x1d501a7 thru 0x1d55a30 (0x588a bytes) 2024-09-14T05:29:49.935 INFO:teuthology.orchestra.run.smithi071.stdout:1714 punch from 0x9d8abcd to 0x9d93a5e, (0x8e91 bytes) 2024-09-14T05:29:49.935 INFO:teuthology.orchestra.run.smithi071.stdout:1715 trunc from 0xe5ee4e6 to 0xce489d4 2024-09-14T05:29:49.935 INFO:teuthology.orchestra.run.smithi071.stdout:1716 write 0xdd12201 thru 0xdd2066b (0xe46b bytes) 2024-09-14T05:29:49.935 INFO:teuthology.orchestra.run.smithi071.stdout:1717 write 0x2575d22 thru 0x257f4e4 (0x97c3 bytes) 2024-09-14T05:29:49.935 INFO:teuthology.orchestra.run.smithi071.stdout:1720 writesame 0x5e31e7 thru 0x5f0023 data_size 0x1(0xce3d bytes) 2024-09-14T05:29:50.032 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:50.032+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:50.079 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:50.079 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:50.079 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:50.080 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:50.080 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:50.088 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:29:50.133 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:50.133+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:50.150 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-14T05:29:50.161 INFO:teuthology.orchestra.run.smithi071.stdout:1724 trunc from 0xdd2066c to 0xbcfac30 2024-09-14T05:29:50.199 INFO:teuthology.orchestra.run.smithi071.stdout:1725 read 0x3442257 thru 0x344de9b (0xbc45 bytes) 2024-09-14T05:29:50.199 INFO:teuthology.orchestra.run.smithi071.stdout:1726 write 0xa4b119c thru 0xa4c0df1 (0xfc56 bytes) 2024-09-14T05:29:50.219 INFO:teuthology.orchestra.run.smithi071.stdout:1727 read 0x24b44a5 thru 0x24b601d (0x1b79 bytes) 2024-09-14T05:29:50.224 INFO:teuthology.orchestra.run.smithi071.stdout:1728 trunc from 0xbcfac30 to 0x856849b 2024-09-14T05:29:50.234 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:50.232+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:50.239 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:50.239 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:50.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:50.239 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:50.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.239 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73757 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2816000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.239 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:29:50.239 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.67706104700000003, 2024-09-14T05:29:50.240 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044614540000000001, 2024-09-14T05:29:50.240 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.240 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.240 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.240 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.240 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.240 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.240 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73757 2024-09-14T05:29:50.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.240 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:29:50.240 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.241 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.241 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:29:50.241 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.241 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.241 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559679+0000", 2024-09-14T05:29:50.241 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4000000000000002e-07 2024-09-14T05:29:50.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.242 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.242 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559681+0000", 2024-09-14T05:29:50.242 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1780000000000002e-06 2024-09-14T05:29:50.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.242 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.242 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559687+0000", 2024-09-14T05:29:50.242 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0270000000000001e-06 2024-09-14T05:29:50.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.242 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.243 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559692+0000", 2024-09-14T05:29:50.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1030000000000001e-06 2024-09-14T05:29:50.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.243 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.243 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604180+0000", 2024-09-14T05:29:50.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044487480000000003 2024-09-14T05:29:50.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.243 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.243 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604224+0000", 2024-09-14T05:29:50.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4357999999999998e-05 2024-09-14T05:29:50.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604293+0000", 2024-09-14T05:29:50.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8954000000000004e-05 2024-09-14T05:29:50.244 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.244 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.244 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.244 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73760 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2848768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.244 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:29:50.244 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.67704094199999998, 2024-09-14T05:29:50.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.04469911, 2024-09-14T05:29:50.244 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.244 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.245 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.245 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.245 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.245 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.245 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73760 2024-09-14T05:29:50.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.245 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.245 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:29:50.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:29:50.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.246 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:29:50.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:29:50.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.246 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559701+0000", 2024-09-14T05:29:50.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1909999999999999e-06 2024-09-14T05:29:50.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.247 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.247 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559708+0000", 2024-09-14T05:29:50.247 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7279999999999998e-06 2024-09-14T05:29:50.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.247 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.247 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559713+0000", 2024-09-14T05:29:50.247 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8389999999999997e-06 2024-09-14T05:29:50.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.247 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.247 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604310+0000", 2024-09-14T05:29:50.247 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044597664000000002 2024-09-14T05:29:50.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.248 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.248 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604352+0000", 2024-09-14T05:29:50.248 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1236000000000003e-05 2024-09-14T05:29:50.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.248 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.248 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604398+0000", 2024-09-14T05:29:50.248 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6091000000000003e-05 2024-09-14T05:29:50.248 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.248 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.248 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.248 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73773 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2801664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.249 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:29:50.249 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.67702077299999996, 2024-09-14T05:29:50.249 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941026000000002, 2024-09-14T05:29:50.249 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.249 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.249 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.249 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.249 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.250 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.250 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73773 2024-09-14T05:29:50.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.250 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.250 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.250 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:29:50.250 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.250 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.250 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:29:50.250 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.251 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.251 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:29:50.251 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8599999999999999e-07 2024-09-14T05:29:50.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.251 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.251 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559721+0000", 2024-09-14T05:29:50.251 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0540000000000002e-06 2024-09-14T05:29:50.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.252 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559729+0000", 2024-09-14T05:29:50.252 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1359999999999997e-06 2024-09-14T05:29:50.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.252 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559734+0000", 2024-09-14T05:29:50.252 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2140000000000004e-06 2024-09-14T05:29:50.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.252 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604441+0000", 2024-09-14T05:29:50.252 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044706828999999997 2024-09-14T05:29:50.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.253 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.253 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604522+0000", 2024-09-14T05:29:50.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0603000000000001e-05 2024-09-14T05:29:50.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.253 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.253 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604660+0000", 2024-09-14T05:29:50.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000137804 2024-09-14T05:29:50.253 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.253 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.253 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.253 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73777 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2834432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.254 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:29:50.254 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.67699837200000001, 2024-09-14T05:29:50.254 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045116270999999999, 2024-09-14T05:29:50.254 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.254 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.254 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.254 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.254 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.254 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.254 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73777 2024-09-14T05:29:50.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.254 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.255 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.255 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:29:50.255 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.255 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.255 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:29:50.255 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.255 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.255 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559742+0000", 2024-09-14T05:29:50.255 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4099999999999999e-07 2024-09-14T05:29:50.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.256 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.256 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559744+0000", 2024-09-14T05:29:50.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0040000000000002e-06 2024-09-14T05:29:50.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.256 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.256 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559749+0000", 2024-09-14T05:29:50.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9739999999999999e-06 2024-09-14T05:29:50.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.256 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.256 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559754+0000", 2024-09-14T05:29:50.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9540000000000003e-06 2024-09-14T05:29:50.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.257 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604695+0000", 2024-09-14T05:29:50.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941003 2024-09-14T05:29:50.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.257 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604766+0000", 2024-09-14T05:29:50.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0296999999999995e-05 2024-09-14T05:29:50.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.257 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604857+0000", 2024-09-14T05:29:50.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1598e-05 2024-09-14T05:29:50.258 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.258 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.258 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.258 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73790 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.258 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:29:50.258 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.67697843700000004, 2024-09-14T05:29:50.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316437000000001, 2024-09-14T05:29:50.258 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.258 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.258 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.258 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.259 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.259 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.259 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73790 2024-09-14T05:29:50.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.259 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.259 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.259 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:29:50.259 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.260 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:29:50.260 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.260 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:29:50.260 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7599999999999998e-07 2024-09-14T05:29:50.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.260 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559763+0000", 2024-09-14T05:29:50.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0880000000000002e-06 2024-09-14T05:29:50.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.261 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.261 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559769+0000", 2024-09-14T05:29:50.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6740000000000002e-06 2024-09-14T05:29:50.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.261 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.261 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559774+0000", 2024-09-14T05:29:50.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6110000000000004e-06 2024-09-14T05:29:50.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.261 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.262 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604905+0000", 2024-09-14T05:29:50.262 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045130827999999998 2024-09-14T05:29:50.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.262 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.262 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604982+0000", 2024-09-14T05:29:50.262 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7410999999999995e-05 2024-09-14T05:29:50.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.262 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.262 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605077+0000", 2024-09-14T05:29:50.262 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5449e-05 2024-09-14T05:29:50.262 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.262 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.263 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.263 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73793 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2836480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.263 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:29:50.263 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.67695923300000005, 2024-09-14T05:29:50.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045441212000000002, 2024-09-14T05:29:50.263 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.263 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.263 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.263 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.263 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.263 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.263 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73793 2024-09-14T05:29:50.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.264 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:29:50.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:29:50.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.265 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559781+0000", 2024-09-14T05:29:50.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2899999999999999e-07 2024-09-14T05:29:50.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.265 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.265 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559783+0000", 2024-09-14T05:29:50.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.075e-06 2024-09-14T05:29:50.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.265 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.265 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559788+0000", 2024-09-14T05:29:50.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5369999999999997e-06 2024-09-14T05:29:50.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.266 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559793+0000", 2024-09-14T05:29:50.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:29:50.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.266 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605109+0000", 2024-09-14T05:29:50.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316229999999999 2024-09-14T05:29:50.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.266 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605168+0000", 2024-09-14T05:29:50.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8949000000000001e-05 2024-09-14T05:29:50.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605221+0000", 2024-09-14T05:29:50.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3260000000000002e-05 2024-09-14T05:29:50.267 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.267 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.267 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.267 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73806 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.267 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:29:50.267 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.67694060700000003, 2024-09-14T05:29:50.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045526189000000002, 2024-09-14T05:29:50.267 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.268 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.268 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.268 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.268 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.268 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.268 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73806 2024-09-14T05:29:50.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.268 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:29:50.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.269 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.269 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:29:50.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.269 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.269 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:29:50.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:29:50.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.269 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.269 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559801+0000", 2024-09-14T05:29:50.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.995e-06 2024-09-14T05:29:50.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.270 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.270 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559807+0000", 2024-09-14T05:29:50.270 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5450000000000003e-06 2024-09-14T05:29:50.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.270 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.270 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559811+0000", 2024-09-14T05:29:50.270 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5090000000000004e-06 2024-09-14T05:29:50.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.270 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.270 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605237+0000", 2024-09-14T05:29:50.270 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045426133 2024-09-14T05:29:50.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.271 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.271 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605278+0000", 2024-09-14T05:29:50.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1084999999999997e-05 2024-09-14T05:29:50.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.271 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.271 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605325+0000", 2024-09-14T05:29:50.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6582999999999998e-05 2024-09-14T05:29:50.271 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.271 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.271 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.272 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73809 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2838528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.272 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:29:50.272 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.67691710199999999, 2024-09-14T05:29:50.272 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045596062999999999, 2024-09-14T05:29:50.272 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.272 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.272 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.272 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.272 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.272 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.272 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73809 2024-09-14T05:29:50.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.273 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:29:50.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:29:50.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559823+0000", 2024-09-14T05:29:50.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.08e-07 2024-09-14T05:29:50.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.274 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.274 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559825+0000", 2024-09-14T05:29:50.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.26e-06 2024-09-14T05:29:50.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.274 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.274 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559831+0000", 2024-09-14T05:29:50.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6690000000000003e-06 2024-09-14T05:29:50.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.274 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.275 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559835+0000", 2024-09-14T05:29:50.275 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5120000000000002e-06 2024-09-14T05:29:50.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.275 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.275 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605342+0000", 2024-09-14T05:29:50.275 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045506957000000001 2024-09-14T05:29:50.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.275 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.275 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605375+0000", 2024-09-14T05:29:50.275 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3250999999999997e-05 2024-09-14T05:29:50.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.276 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.276 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605418+0000", 2024-09-14T05:29:50.276 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3006000000000001e-05 2024-09-14T05:29:50.276 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.276 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.276 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.276 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73822 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2807808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.276 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:29:50.276 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.67688944900000003, 2024-09-14T05:29:50.276 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045661405000000002, 2024-09-14T05:29:50.276 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.277 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.277 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.277 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.277 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.277 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.277 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73822 2024-09-14T05:29:50.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.277 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:29:50.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:29:50.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559851+0000", 2024-09-14T05:29:50.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4000000000000002e-07 2024-09-14T05:29:50.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559853+0000", 2024-09-14T05:29:50.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5380000000000001e-06 2024-09-14T05:29:50.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.279 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.279 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559859+0000", 2024-09-14T05:29:50.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0619999999999997e-06 2024-09-14T05:29:50.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.279 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.279 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559864+0000", 2024-09-14T05:29:50.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.977e-06 2024-09-14T05:29:50.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.279 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.279 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605435+0000", 2024-09-14T05:29:50.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045570672 2024-09-14T05:29:50.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.280 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.280 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605467+0000", 2024-09-14T05:29:50.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2486e-05 2024-09-14T05:29:50.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.280 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.280 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605511+0000", 2024-09-14T05:29:50.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4129999999999999e-05 2024-09-14T05:29:50.280 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.280 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.280 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.281 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73825 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2840576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.281 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:29:50.281 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.67686608500000001, 2024-09-14T05:29:50.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045728694, 2024-09-14T05:29:50.281 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.281 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.281 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.281 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.281 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.281 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.281 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73825 2024-09-14T05:29:50.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.281 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:29:50.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:29:50.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559874+0000", 2024-09-14T05:29:50.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5400000000000002e-07 2024-09-14T05:29:50.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559876+0000", 2024-09-14T05:29:50.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.131e-06 2024-09-14T05:29:50.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559888+0000", 2024-09-14T05:29:50.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2216e-05 2024-09-14T05:29:50.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559893+0000", 2024-09-14T05:29:50.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6539999999999998e-06 2024-09-14T05:29:50.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605526+0000", 2024-09-14T05:29:50.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045633119 2024-09-14T05:29:50.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605560+0000", 2024-09-14T05:29:50.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3825999999999998e-05 2024-09-14T05:29:50.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605602+0000", 2024-09-14T05:29:50.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2293999999999999e-05 2024-09-14T05:29:50.285 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.285 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.285 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.285 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74023 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2863104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.285 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:29:50.285 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.67568010999999994, 2024-09-14T05:29:50.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044635385999999999, 2024-09-14T05:29:50.285 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.285 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.286 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.286 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.286 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.286 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.286 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74023 2024-09-14T05:29:50.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.286 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:29:50.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:29:50.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561060+0000", 2024-09-14T05:29:50.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0699999999999998e-07 2024-09-14T05:29:50.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561072+0000", 2024-09-14T05:29:50.288 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2261e-05 2024-09-14T05:29:50.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.288 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.288 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561079+0000", 2024-09-14T05:29:50.288 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8399999999999997e-06 2024-09-14T05:29:50.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.288 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.288 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561084+0000", 2024-09-14T05:29:50.288 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7380000000000002e-06 2024-09-14T05:29:50.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605620+0000", 2024-09-14T05:29:50.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044536146999999998 2024-09-14T05:29:50.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605653+0000", 2024-09-14T05:29:50.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2969000000000002e-05 2024-09-14T05:29:50.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605695+0000", 2024-09-14T05:29:50.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2024000000000003e-05 2024-09-14T05:29:50.290 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.290 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.290 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.290 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74026 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2895872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.290 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:29:50.290 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.67564864800000002, 2024-09-14T05:29:50.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044692717, 2024-09-14T05:29:50.290 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.290 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.290 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.290 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.290 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.291 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.291 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74026 2024-09-14T05:29:50.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.291 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.291 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:29:50.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.291 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:29:50.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.292 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:29:50.292 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:29:50.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.292 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561093+0000", 2024-09-14T05:29:50.292 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-14T05:29:50.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.292 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561099+0000", 2024-09-14T05:29:50.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0399999999999998e-06 2024-09-14T05:29:50.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.293 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.293 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561104+0000", 2024-09-14T05:29:50.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8180000000000003e-06 2024-09-14T05:29:50.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.293 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.293 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605710+0000", 2024-09-14T05:29:50.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044605657 2024-09-14T05:29:50.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.293 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.293 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605742+0000", 2024-09-14T05:29:50.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1949000000000002e-05 2024-09-14T05:29:50.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.294 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.294 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605784+0000", 2024-09-14T05:29:50.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1884000000000001e-05 2024-09-14T05:29:50.294 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.294 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.294 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.294 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74039 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.294 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:29:50.294 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.67552204999999999, 2024-09-14T05:29:50.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044654905000000002, 2024-09-14T05:29:50.295 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.295 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.295 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.295 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.295 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.295 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.295 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74039 2024-09-14T05:29:50.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.295 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:29:50.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:29:50.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561218+0000", 2024-09-14T05:29:50.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4000000000000001e-07 2024-09-14T05:29:50.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561221+0000", 2024-09-14T05:29:50.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9730000000000002e-06 2024-09-14T05:29:50.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.297 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.297 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561229+0000", 2024-09-14T05:29:50.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1759999999999993e-06 2024-09-14T05:29:50.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.297 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.297 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561235+0000", 2024-09-14T05:29:50.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0109999999999997e-06 2024-09-14T05:29:50.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.297 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.298 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605798+0000", 2024-09-14T05:29:50.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044563285000000001 2024-09-14T05:29:50.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.298 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.298 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605831+0000", 2024-09-14T05:29:50.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2054999999999997e-05 2024-09-14T05:29:50.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.298 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.298 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605872+0000", 2024-09-14T05:29:50.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1764999999999997e-05 2024-09-14T05:29:50.298 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.298 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.299 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.299 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74042 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.299 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:29:50.299 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.67547359699999998, 2024-09-14T05:29:50.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044773234000000002, 2024-09-14T05:29:50.299 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.299 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.299 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.299 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.299 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.299 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.300 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74042 2024-09-14T05:29:50.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.300 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:29:50.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:29:50.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:29:50.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4099999999999998e-07 2024-09-14T05:29:50.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561269+0000", 2024-09-14T05:29:50.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7930000000000002e-06 2024-09-14T05:29:50.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561278+0000", 2024-09-14T05:29:50.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5150000000000006e-06 2024-09-14T05:29:50.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561284+0000", 2024-09-14T05:29:50.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2029999999999997e-06 2024-09-14T05:29:50.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605934+0000", 2024-09-14T05:29:50.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044650449000000002 2024-09-14T05:29:50.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.303 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605983+0000", 2024-09-14T05:29:50.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8943999999999998e-05 2024-09-14T05:29:50.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.303 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.303 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606039+0000", 2024-09-14T05:29:50.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5689000000000002e-05 2024-09-14T05:29:50.303 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.303 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.303 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.303 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74052 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.303 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:29:50.304 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.675446722, 2024-09-14T05:29:50.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044830474000000002, 2024-09-14T05:29:50.304 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.304 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.304 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.304 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.304 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.304 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.304 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74052 2024-09-14T05:29:50.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.304 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.304 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.304 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:29:50.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:29:50.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:29:50.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1099999999999996e-07 2024-09-14T05:29:50.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561295+0000", 2024-09-14T05:29:50.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7770000000000001e-06 2024-09-14T05:29:50.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561303+0000", 2024-09-14T05:29:50.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6860000000000006e-06 2024-09-14T05:29:50.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561308+0000", 2024-09-14T05:29:50.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.023e-06 2024-09-14T05:29:50.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.307 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605893+0000", 2024-09-14T05:29:50.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044585458000000001 2024-09-14T05:29:50.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.307 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605933+0000", 2024-09-14T05:29:50.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9551999999999997e-05 2024-09-14T05:29:50.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.307 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606123+0000", 2024-09-14T05:29:50.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019046699999999999 2024-09-14T05:29:50.308 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.308 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.308 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.308 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74054 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.308 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:29:50.308 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.67542469500000002, 2024-09-14T05:29:50.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093610999999999, 2024-09-14T05:29:50.308 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.308 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.308 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.308 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.308 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.309 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.309 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74054 2024-09-14T05:29:50.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.309 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:29:50.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:29:50.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:29:50.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9699999999999996e-07 2024-09-14T05:29:50.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561317+0000", 2024-09-14T05:29:50.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9740000000000001e-06 2024-09-14T05:29:50.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561324+0000", 2024-09-14T05:29:50.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4590000000000003e-06 2024-09-14T05:29:50.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561329+0000", 2024-09-14T05:29:50.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5969999999999999e-06 2024-09-14T05:29:50.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606142+0000", 2024-09-14T05:29:50.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044812347000000002 2024-09-14T05:29:50.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.312 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606184+0000", 2024-09-14T05:29:50.312 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2441999999999998e-05 2024-09-14T05:29:50.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.312 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.312 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606408+0000", 2024-09-14T05:29:50.312 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022429500000000001 2024-09-14T05:29:50.312 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.312 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.312 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.312 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74056 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.312 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:29:50.313 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.675229992, 2024-09-14T05:29:50.313 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045191685000000002, 2024-09-14T05:29:50.313 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.313 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.313 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.313 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.313 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.313 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.313 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74056 2024-09-14T05:29:50.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.313 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.313 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.313 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:29:50.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.314 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:29:50.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.314 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561510+0000", 2024-09-14T05:29:50.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2399999999999998e-07 2024-09-14T05:29:50.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561513+0000", 2024-09-14T05:29:50.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7219999999999999e-06 2024-09-14T05:29:50.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561524+0000", 2024-09-14T05:29:50.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1357e-05 2024-09-14T05:29:50.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561530+0000", 2024-09-14T05:29:50.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2890000000000003e-06 2024-09-14T05:29:50.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.316 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606438+0000", 2024-09-14T05:29:50.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044907326999999997 2024-09-14T05:29:50.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.316 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606520+0000", 2024-09-14T05:29:50.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1924999999999999e-05 2024-09-14T05:29:50.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.316 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606701+0000", 2024-09-14T05:29:50.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00018144100000000001 2024-09-14T05:29:50.317 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.317 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.317 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.317 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74058 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.317 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:29:50.317 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.67475131899999996, 2024-09-14T05:29:50.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093368000000002, 2024-09-14T05:29:50.317 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.317 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.317 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.317 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.317 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.318 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.318 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74058 2024-09-14T05:29:50.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.318 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.318 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.318 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:29:50.318 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.318 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.318 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:29:50.318 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561989+0000", 2024-09-14T05:29:50.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3300000000000001e-07 2024-09-14T05:29:50.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561993+0000", 2024-09-14T05:29:50.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1790000000000001e-06 2024-09-14T05:29:50.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562007+0000", 2024-09-14T05:29:50.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3434e-05 2024-09-14T05:29:50.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562014+0000", 2024-09-14T05:29:50.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.537e-06 2024-09-14T05:29:50.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606736+0000", 2024-09-14T05:29:50.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044722117999999998 2024-09-14T05:29:50.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606829+0000", 2024-09-14T05:29:50.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2410999999999993e-05 2024-09-14T05:29:50.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607082+0000", 2024-09-14T05:29:50.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025285599999999998 2024-09-14T05:29:50.321 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.321 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.321 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.321 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74059 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.321 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:29:50.321 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.67469332800000004, 2024-09-14T05:29:50.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045479928000000003, 2024-09-14T05:29:50.322 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.322 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.322 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.322 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.322 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.322 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.322 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74059 2024-09-14T05:29:50.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.322 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:29:50.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:29:50.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562047+0000", 2024-09-14T05:29:50.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6899999999999997e-07 2024-09-14T05:29:50.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562050+0000", 2024-09-14T05:29:50.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.943e-06 2024-09-14T05:29:50.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562058+0000", 2024-09-14T05:29:50.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0120000000000005e-06 2024-09-14T05:29:50.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562063+0000", 2024-09-14T05:29:50.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:29:50.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607186+0000", 2024-09-14T05:29:50.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045122348999999999 2024-09-14T05:29:50.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607265+0000", 2024-09-14T05:29:50.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9462999999999995e-05 2024-09-14T05:29:50.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607526+0000", 2024-09-14T05:29:50.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026090400000000001 2024-09-14T05:29:50.325 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.326 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.326 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.326 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74060 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2850816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:50.326 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:29:50.326 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.67466795099999999, 2024-09-14T05:29:50.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045200092999999997, 2024-09-14T05:29:50.326 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.326 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.326 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.326 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.326 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.326 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.327 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74060 2024-09-14T05:29:50.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.327 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:29:50.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:29:50.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.328 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:29:50.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5200000000000002e-07 2024-09-14T05:29:50.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.328 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562075+0000", 2024-09-14T05:29:50.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5200000000000002e-06 2024-09-14T05:29:50.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.328 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562082+0000", 2024-09-14T05:29:50.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8929999999999999e-06 2024-09-14T05:29:50.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562088+0000", 2024-09-14T05:29:50.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5570000000000003e-06 2024-09-14T05:29:50.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607111+0000", 2024-09-14T05:29:50.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045023464999999999 2024-09-14T05:29:50.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607190+0000", 2024-09-14T05:29:50.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8721999999999996e-05 2024-09-14T05:29:50.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607272+0000", 2024-09-14T05:29:50.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1483999999999994e-05 2024-09-14T05:29:50.330 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.330 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.330 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.330 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.330 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.330 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:50.331 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:29:50.332 INFO:teuthology.orchestra.run.smithi071.stdout:1730 read 0x233fe04 thru 0x2340aea (0xce7 bytes) 2024-09-14T05:29:50.332 INFO:teuthology.orchestra.run.smithi071.stdout:1731 trunc from 0x856849b to 0x92deab 2024-09-14T05:29:50.332 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:50.332 INFO:teuthology.orchestra.run.smithi071.stdout:1733 punch from 0x60702 to 0x611f5, (0xaf3 bytes) 2024-09-14T05:29:50.333 INFO:teuthology.orchestra.run.smithi071.stdout:1734 trunc from 0x92deab to 0x4562d39 2024-09-14T05:29:50.333 INFO:teuthology.orchestra.run.smithi071.stdout:1735 write 0x5dd6aa6 thru 0x5de5a4d (0xefa8 bytes) 2024-09-14T05:29:50.335 INFO:teuthology.orchestra.run.smithi071.stdout:1736 writesame 0x67a463c thru 0x67aeb18 data_size 0x17(0xa4dd bytes) 2024-09-14T05:29:50.335 INFO:teuthology.orchestra.run.smithi071.stdout:1738 read 0x1b5a839 thru 0x1b5dbba (0x3382 bytes) 2024-09-14T05:29:50.336 INFO:teuthology.orchestra.run.smithi071.stdout:1742 read 0x66d056c thru 0x66d18df (0x1374 bytes) 2024-09-14T05:29:50.336 INFO:teuthology.orchestra.run.smithi071.stdout:1743 write 0x61579a0 thru 0x6162d4f (0xb3b0 bytes) 2024-09-14T05:29:50.336 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:50.334+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:50.340 INFO:teuthology.orchestra.run.smithi071.stdout:1745 read 0x14e29d thru 0x150a55 (0x27b9 bytes) 2024-09-14T05:29:50.341 INFO:teuthology.orchestra.run.smithi071.stdout:1746 write 0x18935ba thru 0x189600c (0x2a53 bytes) 2024-09-14T05:29:50.352 INFO:teuthology.orchestra.run.smithi071.stdout:1747 read 0x3a4dbc1 thru 0x3a4e35a (0x79a bytes) 2024-09-14T05:29:50.353 INFO:teuthology.orchestra.run.smithi071.stdout:1748 punch from 0xf9fb02 to 0xfaab61, (0xb05f bytes) 2024-09-14T05:29:50.353 INFO:teuthology.orchestra.run.smithi071.stdout:1750 clone 17 order 21 su 32768 sc 12 2024-09-14T05:29:50.359 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:29:50.360 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:50.360 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 48, 2024-09-14T05:29:50.360 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "1127be44-5f14-4e7d-ad23-f608afe154f5", 2024-09-14T05:29:50.360 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2024-09-14T05:28:54.203072+0000", 2024-09-14T05:29:50.360 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-14T05:29:50.138461+0000", 2024-09-14T05:29:50.360 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2024-09-14T05:29:06.755531+0000", 2024-09-14T05:29:50.360 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2024-09-14T05:28:56.535048+0000", 2024-09-14T05:29:50.360 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T05:29:50.360 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5806080, 2024-09-14T05:29:50.360 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2024-09-14T05:29:50.360 INFO:teuthology.orchestra.run.smithi071.stdout: "nodeep-scrub", 2024-09-14T05:29:50.360 INFO:teuthology.orchestra.run.smithi071.stdout: "noscrub", 2024-09-14T05:29:50.361 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2024-09-14T05:29:50.361 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2024-09-14T05:29:50.361 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2024-09-14T05:29:50.361 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2024-09-14T05:29:50.361 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:29:50.361 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 6, 2024-09-14T05:29:50.361 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T05:29:50.361 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T05:29:50.361 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T05:29:50.361 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2024-09-14T05:29:50.361 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 3, 2024-09-14T05:29:50.361 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2024-09-14T05:29:50.361 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2024-09-14T05:29:50.361 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2024-09-14T05:29:50.362 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "squid", 2024-09-14T05:29:50.362 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2024-09-14T05:29:50.362 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2024-09-14T05:29:50.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.362 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2024-09-14T05:29:50.362 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2024-09-14T05:29:50.362 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-14T05:29:06.452744+0000", 2024-09-14T05:29:50.362 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-14T05:29:50.362 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2024-09-14T05:29:50.362 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-14T05:29:50.362 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-14T05:29:50.362 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-14T05:29:50.362 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-14T05:29:50.362 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-14T05:29:50.363 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-14T05:29:50.363 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T05:29:50.363 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T05:29:50.363 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-14T05:29:50.363 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-14T05:29:50.363 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-14T05:29:50.363 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2024-09-14T05:29:50.363 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2024-09-14T05:29:50.363 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2024-09-14T05:29:50.363 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2024-09-14T05:29:50.363 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2024-09-14T05:29:50.363 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-14T05:29:50.363 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-14T05:29:50.363 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-14T05:29:50.364 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-14T05:29:50.364 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-14T05:29:50.364 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-14T05:29:50.364 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-14T05:29:50.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.364 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "18", 2024-09-14T05:29:50.364 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-14T05:29:50.364 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T05:29:50.364 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T05:29:50.364 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-14T05:29:50.364 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-14T05:29:50.364 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2024-09-14T05:29:50.364 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2024-09-14T05:29:50.365 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-14T05:29:50.365 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-14T05:29:50.365 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-14T05:29:50.365 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-14T05:29:50.365 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-14T05:29:50.365 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-14T05:29:50.365 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-14T05:29:50.365 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-14T05:29:50.365 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-14T05:29:50.365 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-14T05:29:50.365 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-14T05:29:50.365 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T05:29:50.365 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T05:29:50.365 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T05:29:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-14T05:29:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-14T05:29:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-14T05:29:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-14T05:29:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-14T05:29:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-14T05:29:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-14T05:29:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-14T05:29:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-14T05:29:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-14T05:29:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T05:29:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-14T05:29:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-14T05:29:50.367 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-14T05:29:50.367 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-14T05:29:50.367 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-14T05:29:50.367 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2024-09-14T05:29:50.367 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2024-09-14T05:29:50.367 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2024-09-14T05:29:50.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.367 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-14T05:29:50.367 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2024-09-14T05:29:50.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.368 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-14T05:29:50.368 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-14T05:29:50.368 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2024-09-14T05:29:50.368 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2024-09-14T05:29:50.368 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2024-09-14T05:29:50.368 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2024-09-14T05:29:50.368 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2024-09-14T05:29:50.368 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.87708091735839844, 2024-09-14T05:29:50.368 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T05:29:50.368 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T05:29:50.368 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.369 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 2, 2024-09-14T05:29:50.369 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "rbd", 2024-09-14T05:29:50.369 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-14T05:29:12.380843+0000", 2024-09-14T05:29:50.369 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 8193, 2024-09-14T05:29:50.369 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T05:29:50.369 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-14T05:29:50.369 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-14T05:29:50.369 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-14T05:29:50.369 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-14T05:29:50.369 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-14T05:29:50.369 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-14T05:29:50.369 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T05:29:50.369 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T05:29:50.370 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-14T05:29:50.370 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-14T05:29:50.370 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-14T05:29:50.370 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 8, 2024-09-14T05:29:50.370 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 8, 2024-09-14T05:29:50.370 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 8, 2024-09-14T05:29:50.370 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 8, 2024-09-14T05:29:50.370 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 8, 2024-09-14T05:29:50.370 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-14T05:29:50.370 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-14T05:29:50.370 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-14T05:29:50.370 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-14T05:29:50.370 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-14T05:29:50.370 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-14T05:29:50.371 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-14T05:29:50.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.371 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "22", 2024-09-14T05:29:50.371 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-14T05:29:50.371 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T05:29:50.371 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T05:29:50.371 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-14T05:29:50.371 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-14T05:29:50.371 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 2, 2024-09-14T05:29:50.371 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 22, 2024-09-14T05:29:50.371 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-14T05:29:50.371 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-14T05:29:50.371 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-14T05:29:50.371 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-14T05:29:50.372 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-14T05:29:50.372 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-14T05:29:50.372 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-14T05:29:50.372 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-14T05:29:50.372 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-14T05:29:50.372 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-14T05:29:50.372 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-14T05:29:50.372 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T05:29:50.372 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T05:29:50.372 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T05:29:50.372 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-14T05:29:50.372 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-14T05:29:50.372 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-14T05:29:50.373 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-14T05:29:50.373 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-14T05:29:50.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.373 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-14T05:29:50.373 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-14T05:29:50.373 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-14T05:29:50.373 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-14T05:29:50.373 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-14T05:29:50.373 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T05:29:50.373 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-14T05:29:50.373 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-14T05:29:50.373 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-14T05:29:50.373 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-14T05:29:50.373 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-14T05:29:50.374 INFO:teuthology.orchestra.run.smithi071.stdout: "options": {}, 2024-09-14T05:29:50.374 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-14T05:29:50.374 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd": {} 2024-09-14T05:29:50.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.374 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-14T05:29:50.374 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-14T05:29:50.374 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 4.9800000190734863, 2024-09-14T05:29:50.374 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 4.9800000190734863, 2024-09-14T05:29:50.374 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T05:29:50.374 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 4.380000114440918, 2024-09-14T05:29:50.374 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 4.380000114440918, 2024-09-14T05:29:50.374 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.87708091735839844, 2024-09-14T05:29:50.374 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T05:29:50.374 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T05:29:50.375 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.375 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 3, 2024-09-14T05:29:50.375 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "pool_client.0", 2024-09-14T05:29:50.375 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-14T05:29:22.335419+0000", 2024-09-14T05:29:50.375 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 8193, 2024-09-14T05:29:50.375 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T05:29:50.375 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-14T05:29:50.375 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-14T05:29:50.375 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-14T05:29:50.375 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-14T05:29:50.375 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-14T05:29:50.375 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-14T05:29:50.376 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T05:29:50.376 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T05:29:50.376 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-14T05:29:50.376 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-14T05:29:50.376 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-14T05:29:50.376 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2024-09-14T05:29:50.376 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2024-09-14T05:29:50.376 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2024-09-14T05:29:50.376 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2024-09-14T05:29:50.376 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2024-09-14T05:29:50.376 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-14T05:29:50.376 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-14T05:29:50.376 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-14T05:29:50.377 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-14T05:29:50.377 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-14T05:29:50.377 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-14T05:29:50.377 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-14T05:29:50.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.377 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "47", 2024-09-14T05:29:50.377 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-14T05:29:50.377 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T05:29:50.377 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T05:29:50.377 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-14T05:29:50.377 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-14T05:29:50.377 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 18, 2024-09-14T05:29:50.377 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 47, 2024-09-14T05:29:50.378 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-14T05:29:50.378 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-14T05:29:50.378 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-14T05:29:50.378 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-14T05:29:50.378 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-14T05:29:50.378 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-14T05:29:50.378 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-14T05:29:50.378 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-14T05:29:50.378 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-14T05:29:50.378 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-14T05:29:50.378 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-14T05:29:50.378 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T05:29:50.378 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T05:29:50.378 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T05:29:50.379 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-14T05:29:50.379 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-14T05:29:50.379 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-14T05:29:50.379 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-14T05:29:50.379 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-14T05:29:50.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.379 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-14T05:29:50.379 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-14T05:29:50.379 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-14T05:29:50.379 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-14T05:29:50.379 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-14T05:29:50.379 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T05:29:50.379 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-14T05:29:50.379 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-14T05:29:50.380 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-14T05:29:50.380 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-14T05:29:50.380 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-14T05:29:50.380 INFO:teuthology.orchestra.run.smithi071.stdout: "options": {}, 2024-09-14T05:29:50.380 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-14T05:29:50.380 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd": {} 2024-09-14T05:29:50.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.380 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-14T05:29:50.380 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-14T05:29:50.380 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2024-09-14T05:29:50.380 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2024-09-14T05:29:50.380 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2024-09-14T05:29:50.380 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2024-09-14T05:29:50.381 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2024-09-14T05:29:50.381 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.87708091735839844, 2024-09-14T05:29:50.381 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T05:29:50.381 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T05:29:50.381 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.381 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.381 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:29:50.381 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2024-09-14T05:29:50.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.381 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2024-09-14T05:29:50.381 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "e7c7817f-4a65-4f66-9af5-d31d0a92cf30", 2024-09-14T05:29:50.381 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:50.381 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:50.381 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:50.382 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:50.382 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:50.382 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:50.382 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 16, 2024-09-14T05:29:50.382 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:29:50.382 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:50.382 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:50.382 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:50.382 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.382 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.382 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2024-09-14T05:29:50.382 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:50.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.383 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.383 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2024-09-14T05:29:50.383 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:50.383 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.383 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.383 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:50.383 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.383 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.383 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2024-09-14T05:29:50.383 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2024-09-14T05:29:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2024-09-14T05:29:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2024-09-14T05:29:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2024-09-14T05:29:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2024-09-14T05:29:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6819/2926109885", 2024-09-14T05:29:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6821/2926109885", 2024-09-14T05:29:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6825/2926109885", 2024-09-14T05:29:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6823/2926109885", 2024-09-14T05:29:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:50.387 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:50.387 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:50.387 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.387 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2024-09-14T05:29:50.387 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "e184604e-b973-4b7a-bba2-6f471753a0fd", 2024-09-14T05:29:50.387 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:50.387 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:50.387 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:50.387 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:50.387 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:50.387 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:50.388 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:50.388 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:29:50.388 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:50.388 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:50.388 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:50.388 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.388 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.388 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2024-09-14T05:29:50.388 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:50.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.388 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.388 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2024-09-14T05:29:50.389 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:50.389 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.389 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.389 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:50.389 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.389 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.389 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2024-09-14T05:29:50.389 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:50.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.389 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.390 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2024-09-14T05:29:50.390 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:50.390 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.390 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.390 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:50.390 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.390 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.390 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6832", 2024-09-14T05:29:50.390 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:50.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.390 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.391 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6833", 2024-09-14T05:29:50.391 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:50.391 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.391 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.391 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:50.391 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.391 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.391 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2024-09-14T05:29:50.391 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:50.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.391 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.392 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2024-09-14T05:29:50.392 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:50.392 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.392 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.392 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6827/345649648", 2024-09-14T05:29:50.392 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6829/345649648", 2024-09-14T05:29:50.392 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6833/345649648", 2024-09-14T05:29:50.392 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6831/345649648", 2024-09-14T05:29:50.392 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:50.392 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:50.392 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:50.392 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.393 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2024-09-14T05:29:50.393 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "d8b70f09-5afb-4296-b8c9-719c172e0e70", 2024-09-14T05:29:50.393 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:50.393 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:50.393 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:50.393 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:50.393 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:50.393 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:50.393 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:50.393 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 39, 2024-09-14T05:29:50.393 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:50.393 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:50.393 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:50.394 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.394 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.394 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2024-09-14T05:29:50.394 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:50.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.394 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.394 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2024-09-14T05:29:50.394 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:50.394 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.394 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.394 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:50.395 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.395 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.395 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2024-09-14T05:29:50.395 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:50.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.395 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.395 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2024-09-14T05:29:50.395 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:50.395 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.395 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.395 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:50.396 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.396 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.396 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2024-09-14T05:29:50.396 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:50.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.396 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.396 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2024-09-14T05:29:50.396 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:50.396 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.396 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2024-09-14T05:29:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2024-09-14T05:29:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/2721860691", 2024-09-14T05:29:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/2721860691", 2024-09-14T05:29:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/2721860691", 2024-09-14T05:29:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/2721860691", 2024-09-14T05:29:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2024-09-14T05:29:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c", 2024-09-14T05:29:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 39, 2024-09-14T05:29:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2024-09-14T05:29:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2024-09-14T05:29:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2024-09-14T05:29:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2024-09-14T05:29:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2024-09-14T05:29:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2024-09-14T05:29:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2024-09-14T05:29:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2024-09-14T05:29:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/494607463", 2024-09-14T05:29:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6811/494607463", 2024-09-14T05:29:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6815/494607463", 2024-09-14T05:29:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6813/494607463", 2024-09-14T05:29:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2024-09-14T05:29:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "515cb7d6-16eb-4aff-973a-7f7ed49325ba", 2024-09-14T05:29:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:29:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6816", 2024-09-14T05:29:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6817", 2024-09-14T05:29:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6818", 2024-09-14T05:29:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6819", 2024-09-14T05:29:50.407 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:50.407 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.407 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.407 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:50.407 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.407 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.407 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6822", 2024-09-14T05:29:50.407 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:50.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.407 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.407 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6823", 2024-09-14T05:29:50.408 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:50.408 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.408 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.408 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:50.408 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.408 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.408 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6820", 2024-09-14T05:29:50.408 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:50.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.408 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.409 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6821", 2024-09-14T05:29:50.409 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:50.409 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.409 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.409 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6817/2240133417", 2024-09-14T05:29:50.409 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6819/2240133417", 2024-09-14T05:29:50.409 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6823/2240133417", 2024-09-14T05:29:50.409 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6821/2240133417", 2024-09-14T05:29:50.409 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:50.409 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:50.409 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:50.409 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.410 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2024-09-14T05:29:50.410 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "c434b53a-04be-45b9-b89a-193654133950", 2024-09-14T05:29:50.410 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:50.410 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:50.410 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:50.410 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:50.410 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:50.410 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:50.410 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:50.410 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 39, 2024-09-14T05:29:50.410 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:50.410 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:50.410 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:50.411 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.411 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.411 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6808", 2024-09-14T05:29:50.411 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:50.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.411 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.411 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6809", 2024-09-14T05:29:50.411 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:50.411 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.411 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.411 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:50.412 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.412 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.412 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6810", 2024-09-14T05:29:50.412 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:50.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.412 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.412 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6811", 2024-09-14T05:29:50.412 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:50.412 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.412 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.412 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:50.413 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.413 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.413 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6814", 2024-09-14T05:29:50.413 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:50.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.413 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.413 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6815", 2024-09-14T05:29:50.413 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:50.413 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.413 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.414 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:50.414 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.414 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.414 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6812", 2024-09-14T05:29:50.414 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:50.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.414 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.414 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6813", 2024-09-14T05:29:50.414 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:50.414 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.414 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.415 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6809/2393825223", 2024-09-14T05:29:50.415 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6811/2393825223", 2024-09-14T05:29:50.415 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6815/2393825223", 2024-09-14T05:29:50.415 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6813/2393825223", 2024-09-14T05:29:50.415 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:50.415 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:50.415 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:50.415 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.415 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2024-09-14T05:29:50.415 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "3d2f8405-9286-4b40-8acd-036f67373ac6", 2024-09-14T05:29:50.415 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:50.415 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:50.416 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:50.416 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 0.0166473388671875, 2024-09-14T05:29:50.416 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:50.416 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:50.416 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:50.416 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:29:50.416 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:50.416 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:50.416 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:50.416 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.416 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.416 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6800", 2024-09-14T05:29:50.417 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:50.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.417 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.417 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6801", 2024-09-14T05:29:50.417 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:50.417 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.417 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.417 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:50.417 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.417 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.417 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6802", 2024-09-14T05:29:50.418 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:50.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.418 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.418 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6803", 2024-09-14T05:29:50.418 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:50.418 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.418 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.418 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:50.418 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.418 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.418 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6806", 2024-09-14T05:29:50.419 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:50.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.419 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.419 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6807", 2024-09-14T05:29:50.419 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:50.419 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.419 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.419 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:50.419 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.419 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.420 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6804", 2024-09-14T05:29:50.420 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:50.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.420 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.420 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6805", 2024-09-14T05:29:50.420 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:50.420 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.420 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.420 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6801/3235729544", 2024-09-14T05:29:50.420 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6803/3235729544", 2024-09-14T05:29:50.420 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6807/3235729544", 2024-09-14T05:29:50.421 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6805/3235729544", 2024-09-14T05:29:50.421 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:50.421 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:50.421 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:50.421 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.421 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2024-09-14T05:29:50.421 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "6724f080-4dc3-465a-939f-b023985bf998", 2024-09-14T05:29:50.421 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:50.421 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:50.421 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:50.421 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:50.421 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:50.422 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:50.422 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:50.422 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:29:50.422 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:50.422 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:50.422 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:50.422 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.422 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.422 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6824", 2024-09-14T05:29:50.422 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:50.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.422 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.423 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6825", 2024-09-14T05:29:50.423 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:50.423 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.423 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.423 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:50.423 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.423 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.423 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6826", 2024-09-14T05:29:50.423 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:50.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.423 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.424 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6827", 2024-09-14T05:29:50.424 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:50.424 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.424 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.424 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:50.424 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.424 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.424 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6830", 2024-09-14T05:29:50.424 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:50.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.425 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.425 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6831", 2024-09-14T05:29:50.425 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:50.425 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.425 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.425 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:50.425 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:50.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.425 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:50.425 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6828", 2024-09-14T05:29:50.425 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:50.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.426 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:50.426 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6829", 2024-09-14T05:29:50.426 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:50.426 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.426 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.426 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6825/2381001355", 2024-09-14T05:29:50.426 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6827/2381001355", 2024-09-14T05:29:50.426 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6831/2381001355", 2024-09-14T05:29:50.426 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6829/2381001355", 2024-09-14T05:29:50.426 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:50.426 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:50.426 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:50.426 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.427 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.427 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:29:50.427 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2024-09-14T05:29:50.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.427 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2024-09-14T05:29:50.427 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:50.427 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:50.427 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:50.427 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:50.427 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:50.427 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.548803+0000", 2024-09-14T05:29:50.427 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:50.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.428 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2024-09-14T05:29:50.428 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:50.428 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:50.428 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:50.428 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:50.428 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:50.428 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.075023+0000", 2024-09-14T05:29:50.428 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:50.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.428 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2024-09-14T05:29:50.428 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:50.428 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:50.428 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:50.429 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:50.429 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:50.429 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.117036+0000", 2024-09-14T05:29:50.429 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:50.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.429 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2024-09-14T05:29:50.429 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:50.429 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:50.429 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:50.429 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:50.429 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:50.429 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.332113+0000", 2024-09-14T05:29:50.429 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:50.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.430 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2024-09-14T05:29:50.430 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:50.430 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:50.430 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:50.430 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:50.430 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:50.430 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.330371+0000", 2024-09-14T05:29:50.430 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:50.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.430 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2024-09-14T05:29:50.430 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:50.431 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:50.431 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:50.431 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:50.431 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:50.431 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:02.753946+0000", 2024-09-14T05:29:50.431 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:50.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.431 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2024-09-14T05:29:50.431 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:50.431 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:50.431 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:50.431 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:50.432 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:50.432 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:02.071186+0000", 2024-09-14T05:29:50.432 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:50.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.432 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2024-09-14T05:29:50.432 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:50.432 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:50.432 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:50.432 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:50.432 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:50.432 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.378781+0000", 2024-09-14T05:29:50.432 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:50.432 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.432 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:29:50.433 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2024-09-14T05:29:50.433 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2024-09-14T05:29:50.433 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2024-09-14T05:29:50.433 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2024-09-14T05:29:50.433 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2024-09-14T05:29:50.433 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2024-09-14T05:29:50.433 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2024-09-14T05:29:50.433 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2024-09-14T05:29:50.433 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2024-09-14T05:29:50.433 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2024-09-14T05:29:50.433 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2024-09-14T05:29:50.433 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2024-09-14T05:29:50.433 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2024-09-14T05:29:50.433 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2024-09-14T05:29:50.434 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.434 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2024-09-14T05:29:50.434 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2024-09-14T05:29:50.434 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2024-09-14T05:29:50.434 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2024-09-14T05:29:50.434 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2024-09-14T05:29:50.434 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2024-09-14T05:29:50.434 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2024-09-14T05:29:50.434 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2024-09-14T05:29:50.434 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2024-09-14T05:29:50.434 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2024-09-14T05:29:50.434 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2024-09-14T05:29:50.435 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.435 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:50.437 INFO:tasks.thrashosds.thrasher:j is {'epoch': 48, 'fsid': '1127be44-5f14-4e7d-ad23-f608afe154f5', 'created': '2024-09-14T05:28:54.203072+0000', 'modified': '2024-09-14T05:29:50.138461+0000', 'last_up_change': '2024-09-14T05:29:06.755531+0000', 'last_in_change': '2024-09-14T05:28:56.535048+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T05:29:06.452744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8770809173583984, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T05:29:12.380843+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.8770809173583984, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T05:29:22.335419+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 18, 'snap_epoch': 47, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8770809173583984, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'e7c7817f-4a65-4f66-9af5-d31d0a92cf30', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6818', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6819', 'nonce': 2926109885}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6820', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6821', 'nonce': 2926109885}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6824', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6825', 'nonce': 2926109885}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6822', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6823', 'nonce': 2926109885}]}, 'public_addr': '172.21.15.71:6819/2926109885', 'cluster_addr': '172.21.15.71:6821/2926109885', 'heartbeat_back_addr': '172.21.15.71:6825/2926109885', 'heartbeat_front_addr': '172.21.15.71:6823/2926109885', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e184604e-b973-4b7a-bba2-6f471753a0fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6826', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6827', 'nonce': 345649648}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6828', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6829', 'nonce': 345649648}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6832', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6833', 'nonce': 345649648}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6830', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6831', 'nonce': 345649648}]}, 'public_addr': '172.21.15.71:6827/345649648', 'cluster_addr': '172.21.15.71:6829/345649648', 'heartbeat_back_addr': '172.21.15.71:6833/345649648', 'heartbeat_front_addr': '172.21.15.71:6831/345649648', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd8b70f09-5afb-4296-b8c9-719c172e0e70', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6800', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6801', 'nonce': 2721860691}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6802', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6803', 'nonce': 2721860691}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6806', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6807', 'nonce': 2721860691}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6804', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6805', 'nonce': 2721860691}]}, 'public_addr': '172.21.15.71:6801/2721860691', 'cluster_addr': '172.21.15.71:6803/2721860691', 'heartbeat_back_addr': '172.21.15.71:6807/2721860691', 'heartbeat_front_addr': '172.21.15.71:6805/2721860691', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6808', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6809', 'nonce': 494607463}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6810', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6811', 'nonce': 494607463}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6814', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6815', 'nonce': 494607463}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6812', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6813', 'nonce': 494607463}]}, 'public_addr': '172.21.15.71:6809/494607463', 'cluster_addr': '172.21.15.71:6811/494607463', 'heartbeat_back_addr': '172.21.15.71:6815/494607463', 'heartbeat_front_addr': '172.21.15.71:6813/494607463', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '515cb7d6-16eb-4aff-973a-7f7ed49325ba', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6816', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6817', 'nonce': 2240133417}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6818', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6819', 'nonce': 2240133417}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6822', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6823', 'nonce': 2240133417}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6820', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6821', 'nonce': 2240133417}]}, 'public_addr': '172.21.15.150:6817/2240133417', 'cluster_addr': '172.21.15.150:6819/2240133417', 'heartbeat_back_addr': '172.21.15.150:6823/2240133417', 'heartbeat_front_addr': '172.21.15.150:6821/2240133417', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'c434b53a-04be-45b9-b89a-193654133950', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6808', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6809', 'nonce': 2393825223}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6810', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6811', 'nonce': 2393825223}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6814', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6815', 'nonce': 2393825223}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6812', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6813', 'nonce': 2393825223}]}, 'public_addr': '172.21.15.150:6809/2393825223', 'cluster_addr': '172.21.15.150:6811/2393825223', 'heartbeat_back_addr': '172.21.15.150:6815/2393825223', 'heartbeat_front_addr': '172.21.15.150:6813/2393825223', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '3d2f8405-9286-4b40-8acd-036f67373ac6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0166473388671875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6800', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6801', 'nonce': 3235729544}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6802', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6803', 'nonce': 3235729544}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6806', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6807', 'nonce': 3235729544}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6804', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6805', 'nonce': 3235729544}]}, 'public_addr': '172.21.15.150:6801/3235729544', 'cluster_addr': '172.21.15.150:6803/3235729544', 'heartbeat_back_addr': '172.21.15.150:6807/3235729544', 'heartbeat_front_addr': '172.21.15.150:6805/3235729544', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '6724f080-4dc3-465a-939f-b023985bf998', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6824', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6825', 'nonce': 2381001355}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6826', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6827', 'nonce': 2381001355}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6830', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6831', 'nonce': 2381001355}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6828', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6829', 'nonce': 2381001355}]}, 'public_addr': '172.21.15.150:6825/2381001355', 'cluster_addr': '172.21.15.150:6827/2381001355', 'heartbeat_back_addr': '172.21.15.150:6831/2381001355', 'heartbeat_front_addr': '172.21.15.150:6829/2381001355', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.548803+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.075023+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.117036+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.332113+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.330371+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:02.753946+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:02.071186+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.378781+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T05:29:50.437 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-14T05:29:50.437 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:50.434+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:50.441 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:50.441 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:50.441 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:29:50.441 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:50.450 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:29:50.536 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:50.536+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:50.592 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:50.593 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:50.593 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:50.593 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:50.593 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:50.601 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:29:50.638 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:50.637+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:50.705 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:50.705 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:50.705 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:50.705 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:50.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.705 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4235.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-14T05:29:50.705 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:29:50.705 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 36.863867431999999, 2024-09-14T05:29:50.706 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12887758399999999, 2024-09-14T05:29:50.706 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.706 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.706 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:29:50.706 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.706 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4235", 2024-09-14T05:29:50.706 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/3211559243", 2024-09-14T05:29:50.706 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 2 2024-09-14T05:29:50.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.706 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.706 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.706 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:29:50.707 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.707 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.707 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:29:50.707 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.707 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.707 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839833+0000", 2024-09-14T05:29:50.707 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:29:50.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.708 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.708 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839838+0000", 2024-09-14T05:29:50.708 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2440000000000001e-06 2024-09-14T05:29:50.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.708 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.708 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839848+0000", 2024-09-14T05:29:50.708 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0030000000000001e-05 2024-09-14T05:29:50.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839854+0000", 2024-09-14T05:29:50.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3750000000000002e-06 2024-09-14T05:29:50.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839908+0000", 2024-09-14T05:29:50.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3965999999999998e-05 2024-09-14T05:29:50.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:29:50.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839914+0000", 2024-09-14T05:29:50.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6139999999999997e-06 2024-09-14T05:29:50.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.710 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.710 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968558+0000", 2024-09-14T05:29:50.710 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12864361199999999 2024-09-14T05:29:50.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.710 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.710 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968710+0000", 2024-09-14T05:29:50.710 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015177300000000001 2024-09-14T05:29:50.710 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.710 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.710 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.711 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27764 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 188416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.711 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:29:50.711 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4925410499999998, 2024-09-14T05:29:50.711 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080826920999999996, 2024-09-14T05:29:50.711 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.711 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.711 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.711 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.711 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.711 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.711 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27764 2024-09-14T05:29:50.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.712 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.712 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.712 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:29:50.712 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.712 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.712 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:29:50.712 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.712 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.712 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211159+0000", 2024-09-14T05:29:50.713 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1800000000000001e-07 2024-09-14T05:29:50.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.713 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.713 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211161+0000", 2024-09-14T05:29:50.713 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.968e-06 2024-09-14T05:29:50.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.713 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.713 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211169+0000", 2024-09-14T05:29:50.713 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4800000000000001e-06 2024-09-14T05:29:50.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.713 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.714 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211174+0000", 2024-09-14T05:29:50.714 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1880000000000001e-06 2024-09-14T05:29:50.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.714 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.714 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291670+0000", 2024-09-14T05:29:50.714 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080495199000000003 2024-09-14T05:29:50.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.714 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.714 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291775+0000", 2024-09-14T05:29:50.714 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000105375 2024-09-14T05:29:50.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.715 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.715 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291985+0000", 2024-09-14T05:29:50.715 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021029299999999999 2024-09-14T05:29:50.715 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.715 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.715 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.715 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.715 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27765 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.715 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:29:50.715 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4925183000000004, 2024-09-14T05:29:50.715 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081115197999999999, 2024-09-14T05:29:50.715 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.715 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.716 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.716 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.716 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.716 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.716 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27765 2024-09-14T05:29:50.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.716 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.716 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.716 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.716 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:29:50.716 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.716 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.717 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.717 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:29:50.717 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.717 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.717 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211182+0000", 2024-09-14T05:29:50.717 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3599999999999999e-07 2024-09-14T05:29:50.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.717 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.717 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211184+0000", 2024-09-14T05:29:50.717 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.03e-06 2024-09-14T05:29:50.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.718 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.718 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211192+0000", 2024-09-14T05:29:50.718 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2309999999999996e-06 2024-09-14T05:29:50.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.718 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.718 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211197+0000", 2024-09-14T05:29:50.718 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4199999999999998e-06 2024-09-14T05:29:50.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.718 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.718 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292017+0000", 2024-09-14T05:29:50.718 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080819516999999994 2024-09-14T05:29:50.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.719 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.719 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292087+0000", 2024-09-14T05:29:50.719 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0056999999999997e-05 2024-09-14T05:29:50.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.719 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.719 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292296+0000", 2024-09-14T05:29:50.719 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000209507 2024-09-14T05:29:50.719 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.719 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.719 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.720 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27766 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.720 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:29:50.720 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4924950859999999, 2024-09-14T05:29:50.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081359077000000002, 2024-09-14T05:29:50.720 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.720 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.720 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.720 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.720 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.720 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.720 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27766 2024-09-14T05:29:50.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.720 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.721 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.721 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:29:50.721 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.721 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.721 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:29:50.721 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.721 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.721 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211205+0000", 2024-09-14T05:29:50.721 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7e-07 2024-09-14T05:29:50.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.722 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.722 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211207+0000", 2024-09-14T05:29:50.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4540000000000001e-06 2024-09-14T05:29:50.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.722 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.722 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211216+0000", 2024-09-14T05:29:50.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3680000000000002e-06 2024-09-14T05:29:50.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.722 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.722 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211221+0000", 2024-09-14T05:29:50.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3499999999999996e-06 2024-09-14T05:29:50.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.723 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.723 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292329+0000", 2024-09-14T05:29:50.723 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081108240999999998 2024-09-14T05:29:50.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.723 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.723 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292370+0000", 2024-09-14T05:29:50.723 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.121e-05 2024-09-14T05:29:50.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.723 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.723 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292563+0000", 2024-09-14T05:29:50.724 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019298399999999999 2024-09-14T05:29:50.724 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.724 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.724 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.724 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27767 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.724 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:29:50.724 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4924715720000004, 2024-09-14T05:29:50.724 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081623119999999993, 2024-09-14T05:29:50.725 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.725 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.725 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.725 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.725 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.725 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.725 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27767 2024-09-14T05:29:50.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.725 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.725 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.725 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.726 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:29:50.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.726 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.726 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:29:50.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.726 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.726 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:29:50.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0499999999999999e-07 2024-09-14T05:29:50.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211230+0000", 2024-09-14T05:29:50.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0700000000000001e-06 2024-09-14T05:29:50.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211236+0000", 2024-09-14T05:29:50.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8429999999999998e-06 2024-09-14T05:29:50.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211241+0000", 2024-09-14T05:29:50.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:29:50.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292594+0000", 2024-09-14T05:29:50.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081352868999999994 2024-09-14T05:29:50.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292631+0000", 2024-09-14T05:29:50.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7083999999999999e-05 2024-09-14T05:29:50.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292851+0000", 2024-09-14T05:29:50.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022011700000000001 2024-09-14T05:29:50.729 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.729 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.729 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.729 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27768 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 274432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.729 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:29:50.729 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4924530359999997, 2024-09-14T05:29:50.729 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081875032, 2024-09-14T05:29:50.729 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.729 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.729 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.730 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.730 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.730 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.730 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27768 2024-09-14T05:29:50.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.730 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:29:50.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:29:50.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211247+0000", 2024-09-14T05:29:50.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.46e-07 2024-09-14T05:29:50.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211248+0000", 2024-09-14T05:29:50.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7349999999999999e-06 2024-09-14T05:29:50.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.732 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211253+0000", 2024-09-14T05:29:50.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.899e-06 2024-09-14T05:29:50.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.732 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.732 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211258+0000", 2024-09-14T05:29:50.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2810000000000002e-06 2024-09-14T05:29:50.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.732 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.732 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292888+0000", 2024-09-14T05:29:50.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081629955000000004 2024-09-14T05:29:50.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292926+0000", 2024-09-14T05:29:50.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7947000000000001e-05 2024-09-14T05:29:50.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293121+0000", 2024-09-14T05:29:50.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019586899999999999 2024-09-14T05:29:50.733 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.733 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.733 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.733 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27769 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 294912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.733 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:29:50.734 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4924349469999996, 2024-09-14T05:29:50.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082181899000000003, 2024-09-14T05:29:50.734 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.734 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.734 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.734 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.734 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.734 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.734 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27769 2024-09-14T05:29:50.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.734 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:29:50.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:29:50.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211265+0000", 2024-09-14T05:29:50.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2199999999999999e-07 2024-09-14T05:29:50.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211267+0000", 2024-09-14T05:29:50.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2239999999999998e-06 2024-09-14T05:29:50.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211273+0000", 2024-09-14T05:29:50.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6540000000000004e-06 2024-09-14T05:29:50.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211277+0000", 2024-09-14T05:29:50.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6299999999999997e-06 2024-09-14T05:29:50.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293150+0000", 2024-09-14T05:29:50.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081872575000000003 2024-09-14T05:29:50.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293245+0000", 2024-09-14T05:29:50.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5327000000000004e-05 2024-09-14T05:29:50.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293446+0000", 2024-09-14T05:29:50.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000201067 2024-09-14T05:29:50.737 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.738 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.738 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.738 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27770 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.738 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:29:50.738 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4924147850000002, 2024-09-14T05:29:50.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082663707000000003, 2024-09-14T05:29:50.738 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.738 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.738 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.738 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.738 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.738 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.739 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27770 2024-09-14T05:29:50.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.739 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:29:50.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:29:50.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.740 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.740 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:29:50.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1199999999999998e-07 2024-09-14T05:29:50.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.740 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.740 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211292+0000", 2024-09-14T05:29:50.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1609999999999997e-06 2024-09-14T05:29:50.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.740 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.740 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211299+0000", 2024-09-14T05:29:50.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3300000000000004e-06 2024-09-14T05:29:50.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.741 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.741 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211303+0000", 2024-09-14T05:29:50.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7859999999999996e-06 2024-09-14T05:29:50.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.741 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.741 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293628+0000", 2024-09-14T05:29:50.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082324283999999998 2024-09-14T05:29:50.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.741 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.742 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293694+0000", 2024-09-14T05:29:50.742 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6277000000000003e-05 2024-09-14T05:29:50.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.742 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293948+0000", 2024-09-14T05:29:50.742 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025445699999999998 2024-09-14T05:29:50.742 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.742 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.742 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.743 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27790 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.743 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:29:50.743 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4923883809999996, 2024-09-14T05:29:50.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082461006000000003, 2024-09-14T05:29:50.743 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.743 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.743 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.743 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.743 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.743 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.743 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27790 2024-09-14T05:29:50.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.743 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.744 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:29:50.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.744 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:29:50.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.744 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211312+0000", 2024-09-14T05:29:50.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9299999999999998e-07 2024-09-14T05:29:50.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.745 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211314+0000", 2024-09-14T05:29:50.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000002e-06 2024-09-14T05:29:50.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.745 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211320+0000", 2024-09-14T05:29:50.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1589999999999998e-06 2024-09-14T05:29:50.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.745 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211325+0000", 2024-09-14T05:29:50.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9609999999999996e-06 2024-09-14T05:29:50.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.746 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.746 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293474+0000", 2024-09-14T05:29:50.746 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082149080999999999 2024-09-14T05:29:50.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.746 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.746 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293542+0000", 2024-09-14T05:29:50.746 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8497000000000005e-05 2024-09-14T05:29:50.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.746 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.746 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293772+0000", 2024-09-14T05:29:50.746 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022969900000000001 2024-09-14T05:29:50.746 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.747 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.747 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.747 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27791 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.747 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:29:50.747 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4923680670000001, 2024-09-14T05:29:50.747 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082700838999999998, 2024-09-14T05:29:50.747 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.747 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.747 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.747 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.747 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.747 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.748 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27791 2024-09-14T05:29:50.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.748 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.748 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.748 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:29:50.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.748 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.748 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:29:50.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211332+0000", 2024-09-14T05:29:50.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7399999999999999e-07 2024-09-14T05:29:50.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211334+0000", 2024-09-14T05:29:50.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8500000000000001e-06 2024-09-14T05:29:50.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211340+0000", 2024-09-14T05:29:50.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3010000000000003e-06 2024-09-14T05:29:50.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211345+0000", 2024-09-14T05:29:50.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.938e-06 2024-09-14T05:29:50.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293797+0000", 2024-09-14T05:29:50.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082452152000000001 2024-09-14T05:29:50.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293869+0000", 2024-09-14T05:29:50.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2236000000000004e-05 2024-09-14T05:29:50.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.751 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.751 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294032+0000", 2024-09-14T05:29:50.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016298800000000001 2024-09-14T05:29:50.751 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.751 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.751 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.751 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27792 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.751 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:29:50.751 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4923485359999997, 2024-09-14T05:29:50.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082897304000000005, 2024-09-14T05:29:50.752 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.752 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.752 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.752 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.752 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.752 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.752 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27792 2024-09-14T05:29:50.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.752 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:29:50.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:29:50.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:29:50.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2000000000000001e-07 2024-09-14T05:29:50.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211353+0000", 2024-09-14T05:29:50.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.761e-06 2024-09-14T05:29:50.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211359+0000", 2024-09-14T05:29:50.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2430000000000002e-06 2024-09-14T05:29:50.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211364+0000", 2024-09-14T05:29:50.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0239999999999997e-06 2024-09-14T05:29:50.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294059+0000", 2024-09-14T05:29:50.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082694834999999994 2024-09-14T05:29:50.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.755 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294114+0000", 2024-09-14T05:29:50.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4823e-05 2024-09-14T05:29:50.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.755 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294248+0000", 2024-09-14T05:29:50.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013429800000000001 2024-09-14T05:29:50.755 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.755 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.755 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.756 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27793 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.756 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:29:50.756 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4923287370000002, 2024-09-14T05:29:50.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083153362999999994, 2024-09-14T05:29:50.756 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.756 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.756 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.757 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.757 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.757 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.757 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27793 2024-09-14T05:29:50.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.757 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:29:50.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:29:50.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:29:50.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5100000000000001e-07 2024-09-14T05:29:50.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211373+0000", 2024-09-14T05:29:50.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.001e-06 2024-09-14T05:29:50.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211379+0000", 2024-09-14T05:29:50.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4559999999999997e-06 2024-09-14T05:29:50.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211384+0000", 2024-09-14T05:29:50.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9749999999999998e-06 2024-09-14T05:29:50.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294279+0000", 2024-09-14T05:29:50.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082894282999999999 2024-09-14T05:29:50.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294327+0000", 2024-09-14T05:29:50.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8208000000000003e-05 2024-09-14T05:29:50.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294524+0000", 2024-09-14T05:29:50.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019708900000000001 2024-09-14T05:29:50.761 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.761 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.761 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.761 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27794 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.761 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:29:50.761 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4923084129999999, 2024-09-14T05:29:50.761 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588590000000004, 2024-09-14T05:29:50.761 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.761 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.761 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.761 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.761 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.762 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.762 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27794 2024-09-14T05:29:50.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.762 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:29:50.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:29:50.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:29:50.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4499999999999997e-07 2024-09-14T05:29:50.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.763 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211393+0000", 2024-09-14T05:29:50.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.815e-06 2024-09-14T05:29:50.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.763 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211400+0000", 2024-09-14T05:29:50.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2899999999999999e-06 2024-09-14T05:29:50.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211404+0000", 2024-09-14T05:29:50.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:29:50.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294560+0000", 2024-09-14T05:29:50.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083155534000000003 2024-09-14T05:29:50.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294631+0000", 2024-09-14T05:29:50.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1378999999999995e-05 2024-09-14T05:29:50.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294980+0000", 2024-09-14T05:29:50.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00034844900000000002 2024-09-14T05:29:50.765 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.765 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.765 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.765 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27795 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.766 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:29:50.766 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4922890549999996, 2024-09-14T05:29:50.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083970367000000004, 2024-09-14T05:29:50.766 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.766 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.766 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.766 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.766 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.766 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.766 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27795 2024-09-14T05:29:50.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.766 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:29:50.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:29:50.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211411+0000", 2024-09-14T05:29:50.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5499999999999999e-07 2024-09-14T05:29:50.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211413+0000", 2024-09-14T05:29:50.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:29:50.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211419+0000", 2024-09-14T05:29:50.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1720000000000004e-06 2024-09-14T05:29:50.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211424+0000", 2024-09-14T05:29:50.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0690000000000001e-06 2024-09-14T05:29:50.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.769 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.769 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295013+0000", 2024-09-14T05:29:50.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588898999999994 2024-09-14T05:29:50.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.769 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.769 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295102+0000", 2024-09-14T05:29:50.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9462000000000005e-05 2024-09-14T05:29:50.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.769 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.769 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295381+0000", 2024-09-14T05:29:50.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000278566 2024-09-14T05:29:50.769 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.770 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.770 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.770 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27796 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.770 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:29:50.770 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4922693259999997, 2024-09-14T05:29:50.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084379107999999994, 2024-09-14T05:29:50.770 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.770 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.770 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.770 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.770 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.770 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.770 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27796 2024-09-14T05:29:50.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.771 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:29:50.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:29:50.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:29:50.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2800000000000002e-07 2024-09-14T05:29:50.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211432+0000", 2024-09-14T05:29:50.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:29:50.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211439+0000", 2024-09-14T05:29:50.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5239999999999997e-06 2024-09-14T05:29:50.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211444+0000", 2024-09-14T05:29:50.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.95e-06 2024-09-14T05:29:50.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295602+0000", 2024-09-14T05:29:50.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084158017000000002 2024-09-14T05:29:50.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295674+0000", 2024-09-14T05:29:50.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2385999999999994e-05 2024-09-14T05:29:50.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.774 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295809+0000", 2024-09-14T05:29:50.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013495900000000001 2024-09-14T05:29:50.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.774 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.774 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27797 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 266240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.774 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:29:50.774 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4922493440000002, 2024-09-14T05:29:50.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084898597000000006, 2024-09-14T05:29:50.774 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.775 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.775 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.775 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.775 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.775 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.775 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27797 2024-09-14T05:29:50.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.775 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:29:50.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:29:50.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:29:50.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:29:50.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211452+0000", 2024-09-14T05:29:50.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.708e-06 2024-09-14T05:29:50.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211458+0000", 2024-09-14T05:29:50.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3489999999999997e-06 2024-09-14T05:29:50.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211464+0000", 2024-09-14T05:29:50.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3820000000000003e-06 2024-09-14T05:29:50.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295839+0000", 2024-09-14T05:29:50.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084375055000000004 2024-09-14T05:29:50.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295918+0000", 2024-09-14T05:29:50.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9253999999999997e-05 2024-09-14T05:29:50.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296349+0000", 2024-09-14T05:29:50.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043051 2024-09-14T05:29:50.778 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.778 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.778 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.779 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27798 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.779 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:29:50.779 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4922296289999997, 2024-09-14T05:29:50.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085300275999999994, 2024-09-14T05:29:50.779 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.779 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.779 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.780 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.780 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.780 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.780 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27798 2024-09-14T05:29:50.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.780 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.780 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:29:50.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:29:50.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:29:50.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:29:50.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211472+0000", 2024-09-14T05:29:50.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7579999999999999e-06 2024-09-14T05:29:50.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211478+0000", 2024-09-14T05:29:50.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0310000000000004e-06 2024-09-14T05:29:50.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211483+0000", 2024-09-14T05:29:50.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8110000000000002e-06 2024-09-14T05:29:50.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296469+0000", 2024-09-14T05:29:50.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084986575999999994 2024-09-14T05:29:50.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296545+0000", 2024-09-14T05:29:50.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5749999999999998e-05 2024-09-14T05:29:50.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296770+0000", 2024-09-14T05:29:50.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022498900000000001 2024-09-14T05:29:50.783 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.783 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.783 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.783 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27799 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 307200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.784 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:29:50.784 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4922107159999998, 2024-09-14T05:29:50.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085718461999999995, 2024-09-14T05:29:50.784 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.784 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.784 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.784 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.784 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.784 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.784 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27799 2024-09-14T05:29:50.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.784 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:29:50.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:29:50.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:29:50.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8000000000000001e-07 2024-09-14T05:29:50.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211491+0000", 2024-09-14T05:29:50.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-14T05:29:50.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211497+0000", 2024-09-14T05:29:50.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.857e-06 2024-09-14T05:29:50.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211502+0000", 2024-09-14T05:29:50.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9790000000000001e-06 2024-09-14T05:29:50.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296800+0000", 2024-09-14T05:29:50.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085298099000000002 2024-09-14T05:29:50.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296871+0000", 2024-09-14T05:29:50.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0879999999999999e-05 2024-09-14T05:29:50.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297207+0000", 2024-09-14T05:29:50.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033651699999999999 2024-09-14T05:29:50.789 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.789 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.789 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.789 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27826 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.789 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:29:50.789 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4904955949999996, 2024-09-14T05:29:50.789 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084306729999999996, 2024-09-14T05:29:50.789 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.789 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.789 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.789 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.790 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.790 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.790 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27826 2024-09-14T05:29:50.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.790 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:29:50.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:29:50.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213205+0000", 2024-09-14T05:29:50.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.277e-06 2024-09-14T05:29:50.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213210+0000", 2024-09-14T05:29:50.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3120000000000001e-06 2024-09-14T05:29:50.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213226+0000", 2024-09-14T05:29:50.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5279000000000001e-05 2024-09-14T05:29:50.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213234+0000", 2024-09-14T05:29:50.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7020000000000007e-06 2024-09-14T05:29:50.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297239+0000", 2024-09-14T05:29:50.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084004876000000006 2024-09-14T05:29:50.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297311+0000", 2024-09-14T05:29:50.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1336999999999998e-05 2024-09-14T05:29:50.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297511+0000", 2024-09-14T05:29:50.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019994699999999999 2024-09-14T05:29:50.793 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.793 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.793 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.793 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27827 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:50.794 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:29:50.794 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4904504080000001, 2024-09-14T05:29:50.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084520537000000007, 2024-09-14T05:29:50.794 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:50.794 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:50.794 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:50.794 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:50.794 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:50.794 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:50.794 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27827 2024-09-14T05:29:50.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.794 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:50.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:50.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:29:50.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:50.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:29:50.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:50.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:50.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213250+0000", 2024-09-14T05:29:50.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1099999999999995e-07 2024-09-14T05:29:50.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:50.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213252+0000", 2024-09-14T05:29:50.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6699999999999998e-06 2024-09-14T05:29:50.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:50.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213259+0000", 2024-09-14T05:29:50.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8469999999999998e-06 2024-09-14T05:29:50.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:50.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213265+0000", 2024-09-14T05:29:50.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4940000000000003e-06 2024-09-14T05:29:50.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:50.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297541+0000", 2024-09-14T05:29:50.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084276731999999993 2024-09-14T05:29:50.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:50.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297579+0000", 2024-09-14T05:29:50.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8108999999999997e-05 2024-09-14T05:29:50.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:50.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:50.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:50.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297770+0000", 2024-09-14T05:29:50.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019007400000000001 2024-09-14T05:29:50.798 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.798 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.798 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.798 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:50.798 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:50.798 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:50.799 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:29:50.800 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:50.798+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:50.852 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:29:50.900 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:50.900+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:50.904 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:50.904 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:50.904 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:29:50.904 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:50.912 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:29:50.972 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:29:50.972 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:29:50.988 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:29:50.988 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:29:51.002 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:51.000+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:51.004 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:29:51.004 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:29:51.020 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:29:51.020 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:29:51.036 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:29:51.036 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:29:51.052 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:29:51.052 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:29:51.056 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:51.056 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:51.064 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:29:51.068 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:29:51.068 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:29:51.085 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:29:51.085 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:29:51.109 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:51.108+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:51.173 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:51.173 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:51.182 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:29:51.205 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:51.203+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:51.223 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone16 from 0x67aeb19 (overlap 0x92deab) to 0x66cf11 2024-09-14T05:29:51.306 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:51.305+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:51.320 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-14T05:29:51.342 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:51.342 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:51.354 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:29:51.439 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:51.438+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:51.507 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:51.506+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:51.526 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:51.526 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:51.538 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:29:51.608 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:51.606+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:51.685 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:51.685 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:51.694 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:29:51.725 INFO:teuthology.orchestra.run.smithi071.stdout:1752 read 0x24f0a95 thru 0x24fab29 (0xa095 bytes) 2024-09-14T05:29:51.740 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:51.739+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:51.743 INFO:teuthology.orchestra.run.smithi071.stdout:1753 writesame 0xd10730d thru 0xd10fd47 data_size 0x1(0x8a3b bytes) 2024-09-14T05:29:51.743 INFO:teuthology.orchestra.run.smithi071.stdout:1755 writesame 0xc925d3 thru 0xca116b data_size 0x1(0xeb99 bytes) 2024-09-14T05:29:51.810 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:51.808+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:51.810 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:51.810 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:51.815 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:51.815 INFO:teuthology.orchestra.run.smithi071.stdout:1757 punch from 0x42ebfd3 to 0x42f380b, (0x7838 bytes) 2024-09-14T05:29:51.815 INFO:teuthology.orchestra.run.smithi071.stdout:1758 punch from 0x5beefec to 0x5bf8906, (0x991a bytes) 2024-09-14T05:29:51.819 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:29:51.820 INFO:teuthology.orchestra.run.smithi071.stdout:1759 trunc from 0xd10fd48 to 0x8b19ac0 2024-09-14T05:29:51.835 INFO:teuthology.orchestra.run.smithi071.stdout:1760 write 0xc2852b3 thru 0xc28e547 (0x9295 bytes) 2024-09-14T05:29:51.839 INFO:teuthology.orchestra.run.smithi071.stdout:1761 read 0x6fd9ed5 thru 0x6fe5ca3 (0xbdcf bytes) 2024-09-14T05:29:51.839 INFO:teuthology.orchestra.run.smithi071.stdout:1763 read 0x7a2f813 thru 0x7a3a06d (0xa85b bytes) 2024-09-14T05:29:51.839 INFO:teuthology.orchestra.run.smithi071.stdout:1764 write 0x7cedf9a thru 0x7cf81a0 (0xa207 bytes) 2024-09-14T05:29:51.844 INFO:teuthology.orchestra.run.smithi071.stdout:1765 write 0x7ddff73 thru 0x7def717 (0xf7a5 bytes) 2024-09-14T05:29:51.848 INFO:teuthology.orchestra.run.smithi071.stdout:1766 write 0x2c443a1 thru 0x2c4dba8 (0x9808 bytes) 2024-09-14T05:29:51.854 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:51.854 INFO:teuthology.orchestra.run.smithi071.stdout:1769 write 0x2d45e65 thru 0x2d505f3 (0xa78f bytes) 2024-09-14T05:29:51.861 INFO:teuthology.orchestra.run.smithi071.stdout:1770 read 0x7d8a7dd thru 0x7d94c13 (0xa437 bytes) 2024-09-14T05:29:51.862 INFO:teuthology.orchestra.run.smithi071.stdout:1771 write 0x8c4c41f thru 0x8c5b425 (0xf007 bytes) 2024-09-14T05:29:51.863 INFO:teuthology.orchestra.run.smithi071.stdout:1772 read 0x31df0ba thru 0x31e63d1 (0x7318 bytes) 2024-09-14T05:29:51.863 INFO:teuthology.orchestra.run.smithi071.stdout:1773 write 0x3cd9864 thru 0x3cdc40d (0x2baa bytes) 2024-09-14T05:29:51.867 INFO:teuthology.orchestra.run.smithi071.stdout:1774 read 0x4dc4067 thru 0x4dcaa55 (0x69ef bytes) 2024-09-14T05:29:51.867 INFO:teuthology.orchestra.run.smithi071.stdout:1775 read 0x55f239e thru 0x55f367f (0x12e2 bytes) 2024-09-14T05:29:51.868 INFO:teuthology.orchestra.run.smithi071.stdout:1778 trunc from 0xc28e548 to 0xb460cc5 2024-09-14T05:29:51.878 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:51.878 INFO:teuthology.orchestra.run.smithi071.stdout:1780 write 0xb2e31ed thru 0xb2eb7eb (0x85ff bytes) 2024-09-14T05:29:51.891 INFO:teuthology.orchestra.run.smithi071.stdout:1781 write 0xe09b56d thru 0xe09d221 (0x1cb5 bytes) 2024-09-14T05:29:51.893 INFO:teuthology.orchestra.run.smithi071.stdout:1784 read 0x2495aee thru 0x24a0d2e (0xb241 bytes) 2024-09-14T05:29:51.894 INFO:teuthology.orchestra.run.smithi071.stdout:1785 read 0x4073dfd thru 0x407add6 (0x6fda bytes) 2024-09-14T05:29:51.894 INFO:teuthology.orchestra.run.smithi071.stdout:1786 punch from 0xdab2d70 to 0xdab659d, (0x382d bytes) 2024-09-14T05:29:51.895 INFO:teuthology.orchestra.run.smithi071.stdout:1787 punch from 0xab3301 to 0xac1beb, (0xe8ea bytes) 2024-09-14T05:29:51.895 INFO:teuthology.orchestra.run.smithi071.stdout:1788 write 0xb10ff3e thru 0xb11e077 (0xe13a bytes) 2024-09-14T05:29:51.898 INFO:teuthology.orchestra.run.smithi071.stdout:1789 writesame 0xd91e3c3 thru 0xd925213 data_size 0x1(0x6e51 bytes) 2024-09-14T05:29:51.910 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:51.909+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:51.911 INFO:teuthology.orchestra.run.smithi071.stdout:1791 writesame 0x5da42c4 thru 0x5db1ccd data_size 0x3f(0xda0a bytes) 2024-09-14T05:29:51.926 INFO:teuthology.orchestra.run.smithi071.stdout:1792 read 0x11d25c2 thru 0x11da60d (0x804c bytes) 2024-09-14T05:29:51.935 INFO:teuthology.orchestra.run.smithi071.stdout:1794 trunc from 0xe09d222 to 0x6d61b42 2024-09-14T05:29:51.975 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:51.975 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:51.984 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:29:51.997 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:51.998 INFO:teuthology.orchestra.run.smithi071.stdout:1797 punch from 0x2f2bf49 to 0x2f380cc, (0xc183 bytes) 2024-09-14T05:29:52.013 INFO:teuthology.orchestra.run.smithi071.stdout:1798 writesame 0xea3f13c thru 0xea4ec26 data_size 0x5(0xfaeb bytes) 2024-09-14T05:29:52.013 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.013 INFO:teuthology.orchestra.run.smithi071.stdout:1802 write 0xa8d2230 thru 0xa8dd1b9 (0xaf8a bytes) 2024-09-14T05:29:52.018 INFO:teuthology.orchestra.run.smithi071.stdout:1803 read 0x8918d2e thru 0x891c2c4 (0x3597 bytes) 2024-09-14T05:29:52.018 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.018 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.018 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.018 INFO:teuthology.orchestra.run.smithi071.stdout:1808 write 0xe32bed3 thru 0xe330847 (0x4975 bytes) 2024-09-14T05:29:52.025 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.028 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:52.028+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:52.030 INFO:teuthology.orchestra.run.smithi071.stdout:1810 trunc from 0xea4ec27 to 0xa19d3db 2024-09-14T05:29:52.039 INFO:teuthology.orchestra.run.smithi071.stdout:1811 punch from 0x7bf6347 to 0x7c001ff, (0x9eb8 bytes) 2024-09-14T05:29:52.039 INFO:teuthology.orchestra.run.smithi071.stdout:1812 read 0x1b6f539 thru 0x1b792ca (0x9d92 bytes) 2024-09-14T05:29:52.043 INFO:teuthology.orchestra.run.smithi071.stdout:1814 trunc from 0xa19d3db to 0xcd48b28 2024-09-14T05:29:52.049 INFO:teuthology.orchestra.run.smithi071.stdout:1816 read 0x1b9500b thru 0x1b98bfb (0x3bf1 bytes) 2024-09-14T05:29:52.049 INFO:teuthology.orchestra.run.smithi071.stdout:1817 read 0xbf0881c thru 0xbf0fe63 (0x7648 bytes) 2024-09-14T05:29:52.049 INFO:teuthology.orchestra.run.smithi071.stdout:1818 read 0x45021ac thru 0x451021b (0xe070 bytes) 2024-09-14T05:29:52.049 INFO:teuthology.orchestra.run.smithi071.stdout:1820 punch from 0x54bbe0d to 0x54be0c3, (0x22b6 bytes) 2024-09-14T05:29:52.049 INFO:teuthology.orchestra.run.smithi071.stdout:1821 write 0x8aa4f45 thru 0x8aa8c40 (0x3cfc bytes) 2024-09-14T05:29:52.055 INFO:teuthology.orchestra.run.smithi071.stdout:1823 read 0x6112a6b thru 0x6117ade (0x5074 bytes) 2024-09-14T05:29:52.056 INFO:teuthology.orchestra.run.smithi071.stdout:1824 read 0xa3499c2 thru 0xa34bcc8 (0x2307 bytes) 2024-09-14T05:29:52.056 INFO:teuthology.orchestra.run.smithi071.stdout:1825 punch from 0x87fe513 to 0x88016b1, (0x319e bytes) 2024-09-14T05:29:52.056 INFO:teuthology.orchestra.run.smithi071.stdout:1826 read 0x1170c51 thru 0x1173da8 (0x3158 bytes) 2024-09-14T05:29:52.056 INFO:teuthology.orchestra.run.smithi071.stdout:1827 punch from 0xae6417b to 0xae66f83, (0x2e08 bytes) 2024-09-14T05:29:52.064 INFO:teuthology.orchestra.run.smithi071.stdout:1828 write 0xe303be3 thru 0xe310eba (0xd2d8 bytes) 2024-09-14T05:29:52.069 INFO:teuthology.orchestra.run.smithi071.stdout:1829 write 0x4e89270 thru 0x4e9038f (0x7120 bytes) 2024-09-14T05:29:52.080 INFO:teuthology.orchestra.run.smithi071.stdout:1831 read 0xd60c5dd thru 0xd61b927 (0xf34b bytes) 2024-09-14T05:29:52.080 INFO:teuthology.orchestra.run.smithi071.stdout:1832 write 0x91c239b thru 0x91cebf2 (0xc858 bytes) 2024-09-14T05:29:52.084 INFO:teuthology.orchestra.run.smithi071.stdout:1833 read 0xa60b8d1 thru 0xa60f212 (0x3942 bytes) 2024-09-14T05:29:52.084 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.084 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.084 INFO:teuthology.orchestra.run.smithi071.stdout:1836 write 0x67ea14d thru 0x67f7692 (0xd546 bytes) 2024-09-14T05:29:52.086 INFO:teuthology.orchestra.run.smithi071.stdout:1837 writesame 0x69c25a0 thru 0x69c4a3a data_size 0x1(0x249b bytes) 2024-09-14T05:29:52.090 INFO:teuthology.orchestra.run.smithi071.stdout:1838 writesame 0x6f79708 thru 0x6f8573b data_size 0x2(0xc034 bytes) 2024-09-14T05:29:52.093 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:52.093 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:52.097 INFO:teuthology.orchestra.run.smithi071.stdout:1839 write 0x17764cf thru 0x177696e (0x4a0 bytes) 2024-09-14T05:29:52.100 INFO:teuthology.orchestra.run.smithi071.stdout:1840 trunc from 0xe310ebb to 0xcab2048 2024-09-14T05:29:52.103 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:29:52.110 INFO:teuthology.orchestra.run.smithi071.stdout:1841 writesame 0x5c842aa thru 0x5c94284 data_size 0x7(0xffdb bytes) 2024-09-14T05:29:52.121 INFO:teuthology.orchestra.run.smithi071.stdout:1842 trunc from 0xcab2048 to 0x7556024 2024-09-14T05:29:52.133 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.139 INFO:teuthology.orchestra.run.smithi071.stdout:1845 trunc from 0x7556024 to 0x2626b3c 2024-09-14T05:29:52.143 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:52.143+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:52.170 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T05:29:52.173 INFO:teuthology.orchestra.run.smithi071.stdout:1846 read 0x1df6655 thru 0x1e03061 (0xca0d bytes) 2024-09-14T05:29:52.175 INFO:teuthology.orchestra.run.smithi071.stdout:1848 trunc from 0x2626b3c to 0x12427e3 2024-09-14T05:29:52.213 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:52.211+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:52.288 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:52.288 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:52.297 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:29:52.315 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:52.312+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:52.361 INFO:teuthology.orchestra.run.smithi071.stdout:1849 trunc from 0x12427e3 to 0x231a67c 2024-09-14T05:29:52.366 INFO:teuthology.orchestra.run.smithi071.stdout:1850 punch from 0x11df777 to 0x11e3f38, (0x47c1 bytes) 2024-09-14T05:29:52.366 INFO:teuthology.orchestra.run.smithi071.stdout:1852 read 0x60a4c3 thru 0x618ff3 (0xeb31 bytes) 2024-09-14T05:29:52.383 INFO:teuthology.orchestra.run.smithi071.stdout:1853 writesame 0xb05225b thru 0xb05685d data_size 0x1(0x4603 bytes) 2024-09-14T05:29:52.384 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.388 INFO:teuthology.orchestra.run.smithi071.stdout:1856 writesame 0xb220fae thru 0xb227120 data_size 0x13(0x6173 bytes) 2024-09-14T05:29:52.388 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.388 INFO:teuthology.orchestra.run.smithi071.stdout:1858 punch from 0x8b29302 to 0x8b30020, (0x6d1e bytes) 2024-09-14T05:29:52.397 INFO:teuthology.orchestra.run.smithi071.stdout:1859 writesame 0xd614001 thru 0xd61a598 data_size 0x4(0x6598 bytes) 2024-09-14T05:29:52.397 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.397 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.398 INFO:teuthology.orchestra.run.smithi071.stdout:1863 read 0x2bf7f50 thru 0x2bf7ffb (0xac bytes) 2024-09-14T05:29:52.398 INFO:teuthology.orchestra.run.smithi071.stdout:1865 punch from 0xb52555b to 0xb52a049, (0x4aee bytes) 2024-09-14T05:29:52.398 INFO:teuthology.orchestra.run.smithi071.stdout:1866 write 0x82656 thru 0x8c914 (0xa2bf bytes) 2024-09-14T05:29:52.406 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:52.406 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:52.410 INFO:teuthology.orchestra.run.smithi071.stdout:1867 trunc from 0xd61a599 to 0x3f8edeb 2024-09-14T05:29:52.415 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:52.413+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:52.416 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:29:52.432 INFO:teuthology.orchestra.run.smithi071.stdout:1868 writesame 0xc686754 thru 0xc68ec47 data_size 0x7f(0x84f4 bytes) 2024-09-14T05:29:52.432 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.434 INFO:teuthology.orchestra.run.smithi071.stdout:1870 trunc from 0xc68ec48 to 0xdfd2727 2024-09-14T05:29:52.439 INFO:teuthology.orchestra.run.smithi071.stdout:1871 read 0x52e273e thru 0x52e572b (0x2fee bytes) 2024-09-14T05:29:52.440 INFO:teuthology.orchestra.run.smithi071.stdout:1872 write 0xac32aa2 thru 0xac3362b (0xb8a bytes) 2024-09-14T05:29:52.442 INFO:teuthology.orchestra.run.smithi071.stdout:1873 write 0xdea0933 thru 0xdea9427 (0x8af5 bytes) 2024-09-14T05:29:52.448 INFO:teuthology.orchestra.run.smithi071.stdout:1874 read 0x145f913 thru 0x146583e (0x5f2c bytes) 2024-09-14T05:29:52.448 INFO:teuthology.orchestra.run.smithi071.stdout:1875 writesame 0xcef4149 thru 0xcef6b59 data_size 0xb(0x2a11 bytes) 2024-09-14T05:29:52.453 INFO:teuthology.orchestra.run.smithi071.stdout:1876 read 0x791a092 thru 0x7927cf4 (0xdc63 bytes) 2024-09-14T05:29:52.453 INFO:teuthology.orchestra.run.smithi071.stdout:1877 read 0x2b9420b thru 0x2b9ae67 (0x6c5d bytes) 2024-09-14T05:29:52.453 INFO:teuthology.orchestra.run.smithi071.stdout:1878 write 0xac5365 thru 0xad2318 (0xcfb4 bytes) 2024-09-14T05:29:52.457 INFO:teuthology.orchestra.run.smithi071.stdout:1882 punch from 0xb1bb79a to 0xb1c55ca, (0x9e30 bytes) 2024-09-14T05:29:52.457 INFO:teuthology.orchestra.run.smithi071.stdout:1883 write 0x8ea798f thru 0x8eb09e1 (0x9053 bytes) 2024-09-14T05:29:52.461 INFO:teuthology.orchestra.run.smithi071.stdout:1884 write 0xc2ad174 thru 0xc2b05d6 (0x3463 bytes) 2024-09-14T05:29:52.468 INFO:teuthology.orchestra.run.smithi071.stdout:1887 writesame 0x9e7465a thru 0x9e7e785 data_size 0xa(0xa12c bytes) 2024-09-14T05:29:52.473 INFO:teuthology.orchestra.run.smithi071.stdout:1889 read 0x3267fa9 thru 0x3277754 (0xf7ac bytes) 2024-09-14T05:29:52.473 INFO:teuthology.orchestra.run.smithi071.stdout:1890 read 0x99b22b1 thru 0x99b90aa (0x6dfa bytes) 2024-09-14T05:29:52.473 INFO:teuthology.orchestra.run.smithi071.stdout:1891 punch from 0x929354c to 0x929c849, (0x92fd bytes) 2024-09-14T05:29:52.473 INFO:teuthology.orchestra.run.smithi071.stdout:1892 read 0x2f5a938 thru 0x2f640fd (0x97c6 bytes) 2024-09-14T05:29:52.473 INFO:teuthology.orchestra.run.smithi071.stdout:1893 read 0xba53d44 thru 0xba5f0da (0xb397 bytes) 2024-09-14T05:29:52.473 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.473 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.474 INFO:teuthology.orchestra.run.smithi071.stdout:1897 read 0x2124026 thru 0x2125954 (0x192f bytes) 2024-09-14T05:29:52.474 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.489 INFO:teuthology.orchestra.run.smithi071.stdout:1900 trunc from 0xdfd2727 to 0x207e7c 2024-09-14T05:29:52.505 INFO:teuthology.orchestra.run.smithi071.stdout:1901 punch from 0x1b6892 to 0x1c4cf2, (0xe460 bytes) 2024-09-14T05:29:52.505 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.505 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.505 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:52.506 INFO:teuthology.orchestra.run.smithi071.stdout:1907 read 0x2d84c thru 0x38dfe (0xb5b3 bytes) 2024-09-14T05:29:52.515 INFO:teuthology.orchestra.run.smithi071.stdout:1908 writesame 0x2db67bb thru 0x2db83b0 data_size 0x3(0x1bf6 bytes) 2024-09-14T05:29:52.515 INFO:teuthology.orchestra.run.smithi071.stdout:1909 punch from 0x1442c5f to 0x1449c4b, (0x6fec bytes) 2024-09-14T05:29:52.515 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:52.513+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:52.526 INFO:teuthology.orchestra.run.smithi071.stdout:1911 trunc from 0x2db83b1 to 0xc8bee64 2024-09-14T05:29:52.526 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:52.526 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:52.531 INFO:teuthology.orchestra.run.smithi071.stdout:1912 read 0x14b1cd thru 0x15ab68 (0xf99c bytes) 2024-09-14T05:29:52.536 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:29:52.537 INFO:teuthology.orchestra.run.smithi071.stdout:1913 trunc from 0xc8bee64 to 0x8320faa 2024-09-14T05:29:52.543 INFO:teuthology.orchestra.run.smithi071.stdout:1914 write 0x255c29a thru 0x2568b0c (0xc873 bytes) 2024-09-14T05:29:52.552 INFO:teuthology.orchestra.run.smithi071.stdout:1915 punch from 0x378e454 to 0x378ff9f, (0x1b4b bytes) 2024-09-14T05:29:52.559 INFO:teuthology.orchestra.run.smithi071.stdout:1916 trunc from 0x8320faa to 0x27c0f5b 2024-09-14T05:29:52.573 INFO:teuthology.orchestra.run.smithi071.stdout:1917 write 0x3c1c2ce thru 0x3c1fd29 (0x3a5c bytes) 2024-09-14T05:29:52.576 INFO:teuthology.orchestra.run.smithi071.stdout:1918 read 0x224178 thru 0x226c12 (0x2a9b bytes) 2024-09-14T05:29:52.589 INFO:teuthology.orchestra.run.smithi071.stdout:1919 trunc from 0x3c1fd2a to 0xdb36443 2024-09-14T05:29:52.605 INFO:teuthology.orchestra.run.smithi071.stdout:1920 trunc from 0xdb36443 to 0x560996a 2024-09-14T05:29:52.613 INFO:teuthology.orchestra.run.smithi071.stdout:1921 trunc from 0x560996a to 0x4cacf83 2024-09-14T05:29:52.615 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:52.615+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:52.618 INFO:teuthology.orchestra.run.smithi071.stdout:1923 write 0x11175e thru 0x11ffda (0xe87d bytes) 2024-09-14T05:29:52.628 INFO:teuthology.orchestra.run.smithi071.stdout:1924 writesame 0x6b0a043 thru 0x6b13697 data_size 0x5(0x9655 bytes) 2024-09-14T05:29:52.628 INFO:teuthology.orchestra.run.smithi071.stdout:1927 write 0xe81b95 thru 0xe86b14 (0x4f80 bytes) 2024-09-14T05:29:52.629 INFO:teuthology.orchestra.run.smithi071.stdout:1928 clone 18 order 19 su 65536 sc 11 2024-09-14T05:29:52.651 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:52.651 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:52.661 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:29:52.745 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:52.743+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:52.803 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:52.803 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:52.812 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:29:52.817 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:52.816+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:52.918 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:52.916+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:52.956 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:52.956 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:52.964 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:29:53.049 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:53.047+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:53.096 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:29:53.109 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:53.109 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:53.118 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:29:53.120 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:53.119+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:53.161 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-14T05:29:53.231 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:29:53.231 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:29:53.249 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:29:53.249 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:29:53.262 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:53.219+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:53.266 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:29:53.266 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:29:53.273 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:53.273 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:53.273 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:53.273 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:53.273 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:53.280 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone17 from 0x6b13698 (overlap 0x207e7c) to 0x1d3ea2 2024-09-14T05:29:53.283 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:29:53.286 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:29:53.286 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:29:53.308 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:29:53.309 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:29:53.321 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:53.320+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:53.329 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:29:53.329 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:29:53.346 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:29:53.346 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:29:53.365 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:29:53.365 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:29:53.388 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-14T05:29:53.415 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:53.416 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:53.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:53.416 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:53.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.416 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:29:53.416 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:29:53.416 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.636686390000001, 2024-09-14T05:29:53.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.24562984099999999, 2024-09-14T05:29:53.416 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.416 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.417 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:53.417 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.417 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:29:53.417 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:29:53.417 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1 2024-09-14T05:29:53.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.417 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:29:53.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:29:53.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776565+0000", 2024-09-14T05:29:53.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4780000000000001e-06 2024-09-14T05:29:53.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776571+0000", 2024-09-14T05:29:53.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7669999999999998e-06 2024-09-14T05:29:53.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776580+0000", 2024-09-14T05:29:53.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9830000000000002e-06 2024-09-14T05:29:53.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776589+0000", 2024-09-14T05:29:53.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7579999999999998e-06 2024-09-14T05:29:53.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776662+0000", 2024-09-14T05:29:53.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3332999999999997e-05 2024-09-14T05:29:53.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:29:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776670+0000", 2024-09-14T05:29:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9850000000000006e-06 2024-09-14T05:29:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.020867+0000", 2024-09-14T05:29:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.244196265 2024-09-14T05:29:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021022+0000", 2024-09-14T05:29:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000155699 2024-09-14T05:29:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021169+0000", 2024-09-14T05:29:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014661700000000001 2024-09-14T05:29:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:29:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021891+0000", 2024-09-14T05:29:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072146700000000005 2024-09-14T05:29:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022124+0000", 2024-09-14T05:29:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023354199999999999 2024-09-14T05:29:53.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:53.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022164+0000", 2024-09-14T05:29:53.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9897000000000001e-05 2024-09-14T05:29:53.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022193+0000", 2024-09-14T05:29:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9050000000000001e-05 2024-09-14T05:29:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:1989 3.1f e30/24 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head v 30'29, mlcod=30'27)", 2024-09-14T05:29:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:29:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.819915403, 2024-09-14T05:29:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.064242492999999998, 2024-09-14T05:29:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:29:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:29:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593336+0000", 2024-09-14T05:29:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3909999999999999e-06 2024-09-14T05:29:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616632+0000", 2024-09-14T05:29:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.023296598000000002 2024-09-14T05:29:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616657+0000", 2024-09-14T05:29:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5228e-05 2024-09-14T05:29:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616684+0000", 2024-09-14T05:29:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6441999999999999e-05 2024-09-14T05:29:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616755+0000", 2024-09-14T05:29:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1452000000000006e-05 2024-09-14T05:29:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616780+0000", 2024-09-14T05:29:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4436000000000001e-05 2024-09-14T05:29:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:29:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657527+0000", 2024-09-14T05:29:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.040747323000000002 2024-09-14T05:29:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657577+0000", 2024-09-14T05:29:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9623000000000002e-05 2024-09-14T05:29:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70397 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4958208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.370202+0000", 2024-09-14T05:29:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.0430476869999996, 2024-09-14T05:29:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030143445000000001, 2024-09-14T05:29:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70397 2024-09-14T05:29:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370202+0000", 2024-09-14T05:29:53.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370202+0000", 2024-09-14T05:29:53.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370202+0000", 2024-09-14T05:29:53.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3000000000000002e-07 2024-09-14T05:29:53.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370204+0000", 2024-09-14T05:29:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7799999999999999e-06 2024-09-14T05:29:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370209+0000", 2024-09-14T05:29:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.169e-06 2024-09-14T05:29:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370214+0000", 2024-09-14T05:29:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2299999999999999e-06 2024-09-14T05:29:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400201+0000", 2024-09-14T05:29:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029986987999999999 2024-09-14T05:29:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400266+0000", 2024-09-14T05:29:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5061999999999995e-05 2024-09-14T05:29:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400345+0000", 2024-09-14T05:29:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8886000000000003e-05 2024-09-14T05:29:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70400 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4990976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.370229+0000", 2024-09-14T05:29:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.0430203779999996, 2024-09-14T05:29:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030287287, 2024-09-14T05:29:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70400 2024-09-14T05:29:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370229+0000", 2024-09-14T05:29:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370229+0000", 2024-09-14T05:29:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370230+0000", 2024-09-14T05:29:53.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3000000000000001e-07 2024-09-14T05:29:53.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370232+0000", 2024-09-14T05:29:53.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0209999999999998e-06 2024-09-14T05:29:53.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370238+0000", 2024-09-14T05:29:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9340000000000003e-06 2024-09-14T05:29:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370242+0000", 2024-09-14T05:29:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.566e-06 2024-09-14T05:29:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400373+0000", 2024-09-14T05:29:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030131116999999999 2024-09-14T05:29:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400439+0000", 2024-09-14T05:29:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6038999999999996e-05 2024-09-14T05:29:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400516+0000", 2024-09-14T05:29:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7180000000000003e-05 2024-09-14T05:29:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70413 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4960256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.438 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.370249+0000", 2024-09-14T05:29:53.438 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.0430005270000002, 2024-09-14T05:29:53.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030421191, 2024-09-14T05:29:53.438 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.438 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.438 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.438 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.438 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.438 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.438 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70413 2024-09-14T05:29:53.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.438 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.439 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370249+0000", 2024-09-14T05:29:53.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.439 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370249+0000", 2024-09-14T05:29:53.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.439 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370249+0000", 2024-09-14T05:29:53.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9400000000000001e-07 2024-09-14T05:29:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370251+0000", 2024-09-14T05:29:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.013e-06 2024-09-14T05:29:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370258+0000", 2024-09-14T05:29:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0809999999999999e-06 2024-09-14T05:29:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370262+0000", 2024-09-14T05:29:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7230000000000004e-06 2024-09-14T05:29:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400543+0000", 2024-09-14T05:29:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030281129 2024-09-14T05:29:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400593+0000", 2024-09-14T05:29:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9645000000000003e-05 2024-09-14T05:29:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400670+0000", 2024-09-14T05:29:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7206000000000006e-05 2024-09-14T05:29:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70416 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4993024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.370268+0000", 2024-09-14T05:29:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.0429817469999998, 2024-09-14T05:29:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030571867999999999, 2024-09-14T05:29:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70416 2024-09-14T05:29:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370268+0000", 2024-09-14T05:29:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370268+0000", 2024-09-14T05:29:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370268+0000", 2024-09-14T05:29:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7e-07 2024-09-14T05:29:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370270+0000", 2024-09-14T05:29:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8279999999999999e-06 2024-09-14T05:29:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370275+0000", 2024-09-14T05:29:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4110000000000004e-06 2024-09-14T05:29:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370280+0000", 2024-09-14T05:29:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6940000000000003e-06 2024-09-14T05:29:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400698+0000", 2024-09-14T05:29:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030418207999999999 2024-09-14T05:29:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400764+0000", 2024-09-14T05:29:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5717999999999997e-05 2024-09-14T05:29:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400840+0000", 2024-09-14T05:29:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5638999999999999e-05 2024-09-14T05:29:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70429 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4962304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.370286+0000", 2024-09-14T05:29:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.0429639740000001, 2024-09-14T05:29:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030741230000000001, 2024-09-14T05:29:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70429 2024-09-14T05:29:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370286+0000", 2024-09-14T05:29:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370286+0000", 2024-09-14T05:29:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370286+0000", 2024-09-14T05:29:53.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5600000000000001e-07 2024-09-14T05:29:53.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370301+0000", 2024-09-14T05:29:53.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5268999999999999e-05 2024-09-14T05:29:53.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370308+0000", 2024-09-14T05:29:53.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3430000000000001e-06 2024-09-14T05:29:53.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370313+0000", 2024-09-14T05:29:53.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0579999999999997e-06 2024-09-14T05:29:53.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400868+0000", 2024-09-14T05:29:53.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030555045999999999 2024-09-14T05:29:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400940+0000", 2024-09-14T05:29:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2304000000000004e-05 2024-09-14T05:29:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401027+0000", 2024-09-14T05:29:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6854000000000006e-05 2024-09-14T05:29:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.452 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70432 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4995072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.452 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.370319+0000", 2024-09-14T05:29:53.452 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.042930084, 2024-09-14T05:29:53.452 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030873629, 2024-09-14T05:29:53.452 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.452 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.452 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.452 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.452 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.452 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70432 2024-09-14T05:29:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370319+0000", 2024-09-14T05:29:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370319+0000", 2024-09-14T05:29:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370320+0000", 2024-09-14T05:29:53.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2399999999999999e-07 2024-09-14T05:29:53.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370322+0000", 2024-09-14T05:29:53.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.824e-06 2024-09-14T05:29:53.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370327+0000", 2024-09-14T05:29:53.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6219999999999997e-06 2024-09-14T05:29:53.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370332+0000", 2024-09-14T05:29:53.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3080000000000001e-06 2024-09-14T05:29:53.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401054+0000", 2024-09-14T05:29:53.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030722791999999999 2024-09-14T05:29:53.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401120+0000", 2024-09-14T05:29:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5309999999999996e-05 2024-09-14T05:29:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401193+0000", 2024-09-14T05:29:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3349000000000005e-05 2024-09-14T05:29:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70446 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4947968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.371258+0000", 2024-09-14T05:29:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.0419913220000003, 2024-09-14T05:29:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030101513999999999, 2024-09-14T05:29:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70446 2024-09-14T05:29:53.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.458 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371258+0000", 2024-09-14T05:29:53.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371258+0000", 2024-09-14T05:29:53.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371259+0000", 2024-09-14T05:29:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2200000000000003e-07 2024-09-14T05:29:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371263+0000", 2024-09-14T05:29:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8940000000000003e-06 2024-09-14T05:29:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371272+0000", 2024-09-14T05:29:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6449999999999994e-06 2024-09-14T05:29:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371278+0000", 2024-09-14T05:29:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6660000000000002e-06 2024-09-14T05:29:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401221+0000", 2024-09-14T05:29:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029942368 2024-09-14T05:29:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401285+0000", 2024-09-14T05:29:53.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4710999999999998e-05 2024-09-14T05:29:53.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.461 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401360+0000", 2024-09-14T05:29:53.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4407999999999996e-05 2024-09-14T05:29:53.461 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.461 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.461 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.461 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70448 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4980736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.461 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.371289+0000", 2024-09-14T05:29:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.0419601140000001, 2024-09-14T05:29:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030246810999999998, 2024-09-14T05:29:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70448 2024-09-14T05:29:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371289+0000", 2024-09-14T05:29:53.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371289+0000", 2024-09-14T05:29:53.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371290+0000", 2024-09-14T05:29:53.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2800000000000002e-07 2024-09-14T05:29:53.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371292+0000", 2024-09-14T05:29:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0049999999999999e-06 2024-09-14T05:29:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371297+0000", 2024-09-14T05:29:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6250000000000004e-06 2024-09-14T05:29:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371302+0000", 2024-09-14T05:29:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9960000000000001e-06 2024-09-14T05:29:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401387+0000", 2024-09-14T05:29:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030084818999999999 2024-09-14T05:29:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401451+0000", 2024-09-14T05:29:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3331999999999996e-05 2024-09-14T05:29:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401536+0000", 2024-09-14T05:29:53.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5606000000000007e-05 2024-09-14T05:29:53.466 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.466 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.466 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.466 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70461 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4950016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.466 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.371310+0000", 2024-09-14T05:29:53.466 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.0419397320000003, 2024-09-14T05:29:53.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030393710000000001, 2024-09-14T05:29:53.466 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.466 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.466 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70461 2024-09-14T05:29:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371310+0000", 2024-09-14T05:29:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371310+0000", 2024-09-14T05:29:53.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371310+0000", 2024-09-14T05:29:53.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4000000000000002e-07 2024-09-14T05:29:53.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371312+0000", 2024-09-14T05:29:53.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.04e-06 2024-09-14T05:29:53.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371318+0000", 2024-09-14T05:29:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7089999999999999e-06 2024-09-14T05:29:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371323+0000", 2024-09-14T05:29:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5820000000000003e-06 2024-09-14T05:29:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401563+0000", 2024-09-14T05:29:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030240466000000001 2024-09-14T05:29:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401629+0000", 2024-09-14T05:29:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5631999999999998e-05 2024-09-14T05:29:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401704+0000", 2024-09-14T05:29:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4840999999999997e-05 2024-09-14T05:29:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.471 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70464 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4982784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.471 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.371328+0000", 2024-09-14T05:29:53.471 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.0419212059999996, 2024-09-14T05:29:53.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030558068000000001, 2024-09-14T05:29:53.471 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.471 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.471 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.471 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.471 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.471 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.471 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70464 2024-09-14T05:29:53.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371328+0000", 2024-09-14T05:29:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371328+0000", 2024-09-14T05:29:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371329+0000", 2024-09-14T05:29:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0900000000000002e-07 2024-09-14T05:29:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371331+0000", 2024-09-14T05:29:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.792e-06 2024-09-14T05:29:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371335+0000", 2024-09-14T05:29:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9300000000000002e-06 2024-09-14T05:29:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371340+0000", 2024-09-14T05:29:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.605e-06 2024-09-14T05:29:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401730+0000", 2024-09-14T05:29:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030390225 2024-09-14T05:29:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401800+0000", 2024-09-14T05:29:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0166000000000005e-05 2024-09-14T05:29:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401886+0000", 2024-09-14T05:29:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5940999999999996e-05 2024-09-14T05:29:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70609 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4972544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.375484+0000", 2024-09-14T05:29:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.0377651920000002, 2024-09-14T05:29:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.031716407000000002, 2024-09-14T05:29:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.476 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.476 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.476 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.476 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.476 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.476 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70609 2024-09-14T05:29:53.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.476 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375484+0000", 2024-09-14T05:29:53.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375484+0000", 2024-09-14T05:29:53.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375485+0000", 2024-09-14T05:29:53.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6100000000000001e-07 2024-09-14T05:29:53.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375487+0000", 2024-09-14T05:29:53.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3700000000000002e-06 2024-09-14T05:29:53.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375494+0000", 2024-09-14T05:29:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.737e-06 2024-09-14T05:29:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375499+0000", 2024-09-14T05:29:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4500000000000003e-06 2024-09-14T05:29:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405175+0000", 2024-09-14T05:29:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029676022999999999 2024-09-14T05:29:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405243+0000", 2024-09-14T05:29:53.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7620000000000006e-05 2024-09-14T05:29:53.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407201+0000", 2024-09-14T05:29:53.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019577459999999998 2024-09-14T05:29:53.482 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.482 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.482 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.483 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70612 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 5005312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.483 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.375507+0000", 2024-09-14T05:29:53.483 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.0377428740000001, 2024-09-14T05:29:53.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030049329999999999, 2024-09-14T05:29:53.483 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.483 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.483 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.483 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.483 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.483 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.483 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70612 2024-09-14T05:29:53.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.484 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375507+0000", 2024-09-14T05:29:53.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375507+0000", 2024-09-14T05:29:53.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375507+0000", 2024-09-14T05:29:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:29:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375509+0000", 2024-09-14T05:29:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.967e-06 2024-09-14T05:29:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375516+0000", 2024-09-14T05:29:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.691e-06 2024-09-14T05:29:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375521+0000", 2024-09-14T05:29:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2510000000000002e-06 2024-09-14T05:29:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405380+0000", 2024-09-14T05:29:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029858787000000001 2024-09-14T05:29:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405446+0000", 2024-09-14T05:29:53.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5996999999999998e-05 2024-09-14T05:29:53.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405556+0000", 2024-09-14T05:29:53.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011021100000000001 2024-09-14T05:29:53.487 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.487 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.487 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70625 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4974592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.375529+0000", 2024-09-14T05:29:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.037720728, 2024-09-14T05:29:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030229536000000001, 2024-09-14T05:29:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70625 2024-09-14T05:29:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375529+0000", 2024-09-14T05:29:53.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375529+0000", 2024-09-14T05:29:53.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375529+0000", 2024-09-14T05:29:53.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8299999999999998e-07 2024-09-14T05:29:53.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375531+0000", 2024-09-14T05:29:53.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9659999999999999e-06 2024-09-14T05:29:53.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375538+0000", 2024-09-14T05:29:53.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5089999999999999e-06 2024-09-14T05:29:53.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375543+0000", 2024-09-14T05:29:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3959999999999997e-06 2024-09-14T05:29:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405581+0000", 2024-09-14T05:29:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030037828999999999 2024-09-14T05:29:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405648+0000", 2024-09-14T05:29:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7194999999999996e-05 2024-09-14T05:29:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405758+0000", 2024-09-14T05:29:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000110258 2024-09-14T05:29:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70628 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 5007360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.375620+0000", 2024-09-14T05:29:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.0376295830000002, 2024-09-14T05:29:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030354611, 2024-09-14T05:29:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.493 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.493 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.493 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.493 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.493 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70628 2024-09-14T05:29:53.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.493 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.493 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375620+0000", 2024-09-14T05:29:53.493 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375620+0000", 2024-09-14T05:29:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375621+0000", 2024-09-14T05:29:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9800000000000003e-07 2024-09-14T05:29:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375624+0000", 2024-09-14T05:29:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4429999999999999e-06 2024-09-14T05:29:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375633+0000", 2024-09-14T05:29:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6370000000000005e-06 2024-09-14T05:29:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375639+0000", 2024-09-14T05:29:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3149999999999997e-06 2024-09-14T05:29:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405784+0000", 2024-09-14T05:29:53.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030145110999999999 2024-09-14T05:29:53.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.496 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.496 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405851+0000", 2024-09-14T05:29:53.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7279000000000004e-05 2024-09-14T05:29:53.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.496 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.496 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405975+0000", 2024-09-14T05:29:53.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012312799999999999 2024-09-14T05:29:53.496 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.496 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.497 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.497 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70641 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4976640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.497 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.375666+0000", 2024-09-14T05:29:53.497 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.037584002, 2024-09-14T05:29:53.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030512713, 2024-09-14T05:29:53.497 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.497 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.497 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.497 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.497 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.497 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70641 2024-09-14T05:29:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375666+0000", 2024-09-14T05:29:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375666+0000", 2024-09-14T05:29:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375666+0000", 2024-09-14T05:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2500000000000005e-07 2024-09-14T05:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375669+0000", 2024-09-14T05:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9220000000000001e-06 2024-09-14T05:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375677+0000", 2024-09-14T05:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0809999999999994e-06 2024-09-14T05:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375683+0000", 2024-09-14T05:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6069999999999998e-06 2024-09-14T05:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.406000+0000", 2024-09-14T05:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030316777 2024-09-14T05:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.406072+0000", 2024-09-14T05:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1983000000000005e-05 2024-09-14T05:29:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.406178+0000", 2024-09-14T05:29:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000106718 2024-09-14T05:29:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70644 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 5009408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.375729+0000", 2024-09-14T05:29:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.0375207729999998, 2024-09-14T05:29:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030657856000000001, 2024-09-14T05:29:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70644 2024-09-14T05:29:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375729+0000", 2024-09-14T05:29:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375729+0000", 2024-09-14T05:29:53.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375729+0000", 2024-09-14T05:29:53.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6599999999999996e-07 2024-09-14T05:29:53.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375732+0000", 2024-09-14T05:29:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8499999999999998e-06 2024-09-14T05:29:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375743+0000", 2024-09-14T05:29:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0352e-05 2024-09-14T05:29:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375748+0000", 2024-09-14T05:29:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7899999999999996e-06 2024-09-14T05:29:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.406203+0000", 2024-09-14T05:29:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030454108000000001 2024-09-14T05:29:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.406272+0000", 2024-09-14T05:29:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8979999999999993e-05 2024-09-14T05:29:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.406387+0000", 2024-09-14T05:29:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011511 2024-09-14T05:29:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70689 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4966400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.377471+0000", 2024-09-14T05:29:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.0357786119999997, 2024-09-14T05:29:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029995048999999999, 2024-09-14T05:29:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70689 2024-09-14T05:29:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377471+0000", 2024-09-14T05:29:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377471+0000", 2024-09-14T05:29:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377471+0000", 2024-09-14T05:29:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8099999999999998e-07 2024-09-14T05:29:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377474+0000", 2024-09-14T05:29:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1780000000000002e-06 2024-09-14T05:29:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377480+0000", 2024-09-14T05:29:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0769999999999996e-06 2024-09-14T05:29:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377484+0000", 2024-09-14T05:29:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8040000000000001e-06 2024-09-14T05:29:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407257+0000", 2024-09-14T05:29:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029773079000000001 2024-09-14T05:29:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407324+0000", 2024-09-14T05:29:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6721e-05 2024-09-14T05:29:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407466+0000", 2024-09-14T05:29:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014180900000000001 2024-09-14T05:29:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70692 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4999168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.377495+0000", 2024-09-14T05:29:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.0357549700000002, 2024-09-14T05:29:53.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030228279, 2024-09-14T05:29:53.511 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.511 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.511 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.511 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.511 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.511 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.511 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70692 2024-09-14T05:29:53.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.511 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377495+0000", 2024-09-14T05:29:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377495+0000", 2024-09-14T05:29:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377495+0000", 2024-09-14T05:29:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7700000000000005e-07 2024-09-14T05:29:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377497+0000", 2024-09-14T05:29:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.181e-06 2024-09-14T05:29:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377503+0000", 2024-09-14T05:29:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2550000000000003e-06 2024-09-14T05:29:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377508+0000", 2024-09-14T05:29:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.967e-06 2024-09-14T05:29:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407517+0000", 2024-09-14T05:29:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030008970999999999 2024-09-14T05:29:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407585+0000", 2024-09-14T05:29:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7452000000000004e-05 2024-09-14T05:29:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407723+0000", 2024-09-14T05:29:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013797599999999999 2024-09-14T05:29:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.515 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.515 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.515 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.515 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.515 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:53.516 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:29:53.517 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:53.516+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:53.549 INFO:teuthology.orchestra.run.smithi071.stdout:1931 writesame 0x5eae513 thru 0x5ebde20 data_size 0x2(0xf90e bytes) 2024-09-14T05:29:53.566 INFO:teuthology.orchestra.run.smithi071.stdout:1933 read 0x42c350a thru 0x42d2b6b (0xf662 bytes) 2024-09-14T05:29:53.575 INFO:teuthology.orchestra.run.smithi071.stdout:1935 write 0x88b752b thru 0x88c4af9 (0xd5cf bytes) 2024-09-14T05:29:53.579 INFO:teuthology.orchestra.run.smithi071.stdout:1938 read 0xa89393 thru 0xa8a325 (0xf93 bytes) 2024-09-14T05:29:53.586 INFO:teuthology.orchestra.run.smithi071.stdout:1939 writesame 0x99a8ebd thru 0x99ac6b2 data_size 0x26(0x37f6 bytes) 2024-09-14T05:29:53.586 INFO:teuthology.orchestra.run.smithi071.stdout:1940 punch from 0x22eed45 to 0x22fdf45, (0xf200 bytes) 2024-09-14T05:29:53.586 INFO:teuthology.orchestra.run.smithi071.stdout:1942 read 0x159b3b0 thru 0x15a30e9 (0x7d3a bytes) 2024-09-14T05:29:53.587 INFO:teuthology.orchestra.run.smithi071.stdout:1943 read 0x76bd9bb thru 0x76c64bc (0x8b02 bytes) 2024-09-14T05:29:53.587 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:53.587 INFO:teuthology.orchestra.run.smithi071.stdout:1945 punch from 0x4749f3a to 0x4751390, (0x7456 bytes) 2024-09-14T05:29:53.587 INFO:teuthology.orchestra.run.smithi071.stdout:1946 punch from 0x63702b2 to 0x637ebb2, (0xe900 bytes) 2024-09-14T05:29:53.587 INFO:teuthology.orchestra.run.smithi071.stdout:1947 write 0x6a456cb thru 0x6a50a93 (0xb3c9 bytes) 2024-09-14T05:29:53.590 INFO:teuthology.orchestra.run.smithi071.stdout:1948 punch from 0x4773b05 to 0x477e427, (0xa922 bytes) 2024-09-14T05:29:53.590 INFO:teuthology.orchestra.run.smithi071.stdout:1949 read 0x74fa4db thru 0x74fbf93 (0x1ab9 bytes) 2024-09-14T05:29:53.591 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:53.601 INFO:teuthology.orchestra.run.smithi071.stdout:1951 trunc from 0x99ac6b3 to 0x450b05 2024-09-14T05:29:53.616 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:53.616+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:53.636 INFO:teuthology.orchestra.run.smithi071.stdout:1952 read 0x2b0650 thru 0x2b2633 (0x1fe4 bytes) 2024-09-14T05:29:53.637 INFO:teuthology.orchestra.run.smithi071.stdout:1953 read 0x30ec93 thru 0x30f36b (0x6d9 bytes) 2024-09-14T05:29:53.637 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:53.637 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:53.651 INFO:teuthology.orchestra.run.smithi071.stdout:1956 trunc from 0x450b05 to 0xc7b23ce 2024-09-14T05:29:53.656 INFO:teuthology.orchestra.run.smithi071.stdout:1958 writesame 0x87f9e02 thru 0x87fcd13 data_size 0xa(0x2f12 bytes) 2024-09-14T05:29:53.663 INFO:teuthology.orchestra.run.smithi071.stdout:1959 trunc from 0xc7b23ce to 0x9be41d6 2024-09-14T05:29:53.668 INFO:teuthology.orchestra.run.smithi071.stdout:1960 read 0x31bf77b thru 0x31c07c7 (0x104d bytes) 2024-09-14T05:29:53.668 INFO:teuthology.orchestra.run.smithi071.stdout:1961 read 0x719ef8b thru 0x71a0b94 (0x1c0a bytes) 2024-09-14T05:29:53.669 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:53.669 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:53.669 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:29:53.669 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:53.670 INFO:teuthology.orchestra.run.smithi071.stdout:1962 trunc from 0x9be41d6 to 0x7f6760f 2024-09-14T05:29:53.679 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:29:53.680 INFO:teuthology.orchestra.run.smithi071.stdout:1963 punch from 0x21dd0e5 to 0x21eaed2, (0xdded bytes) 2024-09-14T05:29:53.686 INFO:teuthology.orchestra.run.smithi071.stdout:1964 trunc from 0x7f6760f to 0x21044de 2024-09-14T05:29:53.692 INFO:teuthology.orchestra.run.smithi071.stdout:1965 read 0xae8ce0 thru 0xaf4ff7 (0xc318 bytes) 2024-09-14T05:29:53.702 INFO:teuthology.orchestra.run.smithi071.stdout:1966 writesame 0x59b77da thru 0x59bb0be data_size 0x5(0x38e5 bytes) 2024-09-14T05:29:53.702 INFO:teuthology.orchestra.run.smithi071.stdout:1967 punch from 0x70a5f0 to 0x70c5b6, (0x1fc6 bytes) 2024-09-14T05:29:53.702 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:53.702 INFO:teuthology.orchestra.run.smithi071.stdout:1969 punch from 0x1af76ab to 0x1af9cd4, (0x2629 bytes) 2024-09-14T05:29:53.702 INFO:teuthology.orchestra.run.smithi071.stdout:1970 punch from 0x1f7d6ce to 0x1f8cc0c, (0xf53e bytes) 2024-09-14T05:29:53.702 INFO:teuthology.orchestra.run.smithi071.stdout:1971 write 0x3b6d6cb thru 0x3b75ac9 (0x83ff bytes) 2024-09-14T05:29:53.717 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:53.717+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:53.723 INFO:teuthology.orchestra.run.smithi071.stdout:1972 writesame 0xd32992e thru 0xd330511 data_size 0xa(0x6be4 bytes) 2024-09-14T05:29:53.728 INFO:teuthology.orchestra.run.smithi071.stdout:1973 writesame 0xd6f1456 thru 0xd6f72d1 data_size 0x2(0x5e7c bytes) 2024-09-14T05:29:53.728 INFO:teuthology.orchestra.run.smithi071.stdout:1974 punch from 0x1111306 to 0x11192da, (0x7fd4 bytes) 2024-09-14T05:29:53.728 INFO:teuthology.orchestra.run.smithi071.stdout:1975 write 0x3efa412 thru 0x3f00aa5 (0x6694 bytes) 2024-09-14T05:29:53.734 INFO:teuthology.orchestra.run.smithi071.stdout:1976 read 0x39b022a thru 0x39b9ff2 (0x9dc9 bytes) 2024-09-14T05:29:53.734 INFO:teuthology.orchestra.run.smithi071.stdout:1977 read 0x350c939 thru 0x350f157 (0x281f bytes) 2024-09-14T05:29:53.734 INFO:teuthology.orchestra.run.smithi071.stdout:1978 read 0x863df2c thru 0x864c531 (0xe606 bytes) 2024-09-14T05:29:53.734 INFO:teuthology.orchestra.run.smithi071.stdout:1979 write 0x1af582e thru 0x1af821b (0x29ee bytes) 2024-09-14T05:29:53.738 INFO:teuthology.orchestra.run.smithi071.stdout:1980 write 0x9dae5eb thru 0x9dbb7a1 (0xd1b7 bytes) 2024-09-14T05:29:53.753 INFO:teuthology.orchestra.run.smithi071.stdout:1981 trunc from 0xd6f72d2 to 0x668cf0a 2024-09-14T05:29:53.764 INFO:teuthology.orchestra.run.smithi071.stdout:1983 read 0x4082b16 thru 0x4088914 (0x5dff bytes) 2024-09-14T05:29:53.764 INFO:teuthology.orchestra.run.smithi071.stdout:1984 read 0x2fc4b8b thru 0x2fc876a (0x3be0 bytes) 2024-09-14T05:29:53.770 INFO:teuthology.orchestra.run.smithi071.stdout:1985 write 0x6dd4bb0 thru 0x6dd7a6a (0x2ebb bytes) 2024-09-14T05:29:53.773 INFO:teuthology.orchestra.run.smithi071.stdout:1986 punch from 0x1044547 to 0x104d7ac, (0x9265 bytes) 2024-09-14T05:29:53.781 INFO:teuthology.orchestra.run.smithi071.stdout:1987 trunc from 0x6dd7a6b to 0xe11f8c8 2024-09-14T05:29:53.787 INFO:teuthology.orchestra.run.smithi071.stdout:1989 punch from 0x160bbf7 to 0x1619ff5, (0xe3fe bytes) 2024-09-14T05:29:53.787 INFO:teuthology.orchestra.run.smithi071.stdout:1990 read 0x197a7ea thru 0x19862c1 (0xbad8 bytes) 2024-09-14T05:29:53.787 INFO:teuthology.orchestra.run.smithi071.stdout:1991 write 0x8efd58d thru 0x8f08713 (0xb187 bytes) 2024-09-14T05:29:53.792 INFO:teuthology.orchestra.run.smithi071.stdout:1994 punch from 0xb344a95 to 0xb353dbd, (0xf328 bytes) 2024-09-14T05:29:53.793 INFO:teuthology.orchestra.run.smithi071.stdout:1995 writesame 0x9d73034 thru 0x9d797b2 data_size 0x7(0x677f bytes) 2024-09-14T05:29:53.794 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:53.794 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:53.794 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:53.794 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:53.794 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:53.797 INFO:teuthology.orchestra.run.smithi071.stdout:1996 punch from 0x66143e to 0x66654e, (0x5110 bytes) 2024-09-14T05:29:53.797 INFO:teuthology.orchestra.run.smithi071.stdout:1997 read 0x9ee4b95 thru 0x9ef3f05 (0xf371 bytes) 2024-09-14T05:29:53.797 INFO:teuthology.orchestra.run.smithi071.stdout:1998 read 0x44f83cf thru 0x44f94dc (0x110e bytes) 2024-09-14T05:29:53.797 INFO:teuthology.orchestra.run.smithi071.stdout:1999 writesame 0x55cad70 thru 0x55cfc51 data_size 0x17(0x4ee2 bytes) 2024-09-14T05:29:53.799 INFO:teuthology.orchestra.run.smithi071.stdout:2000 punch from 0xea3ab2 to 0xea8d51, (0x529f bytes) 2024-09-14T05:29:53.804 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:29:53.810 INFO:teuthology.orchestra.run.smithi071.stdout:2001 trunc from 0xe11f8c8 to 0x4f8ffe1 2024-09-14T05:29:53.820 INFO:teuthology.orchestra.run.smithi071.stdout:2003 read 0x27a48fe thru 0x27ad702 (0x8e05 bytes) 2024-09-14T05:29:53.821 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:53.819+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:53.827 INFO:teuthology.orchestra.run.smithi071.stdout:2004 trunc from 0x4f8ffe1 to 0xa9f8360 2024-09-14T05:29:53.832 INFO:teuthology.orchestra.run.smithi071.stdout:2005 read 0xc39f5c thru 0xc4104b (0x70f0 bytes) 2024-09-14T05:29:53.842 INFO:teuthology.orchestra.run.smithi071.stdout:2007 write 0xecf33dd thru 0xecf581a (0x243e bytes) 2024-09-14T05:29:53.844 INFO:teuthology.orchestra.run.smithi071.stdout:2008 write 0x4f934ef thru 0x4f96f45 (0x3a57 bytes) 2024-09-14T05:29:53.847 INFO:teuthology.orchestra.run.smithi071.stdout:2009 read 0xe04b0fb thru 0xe05a056 (0xef5c bytes) 2024-09-14T05:29:53.847 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:53.847 INFO:teuthology.orchestra.run.smithi071.stdout:2011 write 0xdda4267 thru 0xdda9bdf (0x5979 bytes) 2024-09-14T05:29:53.850 INFO:teuthology.orchestra.run.smithi071.stdout:2013 read 0x7e29db7 thru 0x7e2c244 (0x248e bytes) 2024-09-14T05:29:53.850 INFO:teuthology.orchestra.run.smithi071.stdout:2014 read 0x9fd2e4 thru 0xa0238f (0x50ac bytes) 2024-09-14T05:29:53.861 INFO:teuthology.orchestra.run.smithi071.stdout:2015 trunc from 0xecf581b to 0x5f9fac2 2024-09-14T05:29:53.870 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:53.883 INFO:teuthology.orchestra.run.smithi071.stdout:2017 write 0xc6f9c9c thru 0xc6fa4c8 (0x82d bytes) 2024-09-14T05:29:53.888 INFO:teuthology.orchestra.run.smithi071.stdout:2018 read 0xb05e044 thru 0xb06c1aa (0xe167 bytes) 2024-09-14T05:29:53.888 INFO:teuthology.orchestra.run.smithi071.stdout:2019 read 0x55e1648 thru 0x55ef94c (0xe305 bytes) 2024-09-14T05:29:53.888 INFO:teuthology.orchestra.run.smithi071.stdout:2020 punch from 0xbcd09d3 to 0xbcdaaea, (0xa117 bytes) 2024-09-14T05:29:53.888 INFO:teuthology.orchestra.run.smithi071.stdout:2021 read 0x29fb9fd thru 0x2a08aba (0xd0be bytes) 2024-09-14T05:29:53.888 INFO:teuthology.orchestra.run.smithi071.stdout:2022 read 0x1fa21ee thru 0x1fb17ec (0xf5ff bytes) 2024-09-14T05:29:53.888 INFO:teuthology.orchestra.run.smithi071.stdout:2024 read 0x77b7e06 thru 0x77c2408 (0xa603 bytes) 2024-09-14T05:29:53.888 INFO:teuthology.orchestra.run.smithi071.stdout:2025 read 0x9a9da99 thru 0x9aa25e1 (0x4b49 bytes) 2024-09-14T05:29:53.888 INFO:teuthology.orchestra.run.smithi071.stdout:2026 read 0x105be26 thru 0x105cbff (0xdda bytes) 2024-09-14T05:29:53.900 INFO:teuthology.orchestra.run.smithi071.stdout:2027 trunc from 0xc6fa4c9 to 0x1066e46 2024-09-14T05:29:53.917 INFO:teuthology.orchestra.run.smithi071.stdout:2030 write 0x32a7ebe thru 0x32afa6a (0x7bad bytes) 2024-09-14T05:29:53.919 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:53.918+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:53.929 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:53.929 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:53.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:53.930 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:53.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.930 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73757 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2816000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.930 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:29:53.930 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3679880070000001, 2024-09-14T05:29:53.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044614540000000001, 2024-09-14T05:29:53.930 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.930 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.930 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.930 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.930 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.930 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.931 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73757 2024-09-14T05:29:53.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.931 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:29:53.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:29:53.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559679+0000", 2024-09-14T05:29:53.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4000000000000002e-07 2024-09-14T05:29:53.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559681+0000", 2024-09-14T05:29:53.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1780000000000002e-06 2024-09-14T05:29:53.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559687+0000", 2024-09-14T05:29:53.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0270000000000001e-06 2024-09-14T05:29:53.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559692+0000", 2024-09-14T05:29:53.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1030000000000001e-06 2024-09-14T05:29:53.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604180+0000", 2024-09-14T05:29:53.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044487480000000003 2024-09-14T05:29:53.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604224+0000", 2024-09-14T05:29:53.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4357999999999998e-05 2024-09-14T05:29:53.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604293+0000", 2024-09-14T05:29:53.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8954000000000004e-05 2024-09-14T05:29:53.935 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.935 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.935 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.936 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73760 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2848768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.936 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:29:53.936 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3679679020000002, 2024-09-14T05:29:53.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.04469911, 2024-09-14T05:29:53.936 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.936 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.936 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.936 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.936 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.936 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.936 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73760 2024-09-14T05:29:53.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.937 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:29:53.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:29:53.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:29:53.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:29:53.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559701+0000", 2024-09-14T05:29:53.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1909999999999999e-06 2024-09-14T05:29:53.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559708+0000", 2024-09-14T05:29:53.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7279999999999998e-06 2024-09-14T05:29:53.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559713+0000", 2024-09-14T05:29:53.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8389999999999997e-06 2024-09-14T05:29:53.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604310+0000", 2024-09-14T05:29:53.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044597664000000002 2024-09-14T05:29:53.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604352+0000", 2024-09-14T05:29:53.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1236000000000003e-05 2024-09-14T05:29:53.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604398+0000", 2024-09-14T05:29:53.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6091000000000003e-05 2024-09-14T05:29:53.941 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.941 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.941 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.941 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73773 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2801664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.941 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:29:53.941 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3679477330000003, 2024-09-14T05:29:53.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941026000000002, 2024-09-14T05:29:53.941 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.941 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.942 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.942 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.942 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.942 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.942 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73773 2024-09-14T05:29:53.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.942 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.942 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.942 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:29:53.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:29:53.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:29:53.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8599999999999999e-07 2024-09-14T05:29:53.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559721+0000", 2024-09-14T05:29:53.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0540000000000002e-06 2024-09-14T05:29:53.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559729+0000", 2024-09-14T05:29:53.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1359999999999997e-06 2024-09-14T05:29:53.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559734+0000", 2024-09-14T05:29:53.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2140000000000004e-06 2024-09-14T05:29:53.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604441+0000", 2024-09-14T05:29:53.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044706828999999997 2024-09-14T05:29:53.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604522+0000", 2024-09-14T05:29:53.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0603000000000001e-05 2024-09-14T05:29:53.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604660+0000", 2024-09-14T05:29:53.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000137804 2024-09-14T05:29:53.946 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.946 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.946 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.947 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73777 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2834432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.947 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:29:53.947 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3679253320000004, 2024-09-14T05:29:53.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045116270999999999, 2024-09-14T05:29:53.947 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.947 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.947 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.948 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.948 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.948 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.948 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73777 2024-09-14T05:29:53.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.948 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:29:53.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:29:53.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559742+0000", 2024-09-14T05:29:53.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4099999999999999e-07 2024-09-14T05:29:53.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559744+0000", 2024-09-14T05:29:53.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0040000000000002e-06 2024-09-14T05:29:53.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559749+0000", 2024-09-14T05:29:53.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9739999999999999e-06 2024-09-14T05:29:53.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559754+0000", 2024-09-14T05:29:53.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9540000000000003e-06 2024-09-14T05:29:53.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604695+0000", 2024-09-14T05:29:53.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941003 2024-09-14T05:29:53.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.952 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604766+0000", 2024-09-14T05:29:53.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0296999999999995e-05 2024-09-14T05:29:53.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.952 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604857+0000", 2024-09-14T05:29:53.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1598e-05 2024-09-14T05:29:53.952 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.952 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.952 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.953 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73790 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.953 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:29:53.953 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3679053970000004, 2024-09-14T05:29:53.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316437000000001, 2024-09-14T05:29:53.953 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.953 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.953 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.953 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.954 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.954 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.954 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73790 2024-09-14T05:29:53.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.954 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:29:53.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:29:53.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:29:53.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7599999999999998e-07 2024-09-14T05:29:53.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559763+0000", 2024-09-14T05:29:53.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0880000000000002e-06 2024-09-14T05:29:53.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559769+0000", 2024-09-14T05:29:53.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6740000000000002e-06 2024-09-14T05:29:53.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559774+0000", 2024-09-14T05:29:53.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6110000000000004e-06 2024-09-14T05:29:53.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604905+0000", 2024-09-14T05:29:53.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045130827999999998 2024-09-14T05:29:53.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604982+0000", 2024-09-14T05:29:53.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7410999999999995e-05 2024-09-14T05:29:53.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605077+0000", 2024-09-14T05:29:53.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5449e-05 2024-09-14T05:29:53.958 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.958 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.958 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.958 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73793 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2836480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.958 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:29:53.959 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3678861930000004, 2024-09-14T05:29:53.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045441212000000002, 2024-09-14T05:29:53.959 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.959 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.959 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.959 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.959 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.959 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.959 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73793 2024-09-14T05:29:53.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.959 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:29:53.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:29:53.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559781+0000", 2024-09-14T05:29:53.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2899999999999999e-07 2024-09-14T05:29:53.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559783+0000", 2024-09-14T05:29:53.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.075e-06 2024-09-14T05:29:53.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559788+0000", 2024-09-14T05:29:53.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5369999999999997e-06 2024-09-14T05:29:53.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559793+0000", 2024-09-14T05:29:53.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:29:53.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605109+0000", 2024-09-14T05:29:53.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316229999999999 2024-09-14T05:29:53.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605168+0000", 2024-09-14T05:29:53.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8949000000000001e-05 2024-09-14T05:29:53.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605221+0000", 2024-09-14T05:29:53.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3260000000000002e-05 2024-09-14T05:29:53.964 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.964 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.964 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.964 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73806 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.964 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:29:53.964 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3678675670000002, 2024-09-14T05:29:53.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045526189000000002, 2024-09-14T05:29:53.964 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.964 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.964 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.965 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.965 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.965 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.965 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73806 2024-09-14T05:29:53.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.965 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:29:53.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:29:53.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:29:53.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:29:53.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559801+0000", 2024-09-14T05:29:53.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.995e-06 2024-09-14T05:29:53.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559807+0000", 2024-09-14T05:29:53.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5450000000000003e-06 2024-09-14T05:29:53.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559811+0000", 2024-09-14T05:29:53.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5090000000000004e-06 2024-09-14T05:29:53.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605237+0000", 2024-09-14T05:29:53.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045426133 2024-09-14T05:29:53.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605278+0000", 2024-09-14T05:29:53.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1084999999999997e-05 2024-09-14T05:29:53.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605325+0000", 2024-09-14T05:29:53.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6582999999999998e-05 2024-09-14T05:29:53.969 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.969 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.969 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.970 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73809 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2838528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.970 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:29:53.970 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3678440619999996, 2024-09-14T05:29:53.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045596062999999999, 2024-09-14T05:29:53.970 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.970 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.970 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.971 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.971 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.971 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.971 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73809 2024-09-14T05:29:53.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.971 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:29:53.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:29:53.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559823+0000", 2024-09-14T05:29:53.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.08e-07 2024-09-14T05:29:53.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559825+0000", 2024-09-14T05:29:53.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.26e-06 2024-09-14T05:29:53.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559831+0000", 2024-09-14T05:29:53.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6690000000000003e-06 2024-09-14T05:29:53.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559835+0000", 2024-09-14T05:29:53.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5120000000000002e-06 2024-09-14T05:29:53.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605342+0000", 2024-09-14T05:29:53.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045506957000000001 2024-09-14T05:29:53.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605375+0000", 2024-09-14T05:29:53.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3250999999999997e-05 2024-09-14T05:29:53.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605418+0000", 2024-09-14T05:29:53.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3006000000000001e-05 2024-09-14T05:29:53.975 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.975 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.975 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.975 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73822 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2807808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.975 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:29:53.976 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3678164089999996, 2024-09-14T05:29:53.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045661405000000002, 2024-09-14T05:29:53.976 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.976 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.976 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.976 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.976 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.976 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.976 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73822 2024-09-14T05:29:53.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.976 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:29:53.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:29:53.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559851+0000", 2024-09-14T05:29:53.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4000000000000002e-07 2024-09-14T05:29:53.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559853+0000", 2024-09-14T05:29:53.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5380000000000001e-06 2024-09-14T05:29:53.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559859+0000", 2024-09-14T05:29:53.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0619999999999997e-06 2024-09-14T05:29:53.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559864+0000", 2024-09-14T05:29:53.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.977e-06 2024-09-14T05:29:53.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605435+0000", 2024-09-14T05:29:53.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045570672 2024-09-14T05:29:53.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605467+0000", 2024-09-14T05:29:53.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2486e-05 2024-09-14T05:29:53.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605511+0000", 2024-09-14T05:29:53.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4129999999999999e-05 2024-09-14T05:29:53.981 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.981 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.981 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.981 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73825 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2840576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.981 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:29:53.982 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.367793045, 2024-09-14T05:29:53.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045728694, 2024-09-14T05:29:53.982 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.982 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.982 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.982 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.982 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.982 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.982 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73825 2024-09-14T05:29:53.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.982 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:29:53.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:29:53.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559874+0000", 2024-09-14T05:29:53.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5400000000000002e-07 2024-09-14T05:29:53.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559876+0000", 2024-09-14T05:29:53.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.131e-06 2024-09-14T05:29:53.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559888+0000", 2024-09-14T05:29:53.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2216e-05 2024-09-14T05:29:53.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559893+0000", 2024-09-14T05:29:53.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6539999999999998e-06 2024-09-14T05:29:53.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605526+0000", 2024-09-14T05:29:53.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045633119 2024-09-14T05:29:53.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605560+0000", 2024-09-14T05:29:53.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3825999999999998e-05 2024-09-14T05:29:53.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605602+0000", 2024-09-14T05:29:53.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2293999999999999e-05 2024-09-14T05:29:53.987 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.987 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.987 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.987 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74023 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2863104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.987 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:29:53.987 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3666070699999997, 2024-09-14T05:29:53.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044635385999999999, 2024-09-14T05:29:53.987 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.988 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.988 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.988 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.988 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.988 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.988 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74023 2024-09-14T05:29:53.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.988 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:29:53.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:29:53.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561060+0000", 2024-09-14T05:29:53.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0699999999999998e-07 2024-09-14T05:29:53.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561072+0000", 2024-09-14T05:29:53.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2261e-05 2024-09-14T05:29:53.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561079+0000", 2024-09-14T05:29:53.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8399999999999997e-06 2024-09-14T05:29:53.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561084+0000", 2024-09-14T05:29:53.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7380000000000002e-06 2024-09-14T05:29:53.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605620+0000", 2024-09-14T05:29:53.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044536146999999998 2024-09-14T05:29:53.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605653+0000", 2024-09-14T05:29:53.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2969000000000002e-05 2024-09-14T05:29:53.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605695+0000", 2024-09-14T05:29:53.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2024000000000003e-05 2024-09-14T05:29:53.993 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.994 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.994 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.994 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74026 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2895872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:53.994 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:29:53.994 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3665756079999998, 2024-09-14T05:29:53.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044692717, 2024-09-14T05:29:53.994 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:53.994 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:53.994 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:53.995 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:53.995 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:53.995 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:53.995 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74026 2024-09-14T05:29:53.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.995 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:53.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:53.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:29:53.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:53.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:29:53.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:53.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:53.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:29:53.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:29:53.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:53.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561093+0000", 2024-09-14T05:29:53.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-14T05:29:53.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:53.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561099+0000", 2024-09-14T05:29:53.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0399999999999998e-06 2024-09-14T05:29:53.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:53.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561104+0000", 2024-09-14T05:29:53.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8180000000000003e-06 2024-09-14T05:29:53.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:53.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605710+0000", 2024-09-14T05:29:53.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044605657 2024-09-14T05:29:53.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:53.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605742+0000", 2024-09-14T05:29:53.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1949000000000002e-05 2024-09-14T05:29:53.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:53.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:53.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:53.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605784+0000", 2024-09-14T05:29:53.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1884000000000001e-05 2024-09-14T05:29:53.999 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:53.999 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:53.999 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.000 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74039 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:54.000 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:29:54.000 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3664490100000002, 2024-09-14T05:29:54.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044654905000000002, 2024-09-14T05:29:54.000 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.000 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.000 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.000 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.000 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.001 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.001 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74039 2024-09-14T05:29:54.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.001 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:29:54.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:29:54.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561218+0000", 2024-09-14T05:29:54.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4000000000000001e-07 2024-09-14T05:29:54.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561221+0000", 2024-09-14T05:29:54.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9730000000000002e-06 2024-09-14T05:29:54.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561229+0000", 2024-09-14T05:29:54.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1759999999999993e-06 2024-09-14T05:29:54.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561235+0000", 2024-09-14T05:29:54.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0109999999999997e-06 2024-09-14T05:29:54.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605798+0000", 2024-09-14T05:29:54.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044563285000000001 2024-09-14T05:29:54.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605831+0000", 2024-09-14T05:29:54.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2054999999999997e-05 2024-09-14T05:29:54.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605872+0000", 2024-09-14T05:29:54.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1764999999999997e-05 2024-09-14T05:29:54.005 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.005 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.005 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.005 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74042 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:54.006 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:29:54.006 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3664005570000004, 2024-09-14T05:29:54.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044773234000000002, 2024-09-14T05:29:54.006 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.006 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.006 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.006 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.006 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.006 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.007 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74042 2024-09-14T05:29:54.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.007 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:29:54.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:29:54.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:29:54.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4099999999999998e-07 2024-09-14T05:29:54.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561269+0000", 2024-09-14T05:29:54.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7930000000000002e-06 2024-09-14T05:29:54.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561278+0000", 2024-09-14T05:29:54.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5150000000000006e-06 2024-09-14T05:29:54.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561284+0000", 2024-09-14T05:29:54.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2029999999999997e-06 2024-09-14T05:29:54.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605934+0000", 2024-09-14T05:29:54.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044650449000000002 2024-09-14T05:29:54.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605983+0000", 2024-09-14T05:29:54.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8943999999999998e-05 2024-09-14T05:29:54.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606039+0000", 2024-09-14T05:29:54.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5689000000000002e-05 2024-09-14T05:29:54.011 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.011 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.011 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.011 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74052 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:54.011 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:29:54.011 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3663736819999999, 2024-09-14T05:29:54.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044830474000000002, 2024-09-14T05:29:54.011 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.011 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.011 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.012 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.012 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.012 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.012 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74052 2024-09-14T05:29:54.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.012 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:29:54.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:29:54.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:29:54.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1099999999999996e-07 2024-09-14T05:29:54.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561295+0000", 2024-09-14T05:29:54.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7770000000000001e-06 2024-09-14T05:29:54.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561303+0000", 2024-09-14T05:29:54.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6860000000000006e-06 2024-09-14T05:29:54.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561308+0000", 2024-09-14T05:29:54.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.023e-06 2024-09-14T05:29:54.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605893+0000", 2024-09-14T05:29:54.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044585458000000001 2024-09-14T05:29:54.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605933+0000", 2024-09-14T05:29:54.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9551999999999997e-05 2024-09-14T05:29:54.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606123+0000", 2024-09-14T05:29:54.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019046699999999999 2024-09-14T05:29:54.016 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.016 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.016 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.017 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74054 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:54.017 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:29:54.017 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3663516549999999, 2024-09-14T05:29:54.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093610999999999, 2024-09-14T05:29:54.017 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.017 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.017 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.017 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.017 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.017 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.017 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74054 2024-09-14T05:29:54.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.018 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:29:54.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:29:54.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:29:54.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9699999999999996e-07 2024-09-14T05:29:54.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561317+0000", 2024-09-14T05:29:54.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9740000000000001e-06 2024-09-14T05:29:54.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561324+0000", 2024-09-14T05:29:54.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4590000000000003e-06 2024-09-14T05:29:54.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561329+0000", 2024-09-14T05:29:54.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5969999999999999e-06 2024-09-14T05:29:54.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606142+0000", 2024-09-14T05:29:54.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044812347000000002 2024-09-14T05:29:54.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606184+0000", 2024-09-14T05:29:54.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2441999999999998e-05 2024-09-14T05:29:54.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606408+0000", 2024-09-14T05:29:54.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022429500000000001 2024-09-14T05:29:54.022 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.022 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.022 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.022 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74056 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:54.022 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:29:54.023 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3661569519999999, 2024-09-14T05:29:54.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045191685000000002, 2024-09-14T05:29:54.023 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.023 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.023 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.023 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.023 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.023 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.023 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74056 2024-09-14T05:29:54.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.023 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:29:54.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:29:54.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561510+0000", 2024-09-14T05:29:54.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2399999999999998e-07 2024-09-14T05:29:54.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561513+0000", 2024-09-14T05:29:54.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7219999999999999e-06 2024-09-14T05:29:54.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561524+0000", 2024-09-14T05:29:54.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1357e-05 2024-09-14T05:29:54.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561530+0000", 2024-09-14T05:29:54.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2890000000000003e-06 2024-09-14T05:29:54.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606438+0000", 2024-09-14T05:29:54.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044907326999999997 2024-09-14T05:29:54.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606520+0000", 2024-09-14T05:29:54.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1924999999999999e-05 2024-09-14T05:29:54.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606701+0000", 2024-09-14T05:29:54.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00018144100000000001 2024-09-14T05:29:54.027 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.027 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.027 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.027 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74058 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:54.028 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:29:54.028 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3656782789999999, 2024-09-14T05:29:54.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093368000000002, 2024-09-14T05:29:54.028 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.028 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.028 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.028 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.028 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.028 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.028 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74058 2024-09-14T05:29:54.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.029 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:29:54.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:29:54.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561989+0000", 2024-09-14T05:29:54.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3300000000000001e-07 2024-09-14T05:29:54.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561993+0000", 2024-09-14T05:29:54.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1790000000000001e-06 2024-09-14T05:29:54.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562007+0000", 2024-09-14T05:29:54.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3434e-05 2024-09-14T05:29:54.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562014+0000", 2024-09-14T05:29:54.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.537e-06 2024-09-14T05:29:54.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606736+0000", 2024-09-14T05:29:54.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044722117999999998 2024-09-14T05:29:54.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606829+0000", 2024-09-14T05:29:54.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2410999999999993e-05 2024-09-14T05:29:54.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607082+0000", 2024-09-14T05:29:54.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025285599999999998 2024-09-14T05:29:54.033 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.033 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.033 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.033 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74059 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:54.033 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:29:54.033 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3656202879999997, 2024-09-14T05:29:54.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045479928000000003, 2024-09-14T05:29:54.034 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.034 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.034 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.034 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.034 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.034 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.034 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74059 2024-09-14T05:29:54.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.034 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:29:54.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:29:54.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562047+0000", 2024-09-14T05:29:54.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6899999999999997e-07 2024-09-14T05:29:54.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562050+0000", 2024-09-14T05:29:54.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.943e-06 2024-09-14T05:29:54.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562058+0000", 2024-09-14T05:29:54.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0120000000000005e-06 2024-09-14T05:29:54.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562063+0000", 2024-09-14T05:29:54.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:29:54.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607186+0000", 2024-09-14T05:29:54.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045122348999999999 2024-09-14T05:29:54.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607265+0000", 2024-09-14T05:29:54.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9462999999999995e-05 2024-09-14T05:29:54.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607526+0000", 2024-09-14T05:29:54.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026090400000000001 2024-09-14T05:29:54.038 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.039 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.039 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.039 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74060 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2850816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:29:54.039 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:29:54.039 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.3655949109999996, 2024-09-14T05:29:54.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045200092999999997, 2024-09-14T05:29:54.039 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.039 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.039 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.040 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.040 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.040 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.040 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74060 2024-09-14T05:29:54.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.040 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:29:54.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:29:54.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:29:54.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5200000000000002e-07 2024-09-14T05:29:54.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562075+0000", 2024-09-14T05:29:54.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5200000000000002e-06 2024-09-14T05:29:54.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562082+0000", 2024-09-14T05:29:54.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8929999999999999e-06 2024-09-14T05:29:54.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562088+0000", 2024-09-14T05:29:54.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5570000000000003e-06 2024-09-14T05:29:54.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607111+0000", 2024-09-14T05:29:54.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045023464999999999 2024-09-14T05:29:54.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607190+0000", 2024-09-14T05:29:54.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8721999999999996e-05 2024-09-14T05:29:54.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607272+0000", 2024-09-14T05:29:54.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1483999999999994e-05 2024-09-14T05:29:54.044 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.044 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.044 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.044 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.044 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.044 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:54.045 INFO:teuthology.orchestra.run.smithi071.stdout:2031 write 0x8b84a12 thru 0x8b91c9e (0xd28d bytes) 2024-09-14T05:29:54.046 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:29:54.048 INFO:teuthology.orchestra.run.smithi071.stdout:2032 write 0xcaad749 thru 0xcaaf87a (0x2132 bytes) 2024-09-14T05:29:54.048 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:54.049 INFO:teuthology.orchestra.run.smithi071.stdout:2034 trunc from 0xcaaf87b to 0x256ec49 2024-09-14T05:29:54.049 INFO:teuthology.orchestra.run.smithi071.stdout:2035 writesame 0x4768ce6 thru 0x476b357 data_size 0x26(0x2672 bytes) 2024-09-14T05:29:54.049 INFO:teuthology.orchestra.run.smithi071.stdout:2037 write 0x7ae0b70 thru 0x7aedcd9 (0xd16a bytes) 2024-09-14T05:29:54.049 INFO:teuthology.orchestra.run.smithi071.stdout:2038 read 0x71840f1 thru 0x7191b21 (0xda31 bytes) 2024-09-14T05:29:54.049 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:54.049 INFO:teuthology.orchestra.run.smithi071.stdout:2040 write 0xa251c74 thru 0xa25d5a1 (0xb92e bytes) 2024-09-14T05:29:54.049 INFO:teuthology.orchestra.run.smithi071.stdout:2041 trunc from 0xa25d5a2 to 0xcd9bd17 2024-09-14T05:29:54.049 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:54.049 INFO:teuthology.orchestra.run.smithi071.stdout:2044 write 0x22a3850 thru 0x22a8835 (0x4fe6 bytes) 2024-09-14T05:29:54.049 INFO:teuthology.orchestra.run.smithi071.stdout:2046 punch from 0x7666397 to 0x766ba0f, (0x5678 bytes) 2024-09-14T05:29:54.049 INFO:teuthology.orchestra.run.smithi071.stdout:2047 write 0xc799198 thru 0xc79e4f4 (0x535d bytes) 2024-09-14T05:29:54.050 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:54.050 INFO:teuthology.orchestra.run.smithi071.stdout:2049 write 0xcc0e4c8 thru 0xcc148ed (0x6426 bytes) 2024-09-14T05:29:54.050 INFO:teuthology.orchestra.run.smithi071.stdout:2050 trunc from 0xcd9bd17 to 0x4a7988c 2024-09-14T05:29:54.050 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:54.050 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:54.045+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:54.051 INFO:teuthology.orchestra.run.smithi071.stdout:2053 writesame 0xaa68849 thru 0xaa78090 data_size 0x8c(0xf848 bytes) 2024-09-14T05:29:54.051 INFO:teuthology.orchestra.run.smithi071.stdout:2054 punch from 0x2e57426 to 0x2e64cef, (0xd8c9 bytes) 2024-09-14T05:29:54.051 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:54.052 INFO:teuthology.orchestra.run.smithi071.stdout:2056 read 0x81ad9cf thru 0x81bc044 (0xe676 bytes) 2024-09-14T05:29:54.052 INFO:teuthology.orchestra.run.smithi071.stdout:2057 write 0x9d29e72 thru 0x9d37eac (0xe03b bytes) 2024-09-14T05:29:54.054 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:54.054 INFO:teuthology.orchestra.run.smithi071.stdout:2059 read 0x714c5e7 thru 0x714ffc9 (0x39e3 bytes) 2024-09-14T05:29:54.054 INFO:teuthology.orchestra.run.smithi071.stdout:2060 read 0x634b12d thru 0x6357dab (0xcc7f bytes) 2024-09-14T05:29:54.055 INFO:teuthology.orchestra.run.smithi071.stdout:2061 write 0x5783734 thru 0x57928f1 (0xf1be bytes) 2024-09-14T05:29:54.059 INFO:teuthology.orchestra.run.smithi071.stdout:2065 trunc from 0xaa78091 to 0x9eb7054 2024-09-14T05:29:54.070 INFO:teuthology.orchestra.run.smithi071.stdout:2066 writesame 0x9ef1035 thru 0x9ef23d2 data_size 0x3(0x139e bytes) 2024-09-14T05:29:54.070 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:54.070 INFO:teuthology.orchestra.run.smithi071.stdout:2068 read 0x2c6c403 thru 0x2c71221 (0x4e1f bytes) 2024-09-14T05:29:54.071 INFO:teuthology.orchestra.run.smithi071.stdout:2069 punch from 0x4d9d76e to 0x4da2cb0, (0x5542 bytes) 2024-09-14T05:29:54.081 INFO:teuthology.orchestra.run.smithi071.stdout:2070 writesame 0xe8a1104 thru 0xe8a7004 data_size 0x43(0x5f01 bytes) 2024-09-14T05:29:54.081 INFO:teuthology.orchestra.run.smithi071.stdout:2071 punch from 0x4312946 to 0x431c5cc, (0x9c86 bytes) 2024-09-14T05:29:54.093 INFO:teuthology.orchestra.run.smithi071.stdout:2072 trunc from 0xe8a7005 to 0x5a5fd2a 2024-09-14T05:29:54.102 INFO:teuthology.orchestra.run.smithi071.stdout:2074 write 0x14a59dd thru 0x14ab1e5 (0x5809 bytes) 2024-09-14T05:29:54.106 INFO:teuthology.orchestra.run.smithi071.stdout:2076 clone 19 order 24 su 65536 sc 2 2024-09-14T05:29:54.146 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:54.146+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:54.167 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:54.167 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:54.167 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:29:54.167 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:54.178 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:29:54.247 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:54.247+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:54.248 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone18 from 0x5a5fd2a (overlap 0x450b05) to 0x6e780 2024-09-14T05:29:54.335 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:54.336 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:54.336 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:54.336 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:54.336 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:54.343 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:29:54.348 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:54.348+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:54.371 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-14T05:29:54.377 INFO:teuthology.orchestra.run.smithi071.stdout:2077 read 0x526bc2e thru 0x5270297 (0x466a bytes) 2024-09-14T05:29:54.377 INFO:teuthology.orchestra.run.smithi071.stdout:2080 read 0x3079c77 thru 0x307bee2 (0x226c bytes) 2024-09-14T05:29:54.378 INFO:teuthology.orchestra.run.smithi071.stdout:2081 writesame 0x3f51d9e thru 0x3f5df29 data_size 0x6(0xc18c bytes) 2024-09-14T05:29:54.389 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:54.396 INFO:teuthology.orchestra.run.smithi071.stdout:2084 write 0x7486331 thru 0x7494f6b (0xec3b bytes) 2024-09-14T05:29:54.400 INFO:teuthology.orchestra.run.smithi071.stdout:2085 punch from 0x101a2d to 0x10753b, (0x5b0e bytes) 2024-09-14T05:29:54.400 INFO:teuthology.orchestra.run.smithi071.stdout:2086 write 0x5f36ae3 thru 0x5f3a19f (0x36bd bytes) 2024-09-14T05:29:54.405 INFO:teuthology.orchestra.run.smithi071.stdout:2087 writesame 0x130101c thru 0x130b270 data_size 0x1(0xa255 bytes) 2024-09-14T05:29:54.432 INFO:teuthology.orchestra.run.smithi071.stdout:2088 read 0x4d66d71 thru 0x4d7090a (0x9b9a bytes) 2024-09-14T05:29:54.433 INFO:teuthology.orchestra.run.smithi071.stdout:2091 punch from 0x567ece7 to 0x567fc3e, (0xf57 bytes) 2024-09-14T05:29:54.433 INFO:teuthology.orchestra.run.smithi071.stdout:2092 write 0x1f87b62 thru 0x1f967fe (0xec9d bytes) 2024-09-14T05:29:54.440 INFO:teuthology.orchestra.run.smithi071.stdout:2093 punch from 0x3c3b26f to 0x3c49109, (0xde9a bytes) 2024-09-14T05:29:54.440 INFO:teuthology.orchestra.run.smithi071.stdout:2094 read 0x3326da8 thru 0x33315bd (0xa816 bytes) 2024-09-14T05:29:54.449 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:54.448+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:54.453 INFO:teuthology.orchestra.run.smithi071.stdout:2095 writesame 0xddd78ea thru 0xdddcb2d data_size 0x1e(0x5244 bytes) 2024-09-14T05:29:54.453 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:54.453 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:54.453 INFO:teuthology.orchestra.run.smithi071.stdout:2099 write 0x2c58dd3 thru 0x2c58e57 (0x85 bytes) 2024-09-14T05:29:54.454 INFO:teuthology.orchestra.run.smithi071.stdout:2101 writesame 0x4a5612f thru 0x4a58d02 data_size 0x44(0x2bd4 bytes) 2024-09-14T05:29:54.456 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:54.456 INFO:teuthology.orchestra.run.smithi071.stdout:2103 writesame 0xccf28aa thru 0xccfd3a8 data_size 0x55(0xaaff bytes) 2024-09-14T05:29:54.458 INFO:teuthology.orchestra.run.smithi071.stdout:2104 write 0xc4c46f3 thru 0xc4c4fb7 (0x8c5 bytes) 2024-09-14T05:29:54.461 INFO:teuthology.orchestra.run.smithi071.stdout:2105 read 0x545c3f thru 0x55132a (0xb6ec bytes) 2024-09-14T05:29:54.462 INFO:teuthology.orchestra.run.smithi071.stdout:2106 write 0xcc9f25 thru 0xccc0ef (0x21cb bytes) 2024-09-14T05:29:54.463 INFO:teuthology.orchestra.run.smithi071.stdout:2108 writesame 0xd01ecdc thru 0xd029539 data_size 0x17(0xa85e bytes) 2024-09-14T05:29:54.465 INFO:teuthology.orchestra.run.smithi071.stdout:2109 clone 20 order 22 su 32768 sc 12 2024-09-14T05:29:54.483 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:54.483 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:54.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:54.483 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:54.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.483 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4235.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-14T05:29:54.483 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:29:54.483 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.642058523999999, 2024-09-14T05:29:54.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12887758399999999, 2024-09-14T05:29:54.483 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.484 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.484 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:29:54.484 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.484 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4235", 2024-09-14T05:29:54.484 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/3211559243", 2024-09-14T05:29:54.484 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 2 2024-09-14T05:29:54.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.484 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:29:54.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:29:54.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839833+0000", 2024-09-14T05:29:54.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:29:54.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839838+0000", 2024-09-14T05:29:54.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2440000000000001e-06 2024-09-14T05:29:54.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.486 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.486 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839848+0000", 2024-09-14T05:29:54.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0030000000000001e-05 2024-09-14T05:29:54.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.486 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.486 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839854+0000", 2024-09-14T05:29:54.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3750000000000002e-06 2024-09-14T05:29:54.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.486 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.486 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839908+0000", 2024-09-14T05:29:54.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3965999999999998e-05 2024-09-14T05:29:54.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:29:54.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839914+0000", 2024-09-14T05:29:54.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6139999999999997e-06 2024-09-14T05:29:54.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968558+0000", 2024-09-14T05:29:54.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12864361199999999 2024-09-14T05:29:54.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968710+0000", 2024-09-14T05:29:54.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015177300000000001 2024-09-14T05:29:54.488 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.488 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.488 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.488 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27764 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 188416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.488 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:29:54.488 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.270732142, 2024-09-14T05:29:54.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080826920999999996, 2024-09-14T05:29:54.489 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.489 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.489 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.489 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.489 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.489 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.489 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27764 2024-09-14T05:29:54.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.489 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:29:54.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:29:54.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211159+0000", 2024-09-14T05:29:54.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1800000000000001e-07 2024-09-14T05:29:54.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211161+0000", 2024-09-14T05:29:54.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.968e-06 2024-09-14T05:29:54.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211169+0000", 2024-09-14T05:29:54.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4800000000000001e-06 2024-09-14T05:29:54.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211174+0000", 2024-09-14T05:29:54.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1880000000000001e-06 2024-09-14T05:29:54.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291670+0000", 2024-09-14T05:29:54.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080495199000000003 2024-09-14T05:29:54.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.492 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.492 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291775+0000", 2024-09-14T05:29:54.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000105375 2024-09-14T05:29:54.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.492 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.492 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291985+0000", 2024-09-14T05:29:54.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021029299999999999 2024-09-14T05:29:54.492 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.492 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.493 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.493 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27765 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.493 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:29:54.493 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2707093920000005, 2024-09-14T05:29:54.493 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081115197999999999, 2024-09-14T05:29:54.493 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.493 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.493 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.493 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.493 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.493 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.493 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27765 2024-09-14T05:29:54.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.494 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:29:54.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:29:54.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211182+0000", 2024-09-14T05:29:54.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3599999999999999e-07 2024-09-14T05:29:54.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211184+0000", 2024-09-14T05:29:54.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.03e-06 2024-09-14T05:29:54.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211192+0000", 2024-09-14T05:29:54.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2309999999999996e-06 2024-09-14T05:29:54.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.496 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.496 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211197+0000", 2024-09-14T05:29:54.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4199999999999998e-06 2024-09-14T05:29:54.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.496 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.496 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292017+0000", 2024-09-14T05:29:54.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080819516999999994 2024-09-14T05:29:54.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.496 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.496 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292087+0000", 2024-09-14T05:29:54.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0056999999999997e-05 2024-09-14T05:29:54.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292296+0000", 2024-09-14T05:29:54.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000209507 2024-09-14T05:29:54.497 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.497 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.497 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.497 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27766 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.497 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:29:54.497 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2706861780000001, 2024-09-14T05:29:54.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081359077000000002, 2024-09-14T05:29:54.498 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.498 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.498 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.498 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.498 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.498 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.498 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27766 2024-09-14T05:29:54.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.498 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:29:54.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:29:54.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211205+0000", 2024-09-14T05:29:54.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7e-07 2024-09-14T05:29:54.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211207+0000", 2024-09-14T05:29:54.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4540000000000001e-06 2024-09-14T05:29:54.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211216+0000", 2024-09-14T05:29:54.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3680000000000002e-06 2024-09-14T05:29:54.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211221+0000", 2024-09-14T05:29:54.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3499999999999996e-06 2024-09-14T05:29:54.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292329+0000", 2024-09-14T05:29:54.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081108240999999998 2024-09-14T05:29:54.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.501 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.501 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292370+0000", 2024-09-14T05:29:54.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.121e-05 2024-09-14T05:29:54.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.501 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.501 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292563+0000", 2024-09-14T05:29:54.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019298399999999999 2024-09-14T05:29:54.501 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.501 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.501 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.502 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27767 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.502 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:29:54.502 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2706626639999996, 2024-09-14T05:29:54.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081623119999999993, 2024-09-14T05:29:54.502 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.502 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.502 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.502 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.502 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.502 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.502 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27767 2024-09-14T05:29:54.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.503 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:29:54.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:29:54.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:29:54.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0499999999999999e-07 2024-09-14T05:29:54.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211230+0000", 2024-09-14T05:29:54.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0700000000000001e-06 2024-09-14T05:29:54.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211236+0000", 2024-09-14T05:29:54.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8429999999999998e-06 2024-09-14T05:29:54.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211241+0000", 2024-09-14T05:29:54.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:29:54.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292594+0000", 2024-09-14T05:29:54.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081352868999999994 2024-09-14T05:29:54.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292631+0000", 2024-09-14T05:29:54.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7083999999999999e-05 2024-09-14T05:29:54.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.506 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.506 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292851+0000", 2024-09-14T05:29:54.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022011700000000001 2024-09-14T05:29:54.506 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.506 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.506 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.508 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27768 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 274432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.508 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:29:54.508 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2706441280000007, 2024-09-14T05:29:54.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081875032, 2024-09-14T05:29:54.508 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.508 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.508 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.508 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.508 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.509 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.509 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27768 2024-09-14T05:29:54.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.509 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:29:54.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:29:54.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211247+0000", 2024-09-14T05:29:54.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.46e-07 2024-09-14T05:29:54.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211248+0000", 2024-09-14T05:29:54.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7349999999999999e-06 2024-09-14T05:29:54.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211253+0000", 2024-09-14T05:29:54.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.899e-06 2024-09-14T05:29:54.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211258+0000", 2024-09-14T05:29:54.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2810000000000002e-06 2024-09-14T05:29:54.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292888+0000", 2024-09-14T05:29:54.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081629955000000004 2024-09-14T05:29:54.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292926+0000", 2024-09-14T05:29:54.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7947000000000001e-05 2024-09-14T05:29:54.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293121+0000", 2024-09-14T05:29:54.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019586899999999999 2024-09-14T05:29:54.513 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.513 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.513 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.513 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27769 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 294912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.513 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:29:54.513 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2706260389999997, 2024-09-14T05:29:54.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082181899000000003, 2024-09-14T05:29:54.514 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.514 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.514 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.514 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.514 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.514 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.514 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27769 2024-09-14T05:29:54.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.514 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:29:54.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.515 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.515 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:29:54.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.515 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.515 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211265+0000", 2024-09-14T05:29:54.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2199999999999999e-07 2024-09-14T05:29:54.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211267+0000", 2024-09-14T05:29:54.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2239999999999998e-06 2024-09-14T05:29:54.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211273+0000", 2024-09-14T05:29:54.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6540000000000004e-06 2024-09-14T05:29:54.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211277+0000", 2024-09-14T05:29:54.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6299999999999997e-06 2024-09-14T05:29:54.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293150+0000", 2024-09-14T05:29:54.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081872575000000003 2024-09-14T05:29:54.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293245+0000", 2024-09-14T05:29:54.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5327000000000004e-05 2024-09-14T05:29:54.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293446+0000", 2024-09-14T05:29:54.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000201067 2024-09-14T05:29:54.518 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.518 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.518 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.518 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27770 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.518 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:29:54.518 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2706058769999995, 2024-09-14T05:29:54.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082663707000000003, 2024-09-14T05:29:54.518 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.518 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.519 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.519 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.519 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.519 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.519 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27770 2024-09-14T05:29:54.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.519 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.519 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.519 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:29:54.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.519 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.520 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:29:54.520 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.520 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.520 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:29:54.520 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1199999999999998e-07 2024-09-14T05:29:54.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.520 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.520 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211292+0000", 2024-09-14T05:29:54.520 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1609999999999997e-06 2024-09-14T05:29:54.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.520 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211299+0000", 2024-09-14T05:29:54.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3300000000000004e-06 2024-09-14T05:29:54.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211303+0000", 2024-09-14T05:29:54.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7859999999999996e-06 2024-09-14T05:29:54.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293628+0000", 2024-09-14T05:29:54.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082324283999999998 2024-09-14T05:29:54.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.522 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293694+0000", 2024-09-14T05:29:54.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6277000000000003e-05 2024-09-14T05:29:54.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.522 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293948+0000", 2024-09-14T05:29:54.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025445699999999998 2024-09-14T05:29:54.522 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.522 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.522 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.522 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27790 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.523 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:29:54.523 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2705794729999997, 2024-09-14T05:29:54.523 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082461006000000003, 2024-09-14T05:29:54.523 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.523 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.523 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.523 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.523 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.523 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.523 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27790 2024-09-14T05:29:54.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.523 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.523 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:29:54.524 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.524 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.524 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:29:54.524 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.524 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.524 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211312+0000", 2024-09-14T05:29:54.524 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9299999999999998e-07 2024-09-14T05:29:54.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.524 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211314+0000", 2024-09-14T05:29:54.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000002e-06 2024-09-14T05:29:54.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211320+0000", 2024-09-14T05:29:54.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1589999999999998e-06 2024-09-14T05:29:54.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211325+0000", 2024-09-14T05:29:54.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9609999999999996e-06 2024-09-14T05:29:54.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293474+0000", 2024-09-14T05:29:54.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082149080999999999 2024-09-14T05:29:54.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293542+0000", 2024-09-14T05:29:54.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8497000000000005e-05 2024-09-14T05:29:54.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293772+0000", 2024-09-14T05:29:54.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022969900000000001 2024-09-14T05:29:54.526 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.526 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.527 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.527 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27791 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.527 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:29:54.527 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2705591589999994, 2024-09-14T05:29:54.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082700838999999998, 2024-09-14T05:29:54.527 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.527 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.527 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.527 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.527 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.527 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.528 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27791 2024-09-14T05:29:54.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.528 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:29:54.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:29:54.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.529 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.529 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211332+0000", 2024-09-14T05:29:54.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7399999999999999e-07 2024-09-14T05:29:54.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.529 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.529 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211334+0000", 2024-09-14T05:29:54.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8500000000000001e-06 2024-09-14T05:29:54.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.529 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.529 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211340+0000", 2024-09-14T05:29:54.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3010000000000003e-06 2024-09-14T05:29:54.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211345+0000", 2024-09-14T05:29:54.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.938e-06 2024-09-14T05:29:54.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293797+0000", 2024-09-14T05:29:54.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082452152000000001 2024-09-14T05:29:54.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293869+0000", 2024-09-14T05:29:54.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2236000000000004e-05 2024-09-14T05:29:54.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.531 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294032+0000", 2024-09-14T05:29:54.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016298800000000001 2024-09-14T05:29:54.531 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.531 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.531 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.531 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27792 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.532 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:29:54.532 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2705396279999999, 2024-09-14T05:29:54.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082897304000000005, 2024-09-14T05:29:54.532 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.532 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.532 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.532 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.532 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.532 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.532 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27792 2024-09-14T05:29:54.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.532 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:29:54.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:29:54.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:29:54.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2000000000000001e-07 2024-09-14T05:29:54.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.534 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211353+0000", 2024-09-14T05:29:54.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.761e-06 2024-09-14T05:29:54.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.534 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211359+0000", 2024-09-14T05:29:54.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2430000000000002e-06 2024-09-14T05:29:54.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.534 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211364+0000", 2024-09-14T05:29:54.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0239999999999997e-06 2024-09-14T05:29:54.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294059+0000", 2024-09-14T05:29:54.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082694834999999994 2024-09-14T05:29:54.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294114+0000", 2024-09-14T05:29:54.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4823e-05 2024-09-14T05:29:54.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294248+0000", 2024-09-14T05:29:54.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013429800000000001 2024-09-14T05:29:54.535 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.536 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.536 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.536 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27793 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.536 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:29:54.536 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2705198289999995, 2024-09-14T05:29:54.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083153362999999994, 2024-09-14T05:29:54.536 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.536 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.536 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.536 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.536 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.537 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.537 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27793 2024-09-14T05:29:54.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.537 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.537 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:29:54.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.537 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:29:54.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:29:54.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5100000000000001e-07 2024-09-14T05:29:54.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211373+0000", 2024-09-14T05:29:54.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.001e-06 2024-09-14T05:29:54.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211379+0000", 2024-09-14T05:29:54.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4559999999999997e-06 2024-09-14T05:29:54.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211384+0000", 2024-09-14T05:29:54.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9749999999999998e-06 2024-09-14T05:29:54.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294279+0000", 2024-09-14T05:29:54.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082894282999999999 2024-09-14T05:29:54.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294327+0000", 2024-09-14T05:29:54.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8208000000000003e-05 2024-09-14T05:29:54.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294524+0000", 2024-09-14T05:29:54.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019708900000000001 2024-09-14T05:29:54.540 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.540 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.540 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.540 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27794 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.540 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:29:54.541 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2704995050000001, 2024-09-14T05:29:54.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588590000000004, 2024-09-14T05:29:54.541 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.541 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.541 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.541 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.541 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.541 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.541 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27794 2024-09-14T05:29:54.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.541 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:29:54.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:29:54.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:29:54.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4499999999999997e-07 2024-09-14T05:29:54.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.543 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211393+0000", 2024-09-14T05:29:54.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.815e-06 2024-09-14T05:29:54.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.543 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.543 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211400+0000", 2024-09-14T05:29:54.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2899999999999999e-06 2024-09-14T05:29:54.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.543 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.543 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211404+0000", 2024-09-14T05:29:54.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:29:54.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294560+0000", 2024-09-14T05:29:54.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083155534000000003 2024-09-14T05:29:54.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294631+0000", 2024-09-14T05:29:54.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1378999999999995e-05 2024-09-14T05:29:54.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294980+0000", 2024-09-14T05:29:54.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00034844900000000002 2024-09-14T05:29:54.545 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.545 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.545 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.545 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27795 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.545 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:29:54.545 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2704801470000007, 2024-09-14T05:29:54.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083970367000000004, 2024-09-14T05:29:54.545 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.546 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.546 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.546 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.546 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.546 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.546 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27795 2024-09-14T05:29:54.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.546 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:29:54.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:29:54.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211411+0000", 2024-09-14T05:29:54.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5499999999999999e-07 2024-09-14T05:29:54.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211413+0000", 2024-09-14T05:29:54.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:29:54.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211419+0000", 2024-09-14T05:29:54.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1720000000000004e-06 2024-09-14T05:29:54.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.548 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.548 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211424+0000", 2024-09-14T05:29:54.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0690000000000001e-06 2024-09-14T05:29:54.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.548 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.548 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295013+0000", 2024-09-14T05:29:54.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588898999999994 2024-09-14T05:29:54.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295102+0000", 2024-09-14T05:29:54.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9462000000000005e-05 2024-09-14T05:29:54.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295381+0000", 2024-09-14T05:29:54.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000278566 2024-09-14T05:29:54.549 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.549 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.549 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.549 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27796 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.549 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:29:54.550 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2704604180000008, 2024-09-14T05:29:54.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084379107999999994, 2024-09-14T05:29:54.550 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.550 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.550 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.550 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.550 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.550 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.550 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27796 2024-09-14T05:29:54.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.550 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:29:54.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:29:54.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:29:54.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2800000000000002e-07 2024-09-14T05:29:54.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211432+0000", 2024-09-14T05:29:54.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:29:54.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211439+0000", 2024-09-14T05:29:54.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5239999999999997e-06 2024-09-14T05:29:54.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211444+0000", 2024-09-14T05:29:54.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.95e-06 2024-09-14T05:29:54.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295602+0000", 2024-09-14T05:29:54.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084158017000000002 2024-09-14T05:29:54.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295674+0000", 2024-09-14T05:29:54.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2385999999999994e-05 2024-09-14T05:29:54.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295809+0000", 2024-09-14T05:29:54.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013495900000000001 2024-09-14T05:29:54.554 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.554 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.554 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.554 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27797 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 266240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.554 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:29:54.554 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2704404359999995, 2024-09-14T05:29:54.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084898597000000006, 2024-09-14T05:29:54.554 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.554 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.555 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.555 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.555 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.555 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.555 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27797 2024-09-14T05:29:54.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.555 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:29:54.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:29:54.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:29:54.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:29:54.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211452+0000", 2024-09-14T05:29:54.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.708e-06 2024-09-14T05:29:54.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211458+0000", 2024-09-14T05:29:54.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3489999999999997e-06 2024-09-14T05:29:54.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211464+0000", 2024-09-14T05:29:54.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3820000000000003e-06 2024-09-14T05:29:54.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295839+0000", 2024-09-14T05:29:54.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084375055000000004 2024-09-14T05:29:54.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295918+0000", 2024-09-14T05:29:54.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9253999999999997e-05 2024-09-14T05:29:54.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296349+0000", 2024-09-14T05:29:54.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043051 2024-09-14T05:29:54.559 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.559 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.559 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.559 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27798 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.559 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:29:54.559 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2704207210000007, 2024-09-14T05:29:54.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085300275999999994, 2024-09-14T05:29:54.559 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.559 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.559 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.560 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.560 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.560 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.560 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27798 2024-09-14T05:29:54.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.560 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.560 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:29:54.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.560 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:29:54.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:29:54.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:29:54.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211472+0000", 2024-09-14T05:29:54.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7579999999999999e-06 2024-09-14T05:29:54.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211478+0000", 2024-09-14T05:29:54.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0310000000000004e-06 2024-09-14T05:29:54.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211483+0000", 2024-09-14T05:29:54.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8110000000000002e-06 2024-09-14T05:29:54.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296469+0000", 2024-09-14T05:29:54.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084986575999999994 2024-09-14T05:29:54.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296545+0000", 2024-09-14T05:29:54.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5749999999999998e-05 2024-09-14T05:29:54.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296770+0000", 2024-09-14T05:29:54.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022498900000000001 2024-09-14T05:29:54.563 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.564 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.564 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.564 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27799 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 307200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.564 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:29:54.564 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2704018080000008, 2024-09-14T05:29:54.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085718461999999995, 2024-09-14T05:29:54.564 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.564 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.564 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.564 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.565 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.565 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.565 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27799 2024-09-14T05:29:54.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.565 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.565 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:29:54.565 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.565 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:29:54.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.566 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:29:54.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8000000000000001e-07 2024-09-14T05:29:54.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.566 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211491+0000", 2024-09-14T05:29:54.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-14T05:29:54.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211497+0000", 2024-09-14T05:29:54.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.857e-06 2024-09-14T05:29:54.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211502+0000", 2024-09-14T05:29:54.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9790000000000001e-06 2024-09-14T05:29:54.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296800+0000", 2024-09-14T05:29:54.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085298099000000002 2024-09-14T05:29:54.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296871+0000", 2024-09-14T05:29:54.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0879999999999999e-05 2024-09-14T05:29:54.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297207+0000", 2024-09-14T05:29:54.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033651699999999999 2024-09-14T05:29:54.568 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.568 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.568 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.569 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27826 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.569 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:29:54.569 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2686866870000006, 2024-09-14T05:29:54.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084306729999999996, 2024-09-14T05:29:54.569 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.569 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.569 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.569 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.569 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.569 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.569 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27826 2024-09-14T05:29:54.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.570 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.570 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.570 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:29:54.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.570 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.570 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:29:54.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.570 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213205+0000", 2024-09-14T05:29:54.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.277e-06 2024-09-14T05:29:54.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213210+0000", 2024-09-14T05:29:54.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3120000000000001e-06 2024-09-14T05:29:54.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213226+0000", 2024-09-14T05:29:54.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5279000000000001e-05 2024-09-14T05:29:54.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213234+0000", 2024-09-14T05:29:54.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7020000000000007e-06 2024-09-14T05:29:54.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297239+0000", 2024-09-14T05:29:54.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084004876000000006 2024-09-14T05:29:54.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297311+0000", 2024-09-14T05:29:54.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1336999999999998e-05 2024-09-14T05:29:54.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297511+0000", 2024-09-14T05:29:54.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019994699999999999 2024-09-14T05:29:54.573 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.573 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.573 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.573 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27827 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:54.573 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:29:54.573 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 8.2686414999999993, 2024-09-14T05:29:54.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084520537000000007, 2024-09-14T05:29:54.573 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:54.574 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:54.574 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:54.574 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:54.574 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:54.574 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.574 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27827 2024-09-14T05:29:54.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.575 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:54.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.575 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:54.575 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:29:54.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.575 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:54.575 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:29:54.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:54.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.575 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:54.576 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213250+0000", 2024-09-14T05:29:54.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1099999999999995e-07 2024-09-14T05:29:54.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:54.576 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213252+0000", 2024-09-14T05:29:54.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6699999999999998e-06 2024-09-14T05:29:54.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:54.576 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213259+0000", 2024-09-14T05:29:54.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8469999999999998e-06 2024-09-14T05:29:54.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213265+0000", 2024-09-14T05:29:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4940000000000003e-06 2024-09-14T05:29:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297541+0000", 2024-09-14T05:29:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084276731999999993 2024-09-14T05:29:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297579+0000", 2024-09-14T05:29:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8108999999999997e-05 2024-09-14T05:29:54.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:54.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:54.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:54.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297770+0000", 2024-09-14T05:29:54.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019007400000000001 2024-09-14T05:29:54.578 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.578 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.578 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.578 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:54.578 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:54.578 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:54.580 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:29:54.581 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:54.579+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:54.681 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:54.680+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:54.683 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:54.684 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:54.684 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:29:54.684 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:54.692 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:29:54.782 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:54.781+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:54.795 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:54.795 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:54.795 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:29:54.795 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:29:54.795 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:54.803 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:29:54.883 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:54.882+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:54.911 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:54.911 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:29:54.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:29:54.911 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:29:54.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.911 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3523 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6062080~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:54.911 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:29:54.911 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.395072239000001, 2024-09-14T05:29:54.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096911030999999995, 2024-09-14T05:29:54.911 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:54.911 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:54.912 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:54.912 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:54.912 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:54.912 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.912 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3523 2024-09-14T05:29:54.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.912 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:54.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:54.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:29:54.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:54.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:29:54.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:54.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514359+0000", 2024-09-14T05:29:54.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9299999999999999e-07 2024-09-14T05:29:54.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:54.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514361+0000", 2024-09-14T05:29:54.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:29:54.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:54.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514366+0000", 2024-09-14T05:29:54.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1660000000000002e-06 2024-09-14T05:29:54.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514371+0000", 2024-09-14T05:29:54.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.648e-06 2024-09-14T05:29:54.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:54.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.610954+0000", 2024-09-14T05:29:54.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096582170999999994 2024-09-14T05:29:54.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:54.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611002+0000", 2024-09-14T05:29:54.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7923999999999997e-05 2024-09-14T05:29:54.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:54.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611269+0000", 2024-09-14T05:29:54.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026789499999999998 2024-09-14T05:29:54.919 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.919 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:54.920 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.920 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3524 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6553600~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:54.920 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:29:54.920 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.395053047000001, 2024-09-14T05:29:54.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097263347999999999, 2024-09-14T05:29:54.920 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:54.920 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:54.920 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:54.920 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:54.920 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:54.920 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.920 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3524 2024-09-14T05:29:54.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.921 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:54.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:54.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:29:54.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:54.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:29:54.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:54.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:29:54.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:29:54.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:54.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514380+0000", 2024-09-14T05:29:54.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6500000000000001e-06 2024-09-14T05:29:54.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:54.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514386+0000", 2024-09-14T05:29:54.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7360000000000004e-06 2024-09-14T05:29:54.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514391+0000", 2024-09-14T05:29:54.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2830000000000003e-06 2024-09-14T05:29:54.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:54.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611295+0000", 2024-09-14T05:29:54.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096904814000000006 2024-09-14T05:29:54.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:54.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611343+0000", 2024-09-14T05:29:54.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7506999999999998e-05 2024-09-14T05:29:54.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:54.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611641+0000", 2024-09-14T05:29:54.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029798800000000001 2024-09-14T05:29:54.924 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.925 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:54.925 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.925 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3525 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7045120~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:54.925 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:29:54.925 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.395031159999998, 2024-09-14T05:29:54.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097644422999999994, 2024-09-14T05:29:54.925 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:54.925 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:54.925 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:54.925 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:54.925 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:54.926 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.926 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3525 2024-09-14T05:29:54.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.926 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:54.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:54.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:29:54.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:54.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:29:54.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:54.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514400+0000", 2024-09-14T05:29:54.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.77e-07 2024-09-14T05:29:54.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:54.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514402+0000", 2024-09-14T05:29:54.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6530000000000001e-06 2024-09-14T05:29:54.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:54.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514411+0000", 2024-09-14T05:29:54.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7429999999999997e-06 2024-09-14T05:29:54.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514417+0000", 2024-09-14T05:29:54.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2249999999999997e-06 2024-09-14T05:29:54.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:54.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611668+0000", 2024-09-14T05:29:54.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097250681000000005 2024-09-14T05:29:54.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:54.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611716+0000", 2024-09-14T05:29:54.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7336000000000002e-05 2024-09-14T05:29:54.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:54.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612044+0000", 2024-09-14T05:29:54.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032840799999999998 2024-09-14T05:29:54.929 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.929 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:54.929 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.930 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3526 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7536640~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:54.930 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:29:54.930 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.395006676000001, 2024-09-14T05:29:54.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097943884999999994, 2024-09-14T05:29:54.930 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:54.930 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:54.930 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:54.930 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:54.930 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:54.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.931 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3526 2024-09-14T05:29:54.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.931 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:54.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:54.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:29:54.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:54.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:29:54.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:54.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:29:54.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0400000000000002e-07 2024-09-14T05:29:54.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:54.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514426+0000", 2024-09-14T05:29:54.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:29:54.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:54.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514432+0000", 2024-09-14T05:29:54.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.105e-06 2024-09-14T05:29:54.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514438+0000", 2024-09-14T05:29:54.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5149999999999997e-06 2024-09-14T05:29:54.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:54.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612072+0000", 2024-09-14T05:29:54.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097633947999999998 2024-09-14T05:29:54.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:54.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612120+0000", 2024-09-14T05:29:54.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8458999999999998e-05 2024-09-14T05:29:54.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:54.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612368+0000", 2024-09-14T05:29:54.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024761999999999999 2024-09-14T05:29:54.934 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.934 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:54.935 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.935 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3527 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 8028160~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:54.935 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:29:54.935 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.394985963, 2024-09-14T05:29:54.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098244955999999994, 2024-09-14T05:29:54.935 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:54.935 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:54.935 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:54.935 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:54.935 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:54.935 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.935 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3527 2024-09-14T05:29:54.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.936 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:54.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:54.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:29:54.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:54.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:29:54.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:54.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519842+0000", 2024-09-14T05:29:54.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053975230000000004 2024-09-14T05:29:54.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:54.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519851+0000", 2024-09-14T05:29:54.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5599999999999994e-06 2024-09-14T05:29:54.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:54.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519858+0000", 2024-09-14T05:29:54.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2409999999999998e-06 2024-09-14T05:29:54.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519863+0000", 2024-09-14T05:29:54.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:29:54.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:54.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612395+0000", 2024-09-14T05:29:54.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092532552000000004 2024-09-14T05:29:54.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:54.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612442+0000", 2024-09-14T05:29:54.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6486000000000002e-05 2024-09-14T05:29:54.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:54.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612690+0000", 2024-09-14T05:29:54.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024791599999999999 2024-09-14T05:29:54.939 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.939 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:54.939 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.939 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3603 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5177344~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:54.939 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:29:54.940 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.389433518000001, 2024-09-14T05:29:54.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096796436, 2024-09-14T05:29:54.940 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:54.940 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:54.940 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:54.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:54.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:54.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.940 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3603 2024-09-14T05:29:54.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.940 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:54.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:54.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:29:54.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:54.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:29:54.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:54.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:29:54.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1899999999999999e-07 2024-09-14T05:29:54.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:54.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519998+0000", 2024-09-14T05:29:54.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5000000000000001e-07 2024-09-14T05:29:54.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:54.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520002+0000", 2024-09-14T05:29:54.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7299999999999999e-06 2024-09-14T05:29:54.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520004+0000", 2024-09-14T05:29:54.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.429e-06 2024-09-14T05:29:54.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:54.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616493+0000", 2024-09-14T05:29:54.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096488345000000003 2024-09-14T05:29:54.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:54.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616544+0000", 2024-09-14T05:29:54.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1724000000000001e-05 2024-09-14T05:29:54.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:54.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616794+0000", 2024-09-14T05:29:54.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000249139 2024-09-14T05:29:54.944 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.944 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:54.944 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.944 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3604 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5668864~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:54.944 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:29:54.944 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.389418880000001, 2024-09-14T05:29:54.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108222999999994, 2024-09-14T05:29:54.944 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:54.944 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:54.944 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:54.944 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:54.944 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:54.945 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.945 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3604 2024-09-14T05:29:54.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.945 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:54.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:54.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:29:54.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:54.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:29:54.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:54.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:29:54.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7000000000000001e-07 2024-09-14T05:29:54.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:54.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520013+0000", 2024-09-14T05:29:54.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.389e-06 2024-09-14T05:29:54.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:54.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520018+0000", 2024-09-14T05:29:54.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3019999999999996e-06 2024-09-14T05:29:54.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520026+0000", 2024-09-14T05:29:54.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6170000000000007e-06 2024-09-14T05:29:54.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:54.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616816+0000", 2024-09-14T05:29:54.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096789766999999999 2024-09-14T05:29:54.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:54.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616871+0000", 2024-09-14T05:29:54.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4811000000000001e-05 2024-09-14T05:29:54.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:54.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617120+0000", 2024-09-14T05:29:54.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024906700000000001 2024-09-14T05:29:54.948 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.948 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:54.948 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.948 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3605 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6160384~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:54.949 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:29:54.949 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.389400420000001, 2024-09-14T05:29:54.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097423945999999997, 2024-09-14T05:29:54.949 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:54.949 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:54.949 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:54.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:54.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:54.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.949 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3605 2024-09-14T05:29:54.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.949 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:54.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:54.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:29:54.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:54.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:29:54.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:54.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:29:54.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3999999999999998e-07 2024-09-14T05:29:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520032+0000", 2024-09-14T05:29:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.083e-06 2024-09-14T05:29:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520036+0000", 2024-09-14T05:29:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.019e-06 2024-09-14T05:29:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520038+0000", 2024-09-14T05:29:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4849999999999999e-06 2024-09-14T05:29:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617147+0000", 2024-09-14T05:29:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108665999999996 2024-09-14T05:29:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617199+0000", 2024-09-14T05:29:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2558000000000001e-05 2024-09-14T05:29:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617454+0000", 2024-09-14T05:29:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025489499999999999 2024-09-14T05:29:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3606 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6651904~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:29:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.389389697999999, 2024-09-14T05:29:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097745705000000002, 2024-09-14T05:29:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3606 2024-09-14T05:29:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:29:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:29:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:29:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-14T05:29:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520042+0000", 2024-09-14T05:29:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4799999999999997e-07 2024-09-14T05:29:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520046+0000", 2024-09-14T05:29:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8149999999999999e-06 2024-09-14T05:29:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520049+0000", 2024-09-14T05:29:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.576e-06 2024-09-14T05:29:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617476+0000", 2024-09-14T05:29:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097427558999999997 2024-09-14T05:29:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617522+0000", 2024-09-14T05:29:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5927999999999998e-05 2024-09-14T05:29:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617787+0000", 2024-09-14T05:29:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000264663 2024-09-14T05:29:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3607 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7143424~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:29:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.389379171000002, 2024-09-14T05:29:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097871125000000003, 2024-09-14T05:29:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3607 2024-09-14T05:29:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:29:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:29:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520052+0000", 2024-09-14T05:29:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.03e-07 2024-09-14T05:29:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520053+0000", 2024-09-14T05:29:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.85e-07 2024-09-14T05:29:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520056+0000", 2024-09-14T05:29:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9140000000000001e-06 2024-09-14T05:29:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520059+0000", 2024-09-14T05:29:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4600000000000002e-06 2024-09-14T05:29:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617809+0000", 2024-09-14T05:29:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097750216000000001 2024-09-14T05:29:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617864+0000", 2024-09-14T05:29:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5211999999999999e-05 2024-09-14T05:29:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617923+0000", 2024-09-14T05:29:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8235000000000001e-05 2024-09-14T05:29:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3608 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7634944~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:29:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.389368863000001, 2024-09-14T05:29:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098199212999999994, 2024-09-14T05:29:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3608 2024-09-14T05:29:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:29:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:29:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525953+0000", 2024-09-14T05:29:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058911850000000002 2024-09-14T05:29:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525957+0000", 2024-09-14T05:29:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2379999999999999e-06 2024-09-14T05:29:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525967+0000", 2024-09-14T05:29:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5389999999999996e-06 2024-09-14T05:29:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525972+0000", 2024-09-14T05:29:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.541e-06 2024-09-14T05:29:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617945+0000", 2024-09-14T05:29:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091972597000000003 2024-09-14T05:29:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617992+0000", 2024-09-14T05:29:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7397e-05 2024-09-14T05:29:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.618261+0000", 2024-09-14T05:29:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026871600000000001 2024-09-14T05:29:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3640 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6586368~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:29:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.383261020999999, 2024-09-14T05:29:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097068580000000002, 2024-09-14T05:29:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3640 2024-09-14T05:29:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:29:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:29:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:29:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.23e-07 2024-09-14T05:29:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526171+0000", 2024-09-14T05:29:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:29:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526175+0000", 2024-09-14T05:29:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8369999999999999e-06 2024-09-14T05:29:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526177+0000", 2024-09-14T05:29:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4959999999999999e-06 2024-09-14T05:29:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622847+0000", 2024-09-14T05:29:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096669790000000005 2024-09-14T05:29:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622902+0000", 2024-09-14T05:29:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5157000000000001e-05 2024-09-14T05:29:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623238+0000", 2024-09-14T05:29:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033603399999999997 2024-09-14T05:29:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3641 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7077888~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:29:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.383250511, 2024-09-14T05:29:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097371141999999994, 2024-09-14T05:29:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3641 2024-09-14T05:29:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:29:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:29:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531851+0000", 2024-09-14T05:29:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005671265 2024-09-14T05:29:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531855+0000", 2024-09-14T05:29:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0849999999999999e-06 2024-09-14T05:29:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531867+0000", 2024-09-14T05:29:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1028e-05 2024-09-14T05:29:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531872+0000", 2024-09-14T05:29:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000003e-06 2024-09-14T05:29:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623251+0000", 2024-09-14T05:29:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091378980999999998 2024-09-14T05:29:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623295+0000", 2024-09-14T05:29:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3881999999999998e-05 2024-09-14T05:29:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623551+0000", 2024-09-14T05:29:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025636099999999998 2024-09-14T05:29:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3721 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:29:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.377392235999999, 2024-09-14T05:29:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096806472000000005, 2024-09-14T05:29:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3721 2024-09-14T05:29:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:29:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:29:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532039+0000", 2024-09-14T05:29:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.05e-07 2024-09-14T05:29:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532040+0000", 2024-09-14T05:29:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9099999999999991e-07 2024-09-14T05:29:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532043+0000", 2024-09-14T05:29:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6610000000000002e-06 2024-09-14T05:29:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532046+0000", 2024-09-14T05:29:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4080000000000001e-06 2024-09-14T05:29:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628545+0000", 2024-09-14T05:29:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096499615999999996 2024-09-14T05:29:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628592+0000", 2024-09-14T05:29:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6907000000000003e-05 2024-09-14T05:29:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628845+0000", 2024-09-14T05:29:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025268399999999998 2024-09-14T05:29:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3722 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6193152~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:29:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.377382155999999, 2024-09-14T05:29:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097146587000000006, 2024-09-14T05:29:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3722 2024-09-14T05:29:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:29:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:29:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538191+0000", 2024-09-14T05:29:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061429969999999999 2024-09-14T05:29:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538200+0000", 2024-09-14T05:29:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7069999999999998e-06 2024-09-14T05:29:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538212+0000", 2024-09-14T05:29:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1504e-05 2024-09-14T05:29:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538219+0000", 2024-09-14T05:29:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7500000000000003e-06 2024-09-14T05:29:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628869+0000", 2024-09-14T05:29:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090649224 2024-09-14T05:29:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628919+0000", 2024-09-14T05:29:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0144e-05 2024-09-14T05:29:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.629195+0000", 2024-09-14T05:29:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027626099999999998 2024-09-14T05:29:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3758 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7110656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:29:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.37078794, 2024-09-14T05:29:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096977544999999998, 2024-09-14T05:29:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3758 2024-09-14T05:29:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:29:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:29:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:29:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:29:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538645+0000", 2024-09-14T05:29:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.843e-06 2024-09-14T05:29:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538651+0000", 2024-09-14T05:29:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6169999999999998e-06 2024-09-14T05:29:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538655+0000", 2024-09-14T05:29:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5399999999999997e-06 2024-09-14T05:29:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635225+0000", 2024-09-14T05:29:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096570149999999993 2024-09-14T05:29:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635272+0000", 2024-09-14T05:29:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7199e-05 2024-09-14T05:29:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635620+0000", 2024-09-14T05:29:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034782600000000003 2024-09-14T05:29:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3759 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:29:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.370770028999999, 2024-09-14T05:29:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097335331999999997, 2024-09-14T05:29:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3759 2024-09-14T05:29:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:29:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:29:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544013+0000", 2024-09-14T05:29:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053520829999999997 2024-09-14T05:29:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544016+0000", 2024-09-14T05:29:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6660000000000001e-06 2024-09-14T05:29:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544026+0000", 2024-09-14T05:29:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7089999999999997e-06 2024-09-14T05:29:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544043+0000", 2024-09-14T05:29:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833999999999998e-05 2024-09-14T05:29:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635646+0000", 2024-09-14T05:29:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091603544999999995 2024-09-14T05:29:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635696+0000", 2024-09-14T05:29:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9307000000000001e-05 2024-09-14T05:29:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635996+0000", 2024-09-14T05:29:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030018800000000001 2024-09-14T05:29:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3838 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6225920~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:29:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.365108349, 2024-09-14T05:29:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096894235999999995, 2024-09-14T05:29:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3838 2024-09-14T05:29:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:29:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:29:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544323+0000", 2024-09-14T05:29:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-14T05:29:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544325+0000", 2024-09-14T05:29:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.75e-06 2024-09-14T05:29:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544331+0000", 2024-09-14T05:29:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5019999999999998e-06 2024-09-14T05:29:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544335+0000", 2024-09-14T05:29:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2989999999999998e-06 2024-09-14T05:29:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640869+0000", 2024-09-14T05:29:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096533351000000003 2024-09-14T05:29:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640919+0000", 2024-09-14T05:29:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999999e-05 2024-09-14T05:29:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641217+0000", 2024-09-14T05:29:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029717600000000002 2024-09-14T05:29:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3839 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:29:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.365089411, 2024-09-14T05:29:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097240339999999995, 2024-09-14T05:29:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3839 2024-09-14T05:29:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:29:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:29:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549834+0000", 2024-09-14T05:29:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054927819999999999 2024-09-14T05:29:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549838+0000", 2024-09-14T05:29:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8689999999999997e-06 2024-09-14T05:29:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549848+0000", 2024-09-14T05:29:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0154e-05 2024-09-14T05:29:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549853+0000", 2024-09-14T05:29:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9740000000000001e-06 2024-09-14T05:29:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641245+0000", 2024-09-14T05:29:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091392245999999996 2024-09-14T05:29:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641295+0000", 2024-09-14T05:29:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0018999999999997e-05 2024-09-14T05:29:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641582+0000", 2024-09-14T05:29:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028629600000000001 2024-09-14T05:29:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3868 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:29:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.359426833000001, 2024-09-14T05:29:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096859555999999999, 2024-09-14T05:29:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3868 2024-09-14T05:29:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:29:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:29:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556173+0000", 2024-09-14T05:29:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006169038 2024-09-14T05:29:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556177+0000", 2024-09-14T05:29:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0010000000000003e-06 2024-09-14T05:29:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556188+0000", 2024-09-14T05:29:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0675e-05 2024-09-14T05:29:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556199+0000", 2024-09-14T05:29:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.155e-05 2024-09-14T05:29:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646511+0000", 2024-09-14T05:29:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090312187000000002 2024-09-14T05:29:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646563+0000", 2024-09-14T05:29:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2219000000000003e-05 2024-09-14T05:29:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646863+0000", 2024-09-14T05:29:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029988600000000002 2024-09-14T05:29:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.011 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:55.013 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:29:55.013 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:55.011+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:55.113 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:55.113+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:55.156 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:29:55.157 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:55.165 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:29:55.178 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T05:29:55.223 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:55.222+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:55.268 INFO:teuthology.orchestra.run.smithi071.stdout:flattening image image_client.0-clone19 2024-09-14T05:29:55.329 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:55.329 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:55.329 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:29:55.329 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:29:55.329 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:55.330 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:29:55.330 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:55.328+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:55.378 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:29:55.429 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:55.429+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:55.438 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T05:29:55.438 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T05:29:55.438 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T05:29:55.438 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T05:29:55.466 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-14T05:29:55.473 INFO:teuthology.orchestra.run.smithi071.stdout:2113 writesame 0x725ee74 thru 0x72688c4 data_size 0x1(0x9a51 bytes) 2024-09-14T05:29:55.479 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:29:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:29:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:29:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1989 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 16326080~54496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:29:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.915991484999999, 2024-09-14T05:29:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096664322999999996, 2024-09-14T05:29:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1989 2024-09-14T05:29:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:29:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:29:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561455+0000", 2024-09-14T05:29:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1699999999999997e-07 2024-09-14T05:29:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590703+0000", 2024-09-14T05:29:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029248225999999999 2024-09-14T05:29:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590732+0000", 2024-09-14T05:29:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8679000000000001e-05 2024-09-14T05:29:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590759+0000", 2024-09-14T05:29:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7319999999999999e-05 2024-09-14T05:29:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590835+0000", 2024-09-14T05:29:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5543999999999994e-05 2024-09-14T05:29:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590956+0000", 2024-09-14T05:29:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012168 2024-09-14T05:29:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 1", 2024-09-14T05:29:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.591261+0000", 2024-09-14T05:29:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030484799999999998 2024-09-14T05:29:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.637682+0000", 2024-09-14T05:29:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046420550999999997 2024-09-14T05:29:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.657912+0000", 2024-09-14T05:29:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020230011999999999 2024-09-14T05:29:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658007+0000", 2024-09-14T05:29:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5814000000000001e-05 2024-09-14T05:29:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658118+0000", 2024-09-14T05:29:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110832 2024-09-14T05:29:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.486 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2033 3.1e 3:7e47f6eb:::rbd_data.112d1f36d097.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30853661~63531] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:55.486 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:29:55.486 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.804716818999999, 2024-09-14T05:29:55.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065167630000000004, 2024-09-14T05:29:55.486 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.487 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.487 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:55.487 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.487 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.487 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.487 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2033 2024-09-14T05:29:55.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.488 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:29:55.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:29:55.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672730+0000", 2024-09-14T05:29:55.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.393e-06 2024-09-14T05:29:55.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691289+0000", 2024-09-14T05:29:55.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018558991 2024-09-14T05:29:55.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691316+0000", 2024-09-14T05:29:55.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7498999999999999e-05 2024-09-14T05:29:55.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691331+0000", 2024-09-14T05:29:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4652e-05 2024-09-14T05:29:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691397+0000", 2024-09-14T05:29:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6112000000000007e-05 2024-09-14T05:29:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691509+0000", 2024-09-14T05:29:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111599 2024-09-14T05:29:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.692253+0000", 2024-09-14T05:29:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074456200000000004 2024-09-14T05:29:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.719140+0000", 2024-09-14T05:29:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026886542999999999 2024-09-14T05:29:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737789+0000", 2024-09-14T05:29:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018649203999999999 2024-09-14T05:29:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737846+0000", 2024-09-14T05:29:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6501e-05 2024-09-14T05:29:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737896+0000", 2024-09-14T05:29:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0574000000000001e-05 2024-09-14T05:29:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3008 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13729792~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:29:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.173792948999999, 2024-09-14T05:29:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061007651000000003, 2024-09-14T05:29:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3008 2024-09-14T05:29:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:29:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:29:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303653+0000", 2024-09-14T05:29:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0699999999999999e-07 2024-09-14T05:29:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303654+0000", 2024-09-14T05:29:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-07 2024-09-14T05:29:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303657+0000", 2024-09-14T05:29:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2890000000000002e-06 2024-09-14T05:29:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303661+0000", 2024-09-14T05:29:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3290000000000004e-06 2024-09-14T05:29:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364233+0000", 2024-09-14T05:29:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060572134 2024-09-14T05:29:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364315+0000", 2024-09-14T05:29:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1549000000000001e-05 2024-09-14T05:29:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364660+0000", 2024-09-14T05:29:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034525300000000001 2024-09-14T05:29:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3009 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14221312~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:29:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.173778858999999, 2024-09-14T05:29:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061445063000000001, 2024-09-14T05:29:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3009 2024-09-14T05:29:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:29:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:29:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:29:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:29:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303669+0000", 2024-09-14T05:29:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7150000000000001e-06 2024-09-14T05:29:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303675+0000", 2024-09-14T05:29:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5339999999999996e-06 2024-09-14T05:29:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303681+0000", 2024-09-14T05:29:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3639999999999998e-06 2024-09-14T05:29:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364705+0000", 2024-09-14T05:29:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061024123 2024-09-14T05:29:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364786+0000", 2024-09-14T05:29:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1378999999999995e-05 2024-09-14T05:29:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365112+0000", 2024-09-14T05:29:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032558700000000001 2024-09-14T05:29:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3010 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14712832~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:29:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.173757994999999, 2024-09-14T05:29:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061834630000000002, 2024-09-14T05:29:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3010 2024-09-14T05:29:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:29:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:29:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303688+0000", 2024-09-14T05:29:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:29:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303690+0000", 2024-09-14T05:29:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9460000000000001e-06 2024-09-14T05:29:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303696+0000", 2024-09-14T05:29:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2230000000000004e-06 2024-09-14T05:29:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303700+0000", 2024-09-14T05:29:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7290000000000002e-06 2024-09-14T05:29:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365163+0000", 2024-09-14T05:29:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061463093000000003 2024-09-14T05:29:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365214+0000", 2024-09-14T05:29:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0977999999999999e-05 2024-09-14T05:29:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365522+0000", 2024-09-14T05:29:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000308227 2024-09-14T05:29:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3241 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:29:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.148083573000001, 2024-09-14T05:29:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061041892, 2024-09-14T05:29:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3241 2024-09-14T05:29:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:29:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:29:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329496+0000", 2024-09-14T05:29:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013421799999999999 2024-09-14T05:29:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329499+0000", 2024-09-14T05:29:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8459999999999999e-06 2024-09-14T05:29:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329505+0000", 2024-09-14T05:29:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4320000000000004e-06 2024-09-14T05:29:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329509+0000", 2024-09-14T05:29:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0339999999999998e-06 2024-09-14T05:29:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.389947+0000", 2024-09-14T05:29:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060438093999999998 2024-09-14T05:29:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390038+0000", 2024-09-14T05:29:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0253999999999993e-05 2024-09-14T05:29:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390404+0000", 2024-09-14T05:29:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036601400000000001 2024-09-14T05:29:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3242 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:29:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.147928664999998, 2024-09-14T05:29:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061282426000000001, 2024-09-14T05:29:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3242 2024-09-14T05:29:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:29:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:29:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:29:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9900000000000001e-07 2024-09-14T05:29:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329518+0000", 2024-09-14T05:29:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.328e-06 2024-09-14T05:29:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329522+0000", 2024-09-14T05:29:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.563e-06 2024-09-14T05:29:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329525+0000", 2024-09-14T05:29:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7379999999999999e-06 2024-09-14T05:29:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390437+0000", 2024-09-14T05:29:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060912463 2024-09-14T05:29:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390498+0000", 2024-09-14T05:29:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0615999999999997e-05 2024-09-14T05:29:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390799+0000", 2024-09-14T05:29:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030131899999999998 2024-09-14T05:29:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3243 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:29:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.147916879, 2024-09-14T05:29:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061728980000000003, 2024-09-14T05:29:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3243 2024-09-14T05:29:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:29:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:29:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329692+0000", 2024-09-14T05:29:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000163948 2024-09-14T05:29:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329696+0000", 2024-09-14T05:29:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5540000000000002e-06 2024-09-14T05:29:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329706+0000", 2024-09-14T05:29:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7960000000000007e-06 2024-09-14T05:29:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329712+0000", 2024-09-14T05:29:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6109999999999999e-06 2024-09-14T05:29:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390861+0000", 2024-09-14T05:29:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061148767 2024-09-14T05:29:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390942+0000", 2024-09-14T05:29:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1266000000000004e-05 2024-09-14T05:29:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391258+0000", 2024-09-14T05:29:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031503800000000002 2024-09-14T05:29:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3244 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14778368~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:29:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.147721830999998, 2024-09-14T05:29:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062050594000000001, 2024-09-14T05:29:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3244 2024-09-14T05:29:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:29:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:29:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329916+0000", 2024-09-14T05:29:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019290799999999999 2024-09-14T05:29:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329919+0000", 2024-09-14T05:29:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9019999999999999e-06 2024-09-14T05:29:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329926+0000", 2024-09-14T05:29:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9e-06 2024-09-14T05:29:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329932+0000", 2024-09-14T05:29:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.088e-06 2024-09-14T05:29:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391299+0000", 2024-09-14T05:29:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061366136000000002 2024-09-14T05:29:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391391+0000", 2024-09-14T05:29:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2204e-05 2024-09-14T05:29:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391774+0000", 2024-09-14T05:29:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038345600000000001 2024-09-14T05:29:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3921 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 5308416~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:29:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.076178021, 2024-09-14T05:29:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060842694000000003, 2024-09-14T05:29:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3921 2024-09-14T05:29:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:29:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:29:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401705+0000", 2024-09-14T05:29:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768499999999999 2024-09-14T05:29:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401709+0000", 2024-09-14T05:29:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0319999999999997e-06 2024-09-14T05:29:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401720+0000", 2024-09-14T05:29:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1005000000000001e-05 2024-09-14T05:29:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401727+0000", 2024-09-14T05:29:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1879999999999996e-06 2024-09-14T05:29:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461748+0000", 2024-09-14T05:29:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060021199999999997 2024-09-14T05:29:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461793+0000", 2024-09-14T05:29:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4530999999999998e-05 2024-09-14T05:29:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462110+0000", 2024-09-14T05:29:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031705299999999997 2024-09-14T05:29:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3923 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 6291456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:29:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.075531992999998, 2024-09-14T05:29:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060891273000000003, 2024-09-14T05:29:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3923 2024-09-14T05:29:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:29:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:29:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407054+0000", 2024-09-14T05:29:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005140787 2024-09-14T05:29:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407059+0000", 2024-09-14T05:29:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.04e-06 2024-09-14T05:29:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407069+0000", 2024-09-14T05:29:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4429999999999992e-06 2024-09-14T05:29:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407076+0000", 2024-09-14T05:29:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1679999999999998e-06 2024-09-14T05:29:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462514+0000", 2024-09-14T05:29:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055438551000000003 2024-09-14T05:29:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462562+0000", 2024-09-14T05:29:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7649999999999999e-05 2024-09-14T05:29:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462805+0000", 2024-09-14T05:29:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024263400000000001 2024-09-14T05:29:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4052 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 12550144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:29:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.059959983999999, 2024-09-14T05:29:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060872904999999998, 2024-09-14T05:29:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4052 2024-09-14T05:29:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:29:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:29:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417486+0000", 2024-09-14T05:29:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-14T05:29:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417487+0000", 2024-09-14T05:29:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4499999999999995e-07 2024-09-14T05:29:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417489+0000", 2024-09-14T05:29:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8899999999999999e-06 2024-09-14T05:29:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417492+0000", 2024-09-14T05:29:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4389999999999999e-06 2024-09-14T05:29:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478027+0000", 2024-09-14T05:29:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060534959999999999 2024-09-14T05:29:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478079+0000", 2024-09-14T05:29:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2104999999999997e-05 2024-09-14T05:29:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478358+0000", 2024-09-14T05:29:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000279351 2024-09-14T05:29:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4053 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13041664~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:29:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.059950600000001, 2024-09-14T05:29:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061258866000000002, 2024-09-14T05:29:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4053 2024-09-14T05:29:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:29:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:29:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:29:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.17e-07 2024-09-14T05:29:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417496+0000", 2024-09-14T05:29:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0890000000000001e-06 2024-09-14T05:29:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417499+0000", 2024-09-14T05:29:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8629999999999999e-06 2024-09-14T05:29:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417502+0000", 2024-09-14T05:29:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6290000000000002e-06 2024-09-14T05:29:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478387+0000", 2024-09-14T05:29:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060885433000000003 2024-09-14T05:29:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478435+0000", 2024-09-14T05:29:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8167999999999997e-05 2024-09-14T05:29:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478754+0000", 2024-09-14T05:29:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031846700000000001 2024-09-14T05:29:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4054 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13533184~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:29:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.059940716, 2024-09-14T05:29:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061604694000000002, 2024-09-14T05:29:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4054 2024-09-14T05:29:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:29:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:29:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:29:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.04e-07 2024-09-14T05:29:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417506+0000", 2024-09-14T05:29:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8299999999999995e-07 2024-09-14T05:29:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417509+0000", 2024-09-14T05:29:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.88e-06 2024-09-14T05:29:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417511+0000", 2024-09-14T05:29:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5950000000000001e-06 2024-09-14T05:29:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478784+0000", 2024-09-14T05:29:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061272971000000002 2024-09-14T05:29:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478841+0000", 2024-09-14T05:29:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-14T05:29:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479109+0000", 2024-09-14T05:29:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026794600000000003 2024-09-14T05:29:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4055 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14024704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:29:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.059931014, 2024-09-14T05:29:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061932299000000003, 2024-09-14T05:29:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4055 2024-09-14T05:29:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:29:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:29:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417515+0000", 2024-09-14T05:29:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-14T05:29:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417516+0000", 2024-09-14T05:29:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7199999999999997e-07 2024-09-14T05:29:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417519+0000", 2024-09-14T05:29:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.937e-06 2024-09-14T05:29:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417522+0000", 2024-09-14T05:29:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1420000000000002e-06 2024-09-14T05:29:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479140+0000", 2024-09-14T05:29:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061617933999999999 2024-09-14T05:29:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479188+0000", 2024-09-14T05:29:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8717000000000001e-05 2024-09-14T05:29:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479447+0000", 2024-09-14T05:29:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025838800000000002 2024-09-14T05:29:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4056 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14516224~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:29:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.059920841, 2024-09-14T05:29:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062263185999999998, 2024-09-14T05:29:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4056 2024-09-14T05:29:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:29:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:29:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:29:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8700000000000002e-07 2024-09-14T05:29:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417528+0000", 2024-09-14T05:29:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.579e-06 2024-09-14T05:29:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417533+0000", 2024-09-14T05:29:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2400000000000001e-06 2024-09-14T05:29:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417536+0000", 2024-09-14T05:29:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0309999999999999e-06 2024-09-14T05:29:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479477+0000", 2024-09-14T05:29:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061940967 2024-09-14T05:29:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479523+0000", 2024-09-14T05:29:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6335000000000003e-05 2024-09-14T05:29:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479788+0000", 2024-09-14T05:29:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026474700000000002 2024-09-14T05:29:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.575 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.575 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.575 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4057 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15007744~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.575 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:29:55.575 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.059906380000001, 2024-09-14T05:29:55.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062589928000000003, 2024-09-14T05:29:55.575 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.575 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.575 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.576 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4057 2024-09-14T05:29:55.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.576 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:29:55.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:29:55.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422838+0000", 2024-09-14T05:29:55.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052990349999999997 2024-09-14T05:29:55.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422854+0000", 2024-09-14T05:29:55.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5523999999999999e-05 2024-09-14T05:29:55.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422864+0000", 2024-09-14T05:29:55.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0084e-05 2024-09-14T05:29:55.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422872+0000", 2024-09-14T05:29:55.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-14T05:29:55.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479813+0000", 2024-09-14T05:29:55.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056941389000000002 2024-09-14T05:29:55.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479868+0000", 2024-09-14T05:29:55.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4973999999999999e-05 2024-09-14T05:29:55.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.480129+0000", 2024-09-14T05:29:55.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026084200000000001 2024-09-14T05:29:55.579 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.579 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.579 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.579 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4171 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14057472~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.579 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:29:55.579 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.04790882, 2024-09-14T05:29:55.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061184929999999998, 2024-09-14T05:29:55.580 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.580 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.580 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.580 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.580 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.580 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.580 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4171 2024-09-14T05:29:55.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.580 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:29:55.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:29:55.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:29:55.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.39e-07 2024-09-14T05:29:55.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429541+0000", 2024-09-14T05:29:55.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:29:55.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429547+0000", 2024-09-14T05:29:55.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3280000000000002e-06 2024-09-14T05:29:55.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429552+0000", 2024-09-14T05:29:55.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4859999999999997e-06 2024-09-14T05:29:55.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490379+0000", 2024-09-14T05:29:55.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060826681 2024-09-14T05:29:55.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490434+0000", 2024-09-14T05:29:55.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4577999999999998e-05 2024-09-14T05:29:55.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490722+0000", 2024-09-14T05:29:55.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028781400000000001 2024-09-14T05:29:55.583 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.584 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.584 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.584 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4172 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14548992~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.584 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:29:55.584 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.047886812000002, 2024-09-14T05:29:55.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061498256000000001, 2024-09-14T05:29:55.584 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.584 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.584 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.584 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.584 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.585 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.585 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4172 2024-09-14T05:29:55.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.585 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:29:55.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:29:55.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:29:55.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.72e-07 2024-09-14T05:29:55.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429561+0000", 2024-09-14T05:29:55.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.717e-06 2024-09-14T05:29:55.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429566+0000", 2024-09-14T05:29:55.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6529999999999999e-06 2024-09-14T05:29:55.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429571+0000", 2024-09-14T05:29:55.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5979999999999999e-06 2024-09-14T05:29:55.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490751+0000", 2024-09-14T05:29:55.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061180336000000002 2024-09-14T05:29:55.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490798+0000", 2024-09-14T05:29:55.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6875000000000001e-05 2024-09-14T05:29:55.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491057+0000", 2024-09-14T05:29:55.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025870500000000002 2024-09-14T05:29:55.588 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.588 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.588 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.588 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4173 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15040512~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:55.588 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:29:55.588 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.047869186, 2024-09-14T05:29:55.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061894820000000003, 2024-09-14T05:29:55.589 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:55.589 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:55.589 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:55.589 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:55.589 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:55.589 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:55.589 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4173 2024-09-14T05:29:55.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.589 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:55.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:55.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:29:55.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:55.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:29:55.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:55.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:55.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429577+0000", 2024-09-14T05:29:55.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-14T05:29:55.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:55.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429578+0000", 2024-09-14T05:29:55.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6980000000000001e-06 2024-09-14T05:29:55.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:55.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429585+0000", 2024-09-14T05:29:55.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.725e-06 2024-09-14T05:29:55.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:55.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429590+0000", 2024-09-14T05:29:55.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.938e-06 2024-09-14T05:29:55.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:55.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491091+0000", 2024-09-14T05:29:55.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061500773000000002 2024-09-14T05:29:55.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:55.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491136+0000", 2024-09-14T05:29:55.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5284000000000002e-05 2024-09-14T05:29:55.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:55.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:55.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:55.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491471+0000", 2024-09-14T05:29:55.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033485699999999998 2024-09-14T05:29:55.592 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.593 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.593 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.593 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:55.593 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:55.593 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:55.594 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:55.595 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:55.595 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:55.595 INFO:teuthology.orchestra.run.smithi071.stdout:2120 read 0xb232e4f thru 0xb237de4 (0x4f96 bytes) 2024-09-14T05:29:55.595 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:55.595 INFO:teuthology.orchestra.run.smithi071.stdout:2122 write 0xbc1276a thru 0xbc19e13 (0x76aa bytes) 2024-09-14T05:29:55.595 INFO:teuthology.orchestra.run.smithi071.stdout:2125 read 0x3b6b5a thru 0x3ba4c2 (0x3969 bytes) 2024-09-14T05:29:55.595 INFO:teuthology.orchestra.run.smithi071.stdout:2126 write 0x9bb2dae thru 0x9bbc7ae (0x9a01 bytes) 2024-09-14T05:29:55.595 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:29:55.596 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:29:55.596 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:29:55.597 INFO:teuthology.orchestra.run.smithi071.stdout:2127 trunc from 0xdddcb2e to 0x6d88562 2024-09-14T05:29:55.597 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:29:55.597 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:29:55.598 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:29:55.598 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:29:55.598 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:55.593+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:55.609 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:29:55.609 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:29:55.627 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:29:55.627 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:29:55.646 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:29:55.646 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:29:55.654 INFO:teuthology.orchestra.run.smithi071.stdout:2128 trunc from 0x6d88562 to 0xadd2492 2024-09-14T05:29:55.659 INFO:teuthology.orchestra.run.smithi071.stdout:2129 read 0x5bf5518 thru 0x5bff5eb (0xa0d4 bytes) 2024-09-14T05:29:55.662 INFO:teuthology.orchestra.run.smithi071.stdout:2130 trunc from 0xadd2492 to 0xce98e67 2024-09-14T05:29:55.665 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:29:55.665 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:29:55.667 INFO:teuthology.orchestra.run.smithi071.stdout:2131 read 0x443b5a9 thru 0x443c8cd (0x1325 bytes) 2024-09-14T05:29:55.670 INFO:teuthology.orchestra.run.smithi071.stdout:2134 trunc from 0xce98e67 to 0xb53251c 2024-09-14T05:29:55.681 INFO:teuthology.orchestra.run.smithi071.stdout:2136 write 0xc9e4e6d thru 0xc9e8c37 (0x3dcb bytes) 2024-09-14T05:29:55.684 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:29:55.685 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:29:55.685 INFO:teuthology.orchestra.run.smithi071.stdout:2137 read 0x4c3d37e thru 0x4c4620e (0x8e91 bytes) 2024-09-14T05:29:55.686 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:55.686 INFO:teuthology.orchestra.run.smithi071.stdout:2140 read 0xc24fb1e thru 0xc25bcb9 (0xc19c bytes) 2024-09-14T05:29:55.686 INFO:teuthology.orchestra.run.smithi071.stdout:2141 read 0x993d78c thru 0x9941309 (0x3b7e bytes) 2024-09-14T05:29:55.686 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:55.686 INFO:teuthology.orchestra.run.smithi071.stdout:2143 writesame 0x2526dd9 thru 0x252adb3 data_size 0x1(0x3fdb bytes) 2024-09-14T05:29:55.694 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:55.694+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:55.732 INFO:teuthology.orchestra.run.smithi071.stdout:2144 read 0x9ee9d3a thru 0x9ef51f4 (0xb4bb bytes) 2024-09-14T05:29:55.732 INFO:teuthology.orchestra.run.smithi071.stdout:2146 read 0x7355b0e thru 0x73645b2 (0xeaa5 bytes) 2024-09-14T05:29:55.733 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:55.733 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:55.733 INFO:teuthology.orchestra.run.smithi071.stdout:2150 read 0xb2f950 thru 0xb36707 (0x6db8 bytes) 2024-09-14T05:29:55.738 INFO:teuthology.orchestra.run.smithi071.stdout:2152 write 0x7e011a8 thru 0x7e07e77 (0x6cd0 bytes) 2024-09-14T05:29:55.738 INFO:teuthology.orchestra.run.smithi071.stdout:2153 read 0xc3872aa thru 0xc390e33 (0x9b8a bytes) 2024-09-14T05:29:55.738 INFO:teuthology.orchestra.run.smithi071.stdout:2154 write 0x556ffd7 thru 0x5570aa3 (0xacd bytes) 2024-09-14T05:29:55.748 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:55.748 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:55.758 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:29:55.779 INFO:teuthology.orchestra.run.smithi071.stdout:2155 read 0x92c3c5b thru 0x92d050b (0xc8b1 bytes) 2024-09-14T05:29:55.793 INFO:teuthology.orchestra.run.smithi071.stdout:2156 trunc from 0xc9e8c38 to 0xe42ab3 2024-09-14T05:29:55.799 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:55.799+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:55.837 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:29:55.837 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:55.838 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 52, 2024-09-14T05:29:55.838 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "1127be44-5f14-4e7d-ad23-f608afe154f5", 2024-09-14T05:29:55.838 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2024-09-14T05:28:54.203072+0000", 2024-09-14T05:29:55.838 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-14T05:29:55.161527+0000", 2024-09-14T05:29:55.838 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2024-09-14T05:29:06.755531+0000", 2024-09-14T05:29:55.838 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2024-09-14T05:28:56.535048+0000", 2024-09-14T05:29:55.838 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T05:29:55.838 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5804032, 2024-09-14T05:29:55.838 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2024-09-14T05:29:55.838 INFO:teuthology.orchestra.run.smithi071.stdout: "nodeep-scrub", 2024-09-14T05:29:55.838 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2024-09-14T05:29:55.838 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2024-09-14T05:29:55.839 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2024-09-14T05:29:55.839 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2024-09-14T05:29:55.839 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:29:55.839 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 6, 2024-09-14T05:29:55.839 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T05:29:55.839 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T05:29:55.839 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T05:29:55.839 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2024-09-14T05:29:55.839 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 3, 2024-09-14T05:29:55.839 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2024-09-14T05:29:55.839 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2024-09-14T05:29:55.839 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2024-09-14T05:29:55.840 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "squid", 2024-09-14T05:29:55.840 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2024-09-14T05:29:55.840 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2024-09-14T05:29:55.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.840 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2024-09-14T05:29:55.840 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2024-09-14T05:29:55.840 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-14T05:29:06.452744+0000", 2024-09-14T05:29:55.840 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-14T05:29:55.840 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2024-09-14T05:29:55.840 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-14T05:29:55.840 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-14T05:29:55.840 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-14T05:29:55.840 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-14T05:29:55.841 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-14T05:29:55.841 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-14T05:29:55.841 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T05:29:55.841 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T05:29:55.841 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-14T05:29:55.841 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-14T05:29:55.841 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-14T05:29:55.841 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2024-09-14T05:29:55.841 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2024-09-14T05:29:55.841 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2024-09-14T05:29:55.841 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2024-09-14T05:29:55.841 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2024-09-14T05:29:55.841 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-14T05:29:55.842 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-14T05:29:55.842 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-14T05:29:55.842 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-14T05:29:55.842 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-14T05:29:55.842 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-14T05:29:55.842 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-14T05:29:55.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.842 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "18", 2024-09-14T05:29:55.842 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-14T05:29:55.842 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T05:29:55.842 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T05:29:55.842 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-14T05:29:55.843 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-14T05:29:55.843 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2024-09-14T05:29:55.843 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2024-09-14T05:29:55.843 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-14T05:29:55.843 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-14T05:29:55.843 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-14T05:29:55.843 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-14T05:29:55.843 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-14T05:29:55.843 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-14T05:29:55.843 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-14T05:29:55.843 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-14T05:29:55.843 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-14T05:29:55.843 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-14T05:29:55.844 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-14T05:29:55.844 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T05:29:55.844 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T05:29:55.844 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T05:29:55.844 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-14T05:29:55.844 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-14T05:29:55.844 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-14T05:29:55.844 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-14T05:29:55.844 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-14T05:29:55.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.844 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-14T05:29:55.845 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-14T05:29:55.845 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-14T05:29:55.845 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-14T05:29:55.845 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-14T05:29:55.845 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T05:29:55.845 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-14T05:29:55.845 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-14T05:29:55.845 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-14T05:29:55.845 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-14T05:29:55.845 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-14T05:29:55.845 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2024-09-14T05:29:55.845 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2024-09-14T05:29:55.845 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2024-09-14T05:29:55.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.846 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-14T05:29:55.846 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2024-09-14T05:29:55.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.846 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-14T05:29:55.846 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-14T05:29:55.846 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2024-09-14T05:29:55.846 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2024-09-14T05:29:55.846 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2024-09-14T05:29:55.846 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2024-09-14T05:29:55.846 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2024-09-14T05:29:55.846 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.87708091735839844, 2024-09-14T05:29:55.846 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T05:29:55.847 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T05:29:55.847 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.847 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 2, 2024-09-14T05:29:55.847 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "rbd", 2024-09-14T05:29:55.847 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-14T05:29:12.380843+0000", 2024-09-14T05:29:55.847 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 8193, 2024-09-14T05:29:55.847 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T05:29:55.847 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-14T05:29:55.847 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-14T05:29:55.847 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-14T05:29:55.847 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-14T05:29:55.847 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-14T05:29:55.848 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-14T05:29:55.848 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T05:29:55.848 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T05:29:55.848 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-14T05:29:55.848 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-14T05:29:55.848 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-14T05:29:55.848 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 8, 2024-09-14T05:29:55.848 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 8, 2024-09-14T05:29:55.848 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 8, 2024-09-14T05:29:55.848 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 8, 2024-09-14T05:29:55.848 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 8, 2024-09-14T05:29:55.848 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-14T05:29:55.848 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-14T05:29:55.848 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-14T05:29:55.849 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-14T05:29:55.849 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-14T05:29:55.849 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-14T05:29:55.849 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-14T05:29:55.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.849 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "22", 2024-09-14T05:29:55.849 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-14T05:29:55.849 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T05:29:55.849 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T05:29:55.849 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-14T05:29:55.850 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-14T05:29:55.850 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 2, 2024-09-14T05:29:55.850 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 22, 2024-09-14T05:29:55.850 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-14T05:29:55.850 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-14T05:29:55.850 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-14T05:29:55.850 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-14T05:29:55.850 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-14T05:29:55.850 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-14T05:29:55.850 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-14T05:29:55.850 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-14T05:29:55.850 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-14T05:29:55.850 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-14T05:29:55.851 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-14T05:29:55.851 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T05:29:55.851 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T05:29:55.851 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T05:29:55.851 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-14T05:29:55.851 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-14T05:29:55.851 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-14T05:29:55.851 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-14T05:29:55.851 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-14T05:29:55.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.851 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-14T05:29:55.851 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-14T05:29:55.851 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-14T05:29:55.852 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-14T05:29:55.852 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-14T05:29:55.852 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T05:29:55.852 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-14T05:29:55.852 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-14T05:29:55.852 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-14T05:29:55.852 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-14T05:29:55.852 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-14T05:29:55.852 INFO:teuthology.orchestra.run.smithi071.stdout: "options": {}, 2024-09-14T05:29:55.852 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-14T05:29:55.852 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd": {} 2024-09-14T05:29:55.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.852 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-14T05:29:55.853 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-14T05:29:55.853 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 4.9800000190734863, 2024-09-14T05:29:55.853 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 4.9800000190734863, 2024-09-14T05:29:55.853 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T05:29:55.853 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 4.380000114440918, 2024-09-14T05:29:55.853 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 4.380000114440918, 2024-09-14T05:29:55.854 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.87708091735839844, 2024-09-14T05:29:55.854 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T05:29:55.854 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T05:29:55.854 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.854 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 3, 2024-09-14T05:29:55.854 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "pool_client.0", 2024-09-14T05:29:55.854 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-14T05:29:22.335419+0000", 2024-09-14T05:29:55.854 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 8193, 2024-09-14T05:29:55.854 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T05:29:55.854 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-14T05:29:55.855 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-14T05:29:55.855 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-14T05:29:55.855 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-14T05:29:55.855 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-14T05:29:55.855 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-14T05:29:55.855 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T05:29:55.855 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T05:29:55.855 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-14T05:29:55.855 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-14T05:29:55.855 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-14T05:29:55.856 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2024-09-14T05:29:55.856 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2024-09-14T05:29:55.856 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2024-09-14T05:29:55.856 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2024-09-14T05:29:55.856 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2024-09-14T05:29:55.856 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-14T05:29:55.856 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-14T05:29:55.856 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-14T05:29:55.856 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-14T05:29:55.856 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-14T05:29:55.856 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-14T05:29:55.857 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-14T05:29:55.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.857 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "52", 2024-09-14T05:29:55.857 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-14T05:29:55.857 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T05:29:55.857 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T05:29:55.857 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-14T05:29:55.857 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-14T05:29:55.857 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 22, 2024-09-14T05:29:55.857 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 52, 2024-09-14T05:29:55.857 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-14T05:29:55.857 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-14T05:29:55.857 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-14T05:29:55.858 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-14T05:29:55.858 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-14T05:29:55.858 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-14T05:29:55.858 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-14T05:29:55.858 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-14T05:29:55.858 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-14T05:29:55.858 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-14T05:29:55.858 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-14T05:29:55.858 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T05:29:55.858 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T05:29:55.858 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T05:29:55.859 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-14T05:29:55.859 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-14T05:29:55.859 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-14T05:29:55.859 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-14T05:29:55.859 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-14T05:29:55.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.859 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-14T05:29:55.859 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-14T05:29:55.859 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-14T05:29:55.859 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-14T05:29:55.860 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-14T05:29:55.860 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T05:29:55.860 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-14T05:29:55.860 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-14T05:29:55.860 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-14T05:29:55.860 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-14T05:29:55.860 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-14T05:29:55.860 INFO:teuthology.orchestra.run.smithi071.stdout: "options": {}, 2024-09-14T05:29:55.860 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-14T05:29:55.860 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd": {} 2024-09-14T05:29:55.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.860 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-14T05:29:55.860 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-14T05:29:55.861 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2024-09-14T05:29:55.861 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2024-09-14T05:29:55.861 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2024-09-14T05:29:55.861 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2024-09-14T05:29:55.861 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2024-09-14T05:29:55.861 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.87708091735839844, 2024-09-14T05:29:55.861 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T05:29:55.861 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T05:29:55.861 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.861 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.861 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:29:55.861 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2024-09-14T05:29:55.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.862 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2024-09-14T05:29:55.862 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "e7c7817f-4a65-4f66-9af5-d31d0a92cf30", 2024-09-14T05:29:55.862 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:55.862 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:55.862 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:55.862 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:55.862 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:55.862 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:55.862 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 16, 2024-09-14T05:29:55.862 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:29:55.862 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:55.863 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:55.863 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:55.863 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.863 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.863 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2024-09-14T05:29:55.863 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:55.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.863 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.863 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2024-09-14T05:29:55.863 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:55.863 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.863 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.864 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:55.864 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.864 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.864 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2024-09-14T05:29:55.864 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:55.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.865 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.865 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2024-09-14T05:29:55.865 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:55.865 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.865 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.865 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:55.865 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.865 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.865 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2024-09-14T05:29:55.865 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:55.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.866 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.866 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2024-09-14T05:29:55.866 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:55.866 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.866 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.866 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:55.866 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.866 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.866 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2024-09-14T05:29:55.867 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:55.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.867 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.867 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2024-09-14T05:29:55.867 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:29:55.867 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.867 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.867 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6819/2926109885", 2024-09-14T05:29:55.867 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6821/2926109885", 2024-09-14T05:29:55.867 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6825/2926109885", 2024-09-14T05:29:55.867 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6823/2926109885", 2024-09-14T05:29:55.867 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:55.868 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:55.868 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:55.868 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.868 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2024-09-14T05:29:55.868 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "e184604e-b973-4b7a-bba2-6f471753a0fd", 2024-09-14T05:29:55.868 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:55.868 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:55.868 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:55.868 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:55.868 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:55.868 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:55.869 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:55.869 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:29:55.869 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:55.869 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:55.869 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:55.869 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.869 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.869 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2024-09-14T05:29:55.869 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:55.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.869 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.870 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2024-09-14T05:29:55.870 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:55.870 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.870 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.870 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:55.870 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.870 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.870 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2024-09-14T05:29:55.870 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:55.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.870 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.871 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2024-09-14T05:29:55.871 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:55.871 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.871 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.872 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:55.872 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.872 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.872 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6832", 2024-09-14T05:29:55.873 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:55.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.873 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.873 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6833", 2024-09-14T05:29:55.874 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:55.874 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.874 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.874 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:55.874 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.875 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.875 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2024-09-14T05:29:55.875 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:55.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.876 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.876 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2024-09-14T05:29:55.876 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:29:55.877 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.877 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.877 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6827/345649648", 2024-09-14T05:29:55.877 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6829/345649648", 2024-09-14T05:29:55.877 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6833/345649648", 2024-09-14T05:29:55.878 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6831/345649648", 2024-09-14T05:29:55.878 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:55.878 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:55.878 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:55.878 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.879 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2024-09-14T05:29:55.879 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "d8b70f09-5afb-4296-b8c9-719c172e0e70", 2024-09-14T05:29:55.879 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:55.880 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:55.880 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:55.880 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:55.880 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:55.880 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:55.880 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:55.881 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 39, 2024-09-14T05:29:55.881 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:55.881 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:55.881 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:55.881 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.882 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.882 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2024-09-14T05:29:55.882 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:55.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.883 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.883 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2024-09-14T05:29:55.883 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:55.883 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.883 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.884 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:55.884 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.884 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.885 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2024-09-14T05:29:55.885 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:55.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.885 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.886 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2024-09-14T05:29:55.886 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:55.886 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.886 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.887 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:55.887 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.887 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.887 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2024-09-14T05:29:55.888 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:55.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.888 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.888 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2024-09-14T05:29:55.889 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:55.889 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.889 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.889 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:55.890 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.890 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.890 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2024-09-14T05:29:55.890 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:55.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.891 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.891 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2024-09-14T05:29:55.891 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:29:55.892 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.892 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.892 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/2721860691", 2024-09-14T05:29:55.892 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/2721860691", 2024-09-14T05:29:55.892 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/2721860691", 2024-09-14T05:29:55.893 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/2721860691", 2024-09-14T05:29:55.893 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:55.893 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:55.893 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:55.893 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.894 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2024-09-14T05:29:55.894 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c", 2024-09-14T05:29:55.894 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:55.894 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:55.895 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:55.895 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:55.895 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:55.895 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:55.895 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:55.895 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 39, 2024-09-14T05:29:55.896 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:55.896 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:55.896 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:55.896 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.897 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.897 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2024-09-14T05:29:55.897 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:55.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.898 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.898 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2024-09-14T05:29:55.898 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:55.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.898 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.899 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:55.899 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.899 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.900 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2024-09-14T05:29:55.900 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:55.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.900 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.901 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2024-09-14T05:29:55.901 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:55.901 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.901 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.901 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:55.902 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.902 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.902 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2024-09-14T05:29:55.902 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:55.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.903 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.903 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2024-09-14T05:29:55.903 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:55.904 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.904 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.904 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:55.904 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.905 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.905 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2024-09-14T05:29:55.905 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:55.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.906 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.906 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2024-09-14T05:29:55.906 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:29:55.906 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.907 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.907 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/494607463", 2024-09-14T05:29:55.907 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6811/494607463", 2024-09-14T05:29:55.907 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6815/494607463", 2024-09-14T05:29:55.907 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6813/494607463", 2024-09-14T05:29:55.908 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:55.908 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:55.908 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:55.908 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.909 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2024-09-14T05:29:55.909 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "515cb7d6-16eb-4aff-973a-7f7ed49325ba", 2024-09-14T05:29:55.909 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:55.909 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:55.910 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:55.910 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:55.910 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:55.910 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:55.910 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:55.911 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:29:55.911 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:55.911 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:55.911 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:55.911 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.912 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.912 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6816", 2024-09-14T05:29:55.912 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:55.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.913 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.913 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6817", 2024-09-14T05:29:55.913 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:55.913 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.913 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.914 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:55.914 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.914 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.915 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6818", 2024-09-14T05:29:55.915 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:55.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.915 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.916 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6819", 2024-09-14T05:29:55.916 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:55.916 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.916 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.917 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:55.917 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.917 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.917 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6822", 2024-09-14T05:29:55.918 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:55.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.918 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.918 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6823", 2024-09-14T05:29:55.918 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:55.919 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.919 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.919 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:55.919 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.920 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.920 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6820", 2024-09-14T05:29:55.920 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:55.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.921 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.921 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6821", 2024-09-14T05:29:55.921 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:29:55.921 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.921 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.922 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6817/2240133417", 2024-09-14T05:29:55.922 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6819/2240133417", 2024-09-14T05:29:55.922 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6823/2240133417", 2024-09-14T05:29:55.922 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6821/2240133417", 2024-09-14T05:29:55.923 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:55.923 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:55.923 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:55.923 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.924 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2024-09-14T05:29:55.924 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "c434b53a-04be-45b9-b89a-193654133950", 2024-09-14T05:29:55.924 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:55.924 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:55.924 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:55.924 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:55.925 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:55.925 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:55.925 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:55.925 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 39, 2024-09-14T05:29:55.925 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:55.925 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:55.925 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:55.925 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.925 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.925 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6808", 2024-09-14T05:29:55.925 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:55.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.926 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.926 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6809", 2024-09-14T05:29:55.926 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:55.926 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.926 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.926 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:55.926 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.926 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.926 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6810", 2024-09-14T05:29:55.926 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:55.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.927 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.927 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6811", 2024-09-14T05:29:55.927 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:55.927 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.927 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.927 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:55.927 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.928 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.928 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6814", 2024-09-14T05:29:55.928 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:55.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.928 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.928 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6815", 2024-09-14T05:29:55.928 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:55.929 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.929 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.929 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:55.929 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.929 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.929 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6812", 2024-09-14T05:29:55.929 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:55.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.929 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.929 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6813", 2024-09-14T05:29:55.929 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:29:55.930 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.930 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.930 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6809/2393825223", 2024-09-14T05:29:55.930 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6811/2393825223", 2024-09-14T05:29:55.930 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6815/2393825223", 2024-09-14T05:29:55.930 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6813/2393825223", 2024-09-14T05:29:55.930 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:55.930 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:55.930 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:55.930 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.930 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2024-09-14T05:29:55.931 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "3d2f8405-9286-4b40-8acd-036f67373ac6", 2024-09-14T05:29:55.931 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:55.931 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:55.931 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:55.931 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 0.0166473388671875, 2024-09-14T05:29:55.931 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:55.931 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:55.931 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:55.931 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:29:55.931 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:55.931 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:55.931 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:55.931 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.931 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.932 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6800", 2024-09-14T05:29:55.932 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:55.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.932 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.932 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6801", 2024-09-14T05:29:55.932 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:55.932 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.932 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.932 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:55.932 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.932 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.933 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6802", 2024-09-14T05:29:55.933 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:55.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.933 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.933 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6803", 2024-09-14T05:29:55.933 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:55.933 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.933 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.933 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:55.933 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.933 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.934 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6806", 2024-09-14T05:29:55.934 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:55.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.934 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.934 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6807", 2024-09-14T05:29:55.934 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:55.934 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.934 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.934 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:55.934 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.934 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.934 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6804", 2024-09-14T05:29:55.935 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:55.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.935 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.935 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6805", 2024-09-14T05:29:55.935 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:29:55.935 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.935 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.935 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6801/3235729544", 2024-09-14T05:29:55.935 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6803/3235729544", 2024-09-14T05:29:55.935 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6807/3235729544", 2024-09-14T05:29:55.935 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6805/3235729544", 2024-09-14T05:29:55.935 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:55.935 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:55.936 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:55.936 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.936 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2024-09-14T05:29:55.936 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "6724f080-4dc3-465a-939f-b023985bf998", 2024-09-14T05:29:55.936 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:29:55.936 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:29:55.936 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:29:55.936 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:29:55.936 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:29:55.936 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:29:55.936 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:29:55.936 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:29:55.936 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:29:55.937 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:29:55.937 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:29:55.937 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.937 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.941 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6824", 2024-09-14T05:29:55.941 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:55.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.941 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.941 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6825", 2024-09-14T05:29:55.941 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:55.941 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.941 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.941 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:29:55.941 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.942 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.942 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6826", 2024-09-14T05:29:55.942 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:55.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.942 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.942 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6827", 2024-09-14T05:29:55.942 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:55.942 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.942 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.942 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:29:55.942 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.943 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.943 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6830", 2024-09-14T05:29:55.943 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:55.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.943 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.943 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6831", 2024-09-14T05:29:55.943 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:55.943 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.943 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.943 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:29:55.944 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:29:55.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.944 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:29:55.944 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6828", 2024-09-14T05:29:55.944 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:55.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.944 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:29:55.944 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6829", 2024-09-14T05:29:55.944 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:29:55.944 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.944 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.945 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6825/2381001355", 2024-09-14T05:29:55.945 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6827/2381001355", 2024-09-14T05:29:55.945 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6831/2381001355", 2024-09-14T05:29:55.945 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6829/2381001355", 2024-09-14T05:29:55.945 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:29:55.945 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:29:55.945 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:29:55.945 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:55.945 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.945 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:29:55.945 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2024-09-14T05:29:55.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.945 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2024-09-14T05:29:55.945 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:55.946 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:55.946 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:55.946 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:55.946 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:55.946 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.548803+0000", 2024-09-14T05:29:55.946 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:55.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.946 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2024-09-14T05:29:55.946 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:55.946 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:55.946 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:55.947 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:55.947 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:55.947 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.075023+0000", 2024-09-14T05:29:55.947 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:55.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.947 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2024-09-14T05:29:55.947 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:55.947 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:55.947 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:55.947 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:55.947 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:55.947 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.117036+0000", 2024-09-14T05:29:55.947 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:55.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.948 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2024-09-14T05:29:55.948 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:55.948 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:55.948 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:55.948 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:55.948 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:55.948 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.332113+0000", 2024-09-14T05:29:55.948 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:55.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.948 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2024-09-14T05:29:55.949 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:55.949 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:55.949 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:55.949 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:55.949 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:55.949 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.330371+0000", 2024-09-14T05:29:55.949 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:55.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.949 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2024-09-14T05:29:55.949 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:55.949 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:55.949 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:55.949 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:55.949 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:55.950 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:02.753946+0000", 2024-09-14T05:29:55.950 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:55.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.950 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2024-09-14T05:29:55.950 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:55.950 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:55.950 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:55.950 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:55.950 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:55.950 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:02.071186+0000", 2024-09-14T05:29:55.950 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:55.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:55.951 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2024-09-14T05:29:55.951 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:29:55.951 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:29:55.951 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:29:55.951 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:29:55.951 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:29:55.951 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.378781+0000", 2024-09-14T05:29:55.951 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:29:55.951 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.951 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:29:55.951 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2024-09-14T05:29:55.951 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2024-09-14T05:29:55.952 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2024-09-14T05:29:55.952 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2024-09-14T05:29:55.952 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2024-09-14T05:29:55.952 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2024-09-14T05:29:55.952 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2024-09-14T05:29:55.952 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2024-09-14T05:29:55.952 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2024-09-14T05:29:55.952 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2024-09-14T05:29:55.952 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2024-09-14T05:29:55.952 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2024-09-14T05:29:55.952 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2024-09-14T05:29:55.952 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2024-09-14T05:29:55.952 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:55.953 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2024-09-14T05:29:55.953 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2024-09-14T05:29:55.953 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2024-09-14T05:29:55.953 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2024-09-14T05:29:55.953 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2024-09-14T05:29:55.953 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2024-09-14T05:29:55.953 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2024-09-14T05:29:55.953 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2024-09-14T05:29:55.953 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2024-09-14T05:29:55.953 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2024-09-14T05:29:55.953 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2024-09-14T05:29:55.953 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:55.954 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:55.955 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:55.956 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:55.956 INFO:tasks.thrashosds.thrasher:j is {'epoch': 52, 'fsid': '1127be44-5f14-4e7d-ad23-f608afe154f5', 'created': '2024-09-14T05:28:54.203072+0000', 'modified': '2024-09-14T05:29:55.161527+0000', 'last_up_change': '2024-09-14T05:29:06.755531+0000', 'last_in_change': '2024-09-14T05:28:56.535048+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T05:29:06.452744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8770809173583984, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T05:29:12.380843+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.8770809173583984, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T05:29:22.335419+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 22, 'snap_epoch': 52, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8770809173583984, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'e7c7817f-4a65-4f66-9af5-d31d0a92cf30', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6818', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6819', 'nonce': 2926109885}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6820', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6821', 'nonce': 2926109885}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6824', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6825', 'nonce': 2926109885}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6822', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6823', 'nonce': 2926109885}]}, 'public_addr': '172.21.15.71:6819/2926109885', 'cluster_addr': '172.21.15.71:6821/2926109885', 'heartbeat_back_addr': '172.21.15.71:6825/2926109885', 'heartbeat_front_addr': '172.21.15.71:6823/2926109885', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e184604e-b973-4b7a-bba2-6f471753a0fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6826', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6827', 'nonce': 345649648}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6828', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6829', 'nonce': 345649648}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6832', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6833', 'nonce': 345649648}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6830', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6831', 'nonce': 345649648}]}, 'public_addr': '172.21.15.71:6827/345649648', 'cluster_addr': '172.21.15.71:6829/345649648', 'heartbeat_back_addr': '172.21.15.71:6833/345649648', 'heartbeat_front_addr': '172.21.15.71:6831/345649648', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd8b70f09-5afb-4296-b8c9-719c172e0e70', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6800', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6801', 'nonce': 2721860691}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6802', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6803', 'nonce': 2721860691}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6806', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6807', 'nonce': 2721860691}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6804', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6805', 'nonce': 2721860691}]}, 'public_addr': '172.21.15.71:6801/2721860691', 'cluster_addr': '172.21.15.71:6803/2721860691', 'heartbeat_back_addr': '172.21.15.71:6807/2721860691', 'heartbeat_front_addr': '172.21.15.71:6805/2721860691', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6808', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6809', 'nonce': 494607463}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6810', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6811', 'nonce': 494607463}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6814', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6815', 'nonce': 494607463}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6812', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6813', 'nonce': 494607463}]}, 'public_addr': '172.21.15.71:6809/494607463', 'cluster_addr': '172.21.15.71:6811/494607463', 'heartbeat_back_addr': '172.21.15.71:6815/494607463', 'heartbeat_front_addr': '172.21.15.71:6813/494607463', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '515cb7d6-16eb-4aff-973a-7f7ed49325ba', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6816', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6817', 'nonce': 2240133417}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6818', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6819', 'nonce': 2240133417}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6822', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6823', 'nonce': 2240133417}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6820', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6821', 'nonce': 2240133417}]}, 'public_addr': '172.21.15.150:6817/2240133417', 'cluster_addr': '172.21.15.150:6819/2240133417', 'heartbeat_back_addr': '172.21.15.150:6823/2240133417', 'heartbeat_front_addr': '172.21.15.150:6821/2240133417', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'c434b53a-04be-45b9-b89a-193654133950', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6808', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6809', 'nonce': 2393825223}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6810', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6811', 'nonce': 2393825223}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6814', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6815', 'nonce': 2393825223}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6812', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6813', 'nonce': 2393825223}]}, 'public_addr': '172.21.15.150:6809/2393825223', 'cluster_addr': '172.21.15.150:6811/2393825223', 'heartbeat_back_addr': '172.21.15.150:6815/2393825223', 'heartbeat_front_addr': '172.21.15.150:6813/2393825223', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '3d2f8405-9286-4b40-8acd-036f67373ac6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0166473388671875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6800', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6801', 'nonce': 3235729544}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6802', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6803', 'nonce': 3235729544}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6806', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6807', 'nonce': 3235729544}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6804', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6805', 'nonce': 3235729544}]}, 'public_addr': '172.21.15.150:6801/3235729544', 'cluster_addr': '172.21.15.150:6803/3235729544', 'heartbeat_back_addr': '172.21.15.150:6807/3235729544', 'heartbeat_front_addr': '172.21.15.150:6805/3235729544', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '6724f080-4dc3-465a-939f-b023985bf998', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6824', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6825', 'nonce': 2381001355}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6826', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6827', 'nonce': 2381001355}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6830', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6831', 'nonce': 2381001355}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6828', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6829', 'nonce': 2381001355}]}, 'public_addr': '172.21.15.150:6825/2381001355', 'cluster_addr': '172.21.15.150:6827/2381001355', 'heartbeat_back_addr': '172.21.15.150:6831/2381001355', 'heartbeat_front_addr': '172.21.15.150:6829/2381001355', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.548803+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.075023+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.117036+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.332113+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.330371+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:02.753946+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:02.071186+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.378781+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T05:29:55.957 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-14T05:29:55.957 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:29:55.957 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:55.954+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:55.958 INFO:teuthology.orchestra.run.smithi071.stdout:2159 writesame 0x67ccc2f thru 0x67ce3b1 data_size 0x1(0x1783 bytes) 2024-09-14T05:29:55.958 INFO:teuthology.orchestra.run.smithi071.stdout:2161 read 0x60c43e0 thru 0x60cb0a0 (0x6cc1 bytes) 2024-09-14T05:29:55.958 INFO:teuthology.orchestra.run.smithi071.stdout:2163 write 0x77a51cd thru 0x77b47a1 (0xf5d5 bytes) 2024-09-14T05:29:55.962 INFO:teuthology.orchestra.run.smithi071.stdout:2164 read 0x52e8ffd thru 0x52eb5b1 (0x25b5 bytes) 2024-09-14T05:29:55.962 INFO:teuthology.orchestra.run.smithi071.stdout:2165 write 0x42d1cdf thru 0x42de347 (0xc669 bytes) 2024-09-14T05:29:55.966 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:55.966 INFO:teuthology.orchestra.run.smithi071.stdout:2167 punch from 0x632c324 to 0x633a5d8, (0xe2b4 bytes) 2024-09-14T05:29:55.966 INFO:teuthology.orchestra.run.smithi071.stdout:2168 read 0xd34dc thru 0xe1482 (0xdfa7 bytes) 2024-09-14T05:29:55.972 INFO:teuthology.orchestra.run.smithi071.stdout:2169 write 0x7c8add9 thru 0x7c9913e (0xe366 bytes) 2024-09-14T05:29:55.975 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:55.988 INFO:teuthology.orchestra.run.smithi071.stdout:2172 write 0xcfb32a3 thru 0xcfc2b52 (0xf8b0 bytes) 2024-09-14T05:29:55.994 INFO:teuthology.orchestra.run.smithi071.stdout:2173 read 0x81a6ea9 thru 0x81b5f59 (0xf0b1 bytes) 2024-09-14T05:29:55.997 INFO:teuthology.orchestra.run.smithi071.stdout:2174 trunc from 0xcfc2b53 to 0xab6341c 2024-09-14T05:29:56.008 INFO:teuthology.orchestra.run.smithi071.stdout:2175 read 0xa65e2e2 thru 0xa668364 (0xa083 bytes) 2024-09-14T05:29:56.008 INFO:teuthology.orchestra.run.smithi071.stdout:2176 punch from 0x2ac1af1 to 0x2ac23d1, (0x8e0 bytes) 2024-09-14T05:29:56.008 INFO:teuthology.orchestra.run.smithi071.stdout:2178 read 0xb70f33 thru 0xb78b7d (0x7c4b bytes) 2024-09-14T05:29:56.009 INFO:teuthology.orchestra.run.smithi071.stdout:2179 write 0x369e009 thru 0x36a53a7 (0x739f bytes) 2024-09-14T05:29:56.013 INFO:teuthology.orchestra.run.smithi071.stdout:2181 punch from 0x434ed97 to 0x4353aab, (0x4d14 bytes) 2024-09-14T05:29:56.013 INFO:teuthology.orchestra.run.smithi071.stdout:2182 read 0x85dec29 thru 0x85dfbed (0xfc5 bytes) 2024-09-14T05:29:56.013 INFO:teuthology.orchestra.run.smithi071.stdout:2183 punch from 0x612cd15 to 0x612d530, (0x81b bytes) 2024-09-14T05:29:56.013 INFO:teuthology.orchestra.run.smithi071.stdout:2185 punch from 0x6db2281 to 0x6dc1875, (0xf5f4 bytes) 2024-09-14T05:29:56.013 INFO:teuthology.orchestra.run.smithi071.stdout:2186 read 0xa29cb0f thru 0xa2a5d59 (0x924b bytes) 2024-09-14T05:29:56.013 INFO:teuthology.orchestra.run.smithi071.stdout:2187 writesame 0x328ef1d thru 0x328f61b data_size 0x3(0x6ff bytes) 2024-09-14T05:29:56.015 INFO:teuthology.orchestra.run.smithi071.stdout:2190 writesame 0x6e9e267 thru 0x6ea70c0 data_size 0x26(0x8e5a bytes) 2024-09-14T05:29:56.018 INFO:teuthology.orchestra.run.smithi071.stdout:2191 read 0x46541a4 thru 0x465dd7f (0x9bdc bytes) 2024-09-14T05:29:56.019 INFO:teuthology.orchestra.run.smithi071.stdout:2192 write 0x12a446a thru 0x12a83e6 (0x3f7d bytes) 2024-09-14T05:29:56.020 INFO:teuthology.orchestra.run.smithi071.stdout:2193 trunc from 0xab6341c to 0xa67a611 2024-09-14T05:29:56.025 INFO:teuthology.orchestra.run.smithi071.stdout:2194 write 0x4c8bfec thru 0x4c999fe (0xda13 bytes) 2024-09-14T05:29:56.038 INFO:teuthology.orchestra.run.smithi071.stdout:2195 trunc from 0xa67a611 to 0x2290c50 2024-09-14T05:29:56.055 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:56.055+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:56.069 INFO:teuthology.orchestra.run.smithi071.stdout:2196 write 0xb8cd64f thru 0xb8d36a1 (0x6053 bytes) 2024-09-14T05:29:56.074 INFO:teuthology.orchestra.run.smithi071.stdout:2197 read 0x30c54f2 thru 0x30c5c1b (0x72a bytes) 2024-09-14T05:29:56.074 INFO:teuthology.orchestra.run.smithi071.stdout:2198 read 0x578017c thru 0x57855ad (0x5432 bytes) 2024-09-14T05:29:56.074 INFO:teuthology.orchestra.run.smithi071.stdout:2200 writesame 0x7d4e717 thru 0x7d5ac18 data_size 0x2(0xc502 bytes) 2024-09-14T05:29:56.086 INFO:teuthology.orchestra.run.smithi071.stdout:2201 punch from 0xb5dc491 to 0xb5e3acf, (0x763e bytes) 2024-09-14T05:29:56.086 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.086 INFO:teuthology.orchestra.run.smithi071.stdout:2203 read 0x1ccb418 thru 0x1cd4af4 (0x96dd bytes) 2024-09-14T05:29:56.098 INFO:teuthology.orchestra.run.smithi071.stdout:2204 trunc from 0xb8d36a2 to 0x1d2a90b 2024-09-14T05:29:56.108 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:56.108 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:56.112 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.112 INFO:teuthology.orchestra.run.smithi071.stdout:2206 punch from 0x183ab67 to 0x1840331, (0x57ca bytes) 2024-09-14T05:29:56.117 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:29:56.118 INFO:teuthology.orchestra.run.smithi071.stdout:2210 write 0x3247722 thru 0x3255115 (0xd9f4 bytes) 2024-09-14T05:29:56.128 INFO:teuthology.orchestra.run.smithi071.stdout:2211 punch from 0x317e0b4 to 0x318ae81, (0xcdcd bytes) 2024-09-14T05:29:56.128 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.129 INFO:teuthology.orchestra.run.smithi071.stdout:2213 trunc from 0x3255116 to 0x1bbbd72 2024-09-14T05:29:56.145 INFO:teuthology.orchestra.run.smithi071.stdout:2216 punch from 0x1325be3 to 0x133082c, (0xac49 bytes) 2024-09-14T05:29:56.160 INFO:teuthology.orchestra.run.smithi071.stdout:2217 write 0xd3fa26d thru 0xd3fbb75 (0x1909 bytes) 2024-09-14T05:29:56.161 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:56.161+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:56.163 INFO:teuthology.orchestra.run.smithi071.stdout:2219 read 0x7b98454 thru 0x7ba4db5 (0xc962 bytes) 2024-09-14T05:29:56.163 INFO:teuthology.orchestra.run.smithi071.stdout:2221 write 0x8f538b4 thru 0x8f5750f (0x3c5c bytes) 2024-09-14T05:29:56.166 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.173 INFO:teuthology.orchestra.run.smithi071.stdout:2223 writesame 0xd5f8d09 thru 0xd6028de data_size 0x6d(0x9bd6 bytes) 2024-09-14T05:29:56.173 INFO:teuthology.orchestra.run.smithi071.stdout:2224 read 0xbb4dd6f thru 0xbb4ecdf (0xf71 bytes) 2024-09-14T05:29:56.173 INFO:teuthology.orchestra.run.smithi071.stdout:2225 write 0x7b42848 thru 0x7b47d8e (0x5547 bytes) 2024-09-14T05:29:56.174 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-14T05:29:56.187 INFO:teuthology.orchestra.run.smithi071.stdout:2227 write 0xdf144b0 thru 0xdf21789 (0xd2da bytes) 2024-09-14T05:29:56.192 INFO:teuthology.orchestra.run.smithi071.stdout:2228 trunc from 0xdf2178a to 0xdd1324e 2024-09-14T05:29:56.199 INFO:teuthology.orchestra.run.smithi071.stdout:2230 read 0x44301c1 thru 0x443ed03 (0xeb43 bytes) 2024-09-14T05:29:56.199 INFO:teuthology.orchestra.run.smithi071.stdout:2231 writesame 0x45081bc thru 0x450d279 data_size 0xf(0x50be bytes) 2024-09-14T05:29:56.209 INFO:teuthology.orchestra.run.smithi071.stdout:2232 trunc from 0xdd1324e to 0x808a8c0 2024-09-14T05:29:56.220 INFO:teuthology.orchestra.run.smithi071.stdout:2233 write 0x5a4f511 thru 0x5a5646f (0x6f5f bytes) 2024-09-14T05:29:56.227 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.227 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.228 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:56.228 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:56.233 INFO:teuthology.orchestra.run.smithi071.stdout:2236 trunc from 0x808a8c0 to 0xd4d4e0c 2024-09-14T05:29:56.237 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:29:56.238 INFO:teuthology.orchestra.run.smithi071.stdout:2237 write 0x6a91627 thru 0x6a9a223 (0x8bfd bytes) 2024-09-14T05:29:56.242 INFO:teuthology.orchestra.run.smithi071.stdout:2238 write 0x9674776 thru 0x96836e6 (0xef71 bytes) 2024-09-14T05:29:56.250 INFO:teuthology.orchestra.run.smithi071.stdout:2240 read 0x23bd154 thru 0x23ca13e (0xcfeb bytes) 2024-09-14T05:29:56.251 INFO:teuthology.orchestra.run.smithi071.stdout:2242 read 0x571b1fa thru 0x572134f (0x6156 bytes) 2024-09-14T05:29:56.251 INFO:teuthology.orchestra.run.smithi071.stdout:2243 read 0x57c804d thru 0x57c991d (0x18d1 bytes) 2024-09-14T05:29:56.251 INFO:teuthology.orchestra.run.smithi071.stdout:2245 write 0x6b80673 thru 0x6b8c29f (0xbc2d bytes) 2024-09-14T05:29:56.254 INFO:teuthology.orchestra.run.smithi071.stdout:2246 punch from 0x524d340 to 0x525046a, (0x312a bytes) 2024-09-14T05:29:56.256 INFO:teuthology.orchestra.run.smithi071.stdout:2247 trunc from 0xd4d4e0c to 0xb80e82d 2024-09-14T05:29:56.257 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:56.257+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:56.262 INFO:teuthology.orchestra.run.smithi071.stdout:2250 read 0x7cb78b8 thru 0x7cbab6d (0x32b6 bytes) 2024-09-14T05:29:56.262 INFO:teuthology.orchestra.run.smithi071.stdout:2251 punch from 0x823b522 to 0x824a93f, (0xf41d bytes) 2024-09-14T05:29:56.262 INFO:teuthology.orchestra.run.smithi071.stdout:2252 write 0xaba6c6a thru 0xabb08db (0x9c72 bytes) 2024-09-14T05:29:56.267 INFO:teuthology.orchestra.run.smithi071.stdout:2254 read 0x349c3a8 thru 0x349cbb2 (0x80b bytes) 2024-09-14T05:29:56.268 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.268 INFO:teuthology.orchestra.run.smithi071.stdout:2257 read 0x676525f thru 0x676f2e5 (0xa087 bytes) 2024-09-14T05:29:56.268 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.268 INFO:teuthology.orchestra.run.smithi071.stdout:2259 write 0xabc4794 thru 0xabca59d (0x5e0a bytes) 2024-09-14T05:29:56.273 INFO:teuthology.orchestra.run.smithi071.stdout:2261 trunc from 0xb80e82d to 0xbca1571 2024-09-14T05:29:56.278 INFO:teuthology.orchestra.run.smithi071.stdout:2262 punch from 0x546ae62 to 0x546bdd5, (0xf73 bytes) 2024-09-14T05:29:56.278 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.278 INFO:teuthology.orchestra.run.smithi071.stdout:2265 write 0x2c6a126 thru 0x2c70abb (0x6996 bytes) 2024-09-14T05:29:56.287 INFO:teuthology.orchestra.run.smithi071.stdout:2266 writesame 0xe774970 thru 0xe776bc4 data_size 0x11(0x2255 bytes) 2024-09-14T05:29:56.287 INFO:teuthology.orchestra.run.smithi071.stdout:2267 write 0xe4e736b thru 0xe4e89df (0x1675 bytes) 2024-09-14T05:29:56.291 INFO:teuthology.orchestra.run.smithi071.stdout:2268 punch from 0x1d4016f to 0x1d451f2, (0x5083 bytes) 2024-09-14T05:29:56.291 INFO:teuthology.orchestra.run.smithi071.stdout:2270 write 0x4791e4f thru 0x479be55 (0xa007 bytes) 2024-09-14T05:29:56.294 INFO:teuthology.orchestra.run.smithi071.stdout:2271 punch from 0x8aae8b5 to 0x8abb2f3, (0xca3e bytes) 2024-09-14T05:29:56.294 INFO:teuthology.orchestra.run.smithi071.stdout:2272 writesame 0xd2b5a57 thru 0xd2bbc5d data_size 0x23(0x6207 bytes) 2024-09-14T05:29:56.300 INFO:teuthology.orchestra.run.smithi071.stdout:2273 writesame 0xc8cb47a thru 0xc8d021b data_size 0x13(0x4da2 bytes) 2024-09-14T05:29:56.304 INFO:teuthology.orchestra.run.smithi071.stdout:2274 punch from 0x98b380e to 0x98c06ba, (0xceac bytes) 2024-09-14T05:29:56.307 INFO:teuthology.orchestra.run.smithi071.stdout:2276 trunc from 0xe776bc5 to 0xc4ba3c3 2024-09-14T05:29:56.313 INFO:teuthology.orchestra.run.smithi071.stdout:2277 writesame 0x100b500 thru 0x100fef0 data_size 0x1(0x49f1 bytes) 2024-09-14T05:29:56.322 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.322 INFO:teuthology.orchestra.run.smithi071.stdout:2279 read 0x6a314dd thru 0x6a3ca86 (0xb5aa bytes) 2024-09-14T05:29:56.323 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.323 INFO:teuthology.orchestra.run.smithi071.stdout:2281 punch from 0xb928366 to 0xb938196, (0xfe30 bytes) 2024-09-14T05:29:56.323 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.323 INFO:teuthology.orchestra.run.smithi071.stdout:2284 writesame 0x16e3b34 thru 0x16ec72f data_size 0x44(0x8bfc bytes) 2024-09-14T05:29:56.324 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.324 INFO:teuthology.orchestra.run.smithi071.stdout:2287 punch from 0x4cb09bf to 0x4cbe635, (0xdc76 bytes) 2024-09-14T05:29:56.330 INFO:teuthology.orchestra.run.smithi071.stdout:2288 write 0xcdde42c thru 0xcde3317 (0x4eec bytes) 2024-09-14T05:29:56.333 INFO:teuthology.orchestra.run.smithi071.stdout:2289 punch from 0xa4c0184 to 0xa4c5aa3, (0x591f bytes) 2024-09-14T05:29:56.333 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.333 INFO:teuthology.orchestra.run.smithi071.stdout:2291 read 0x931fee3 thru 0x9325150 (0x526e bytes) 2024-09-14T05:29:56.333 INFO:teuthology.orchestra.run.smithi071.stdout:2293 read 0xa07e24 thru 0xa1673d (0xe91a bytes) 2024-09-14T05:29:56.334 INFO:teuthology.orchestra.run.smithi071.stdout:2294 write 0x6e3fd81 thru 0x6e4e116 (0xe396 bytes) 2024-09-14T05:29:56.341 INFO:teuthology.orchestra.run.smithi071.stdout:2295 write 0x53c8514 thru 0x53d6022 (0xdb0f bytes) 2024-09-14T05:29:56.344 INFO:teuthology.orchestra.run.smithi071.stdout:2297 write 0x14ce2d4 thru 0x14d747b (0x91a8 bytes) 2024-09-14T05:29:56.345 INFO:teuthology.orchestra.run.smithi071.stdout:2298 writesame 0xb3b889c thru 0xb3b9558 data_size 0x1(0xcbd bytes) 2024-09-14T05:29:56.359 INFO:teuthology.orchestra.run.smithi071.stdout:2299 trunc from 0xcde3318 to 0x32d000d 2024-09-14T05:29:56.359 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:56.358+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:56.389 INFO:teuthology.orchestra.run.smithi071.stdout:2300 writesame 0xce7bf88 thru 0xce8689e data_size 0x2f(0xa917 bytes) 2024-09-14T05:29:56.390 INFO:teuthology.orchestra.run.smithi071.stdout:2302 read 0x84a9389 thru 0x84b044c (0x70c4 bytes) 2024-09-14T05:29:56.390 INFO:teuthology.orchestra.run.smithi071.stdout:2305 read 0x3cbdb7b thru 0x3cc6ab7 (0x8f3d bytes) 2024-09-14T05:29:56.390 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.390 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:56.390 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:56.398 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:29:56.399 INFO:teuthology.orchestra.run.smithi071.stdout:2307 trunc from 0xce8689f to 0x4fdfab0 2024-09-14T05:29:56.405 INFO:teuthology.orchestra.run.smithi071.stdout:2309 punch from 0x1421bc4 to 0x142342d, (0x1869 bytes) 2024-09-14T05:29:56.406 INFO:teuthology.orchestra.run.smithi071.stdout:2310 punch from 0x4ce772 to 0x4d6e31, (0x86bf bytes) 2024-09-14T05:29:56.406 INFO:teuthology.orchestra.run.smithi071.stdout:2311 read 0xcf9a1d thru 0xcfb4db (0x1abf bytes) 2024-09-14T05:29:56.406 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.406 INFO:teuthology.orchestra.run.smithi071.stdout:2313 punch from 0x45c7245 to 0x45d206a, (0xae25 bytes) 2024-09-14T05:29:56.406 INFO:teuthology.orchestra.run.smithi071.stdout:2314 read 0x14eedb1 thru 0x14fa5af (0xb7ff bytes) 2024-09-14T05:29:56.418 INFO:teuthology.orchestra.run.smithi071.stdout:2315 write 0xac133f6 thru 0xac164c9 (0x30d4 bytes) 2024-09-14T05:29:56.421 INFO:teuthology.orchestra.run.smithi071.stdout:2316 read 0x8bcbc5a thru 0x8bd734c (0xb6f3 bytes) 2024-09-14T05:29:56.421 INFO:teuthology.orchestra.run.smithi071.stdout:2317 punch from 0x4d58a1 to 0x4e2ba1, (0xd300 bytes) 2024-09-14T05:29:56.421 INFO:teuthology.orchestra.run.smithi071.stdout:2318 read 0x1c47f60 thru 0x1c553c1 (0xd462 bytes) 2024-09-14T05:29:56.422 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.422 INFO:teuthology.orchestra.run.smithi071.stdout:2322 write 0xa0d803f thru 0xa0e0482 (0x8444 bytes) 2024-09-14T05:29:56.429 INFO:teuthology.orchestra.run.smithi071.stdout:2323 write 0xac34c4c thru 0xac38a6a (0x3e1f bytes) 2024-09-14T05:29:56.432 INFO:teuthology.orchestra.run.smithi071.stdout:2324 write 0x882c32c thru 0x8836134 (0x9e09 bytes) 2024-09-14T05:29:56.436 INFO:teuthology.orchestra.run.smithi071.stdout:2325 write 0xa54f209 thru 0xa557200 (0x7ff8 bytes) 2024-09-14T05:29:56.443 INFO:teuthology.orchestra.run.smithi071.stdout:2326 trunc from 0xac38a6b to 0xdff77d8 2024-09-14T05:29:56.448 INFO:teuthology.orchestra.run.smithi071.stdout:2327 read 0x74f8878 thru 0x7507ffe (0xf787 bytes) 2024-09-14T05:29:56.448 INFO:teuthology.orchestra.run.smithi071.stdout:2329 writesame 0x50c5b5e thru 0x50d17d8 data_size 0x7(0xbc7b bytes) 2024-09-14T05:29:56.451 INFO:teuthology.orchestra.run.smithi071.stdout:2330 write 0x435f0e3 thru 0x4363127 (0x4045 bytes) 2024-09-14T05:29:56.455 INFO:teuthology.orchestra.run.smithi071.stdout:2331 punch from 0xc428de3 to 0xc42e393, (0x55b0 bytes) 2024-09-14T05:29:56.455 INFO:teuthology.orchestra.run.smithi071.stdout:2333 punch from 0x910d21b to 0x911c11e, (0xef03 bytes) 2024-09-14T05:29:56.455 INFO:teuthology.orchestra.run.smithi071.stdout:2335 write 0x11bd55a thru 0x11cd23e (0xfce5 bytes) 2024-09-14T05:29:56.456 INFO:teuthology.orchestra.run.smithi071.stdout:2336 read 0x6bac5eb thru 0x6badde3 (0x17f9 bytes) 2024-09-14T05:29:56.456 INFO:teuthology.orchestra.run.smithi071.stdout:2337 write 0x30d658f thru 0x30d7665 (0x10d7 bytes) 2024-09-14T05:29:56.459 INFO:teuthology.orchestra.run.smithi071.stdout:2338 read 0xa434858 thru 0xa439043 (0x47ec bytes) 2024-09-14T05:29:56.459 INFO:teuthology.orchestra.run.smithi071.stdout:2339 write 0x87a2384 thru 0x87ae2ec (0xbf69 bytes) 2024-09-14T05:29:56.463 INFO:teuthology.orchestra.run.smithi071.stdout:2340 write 0xc618059 thru 0xc618650 (0x5f8 bytes) 2024-09-14T05:29:56.466 INFO:teuthology.orchestra.run.smithi071.stdout:2342 write 0xafce12b thru 0xafda6eb (0xc5c1 bytes) 2024-09-14T05:29:56.467 INFO:teuthology.orchestra.run.smithi071.stdout:2344 writesame 0x2d0854a thru 0x2d1534f data_size 0x1(0xce06 bytes) 2024-09-14T05:29:56.483 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:56.482+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:56.489 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.490 INFO:teuthology.orchestra.run.smithi071.stdout:2346 write 0xc7f47b6 thru 0xc7f9b95 (0x53e0 bytes) 2024-09-14T05:29:56.493 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.493 INFO:teuthology.orchestra.run.smithi071.stdout:2348 write 0xa8dbc1f thru 0xa8dda09 (0x1deb bytes) 2024-09-14T05:29:56.494 INFO:teuthology.orchestra.run.smithi071.stdout:2349 write 0x2d1b737 thru 0x2d20b8d (0x5457 bytes) 2024-09-14T05:29:56.496 INFO:teuthology.orchestra.run.smithi071.stdout:2351 trunc from 0xdff77d8 to 0xe64c5d5 2024-09-14T05:29:56.500 INFO:teuthology.orchestra.run.smithi071.stdout:2352 punch from 0x3d00829 to 0x3d06f49, (0x6720 bytes) 2024-09-14T05:29:56.501 INFO:teuthology.orchestra.run.smithi071.stdout:2354 trunc from 0xe64c5d5 to 0xdba6234 2024-09-14T05:29:56.507 INFO:teuthology.orchestra.run.smithi071.stdout:2355 write 0xb8eeaef thru 0xb8f45ef (0x5b01 bytes) 2024-09-14T05:29:56.509 INFO:teuthology.orchestra.run.smithi071.stdout:2360 read 0xc87326e thru 0xc882429 (0xf1bc bytes) 2024-09-14T05:29:56.510 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:56.517 INFO:teuthology.orchestra.run.smithi071.stdout:2362 trunc from 0xdba6234 to 0x79b37fd 2024-09-14T05:29:56.544 INFO:teuthology.orchestra.run.smithi071.stdout:2363 writesame 0xc4ff6c5 thru 0xc501cd8 data_size 0x2(0x2614 bytes) 2024-09-14T05:29:56.545 INFO:teuthology.orchestra.run.smithi071.stdout:2365 writesame 0x633be78 thru 0x633d300 data_size 0x1(0x1489 bytes) 2024-09-14T05:29:56.549 INFO:teuthology.orchestra.run.smithi071.stdout:2366 writesame 0x2690eba thru 0x2697729 data_size 0x18(0x6870 bytes) 2024-09-14T05:29:56.552 INFO:teuthology.orchestra.run.smithi071.stdout:2367 write 0x2ef3e5a thru 0x2ef7d9a (0x3f41 bytes) 2024-09-14T05:29:56.553 INFO:teuthology.orchestra.run.smithi071.stdout:2369 trunc from 0xc501cd9 to 0xbf0b5ad 2024-09-14T05:29:56.556 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:56.556 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:56.557 INFO:teuthology.orchestra.run.smithi071.stdout:2370 read 0x46b17d4 thru 0x46b39d8 (0x2205 bytes) 2024-09-14T05:29:56.560 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:56.559+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:56.562 INFO:teuthology.orchestra.run.smithi071.stdout:2371 trunc from 0xbf0b5ad to 0x8c09dc0 2024-09-14T05:29:56.565 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:29:56.572 INFO:teuthology.orchestra.run.smithi071.stdout:2373 writesame 0x1654335 thru 0x1655147 data_size 0x1(0xe13 bytes) 2024-09-14T05:29:56.575 INFO:teuthology.orchestra.run.smithi071.stdout:2375 trunc from 0x8c09dc0 to 0x89adfc5 2024-09-14T05:29:56.581 INFO:teuthology.orchestra.run.smithi071.stdout:2376 write 0x14bb02f thru 0x14c378e (0x8760 bytes) 2024-09-14T05:29:56.582 INFO:teuthology.orchestra.run.smithi071.stdout:2377 clone 21 order 25 su 8388608 sc 8 2024-09-14T05:29:56.661 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:56.660+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:56.669 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:56.670 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:56.678 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:29:56.763 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:56.762+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:56.821 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:56.821 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:56.830 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:29:56.871 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:56.870+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:56.963 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:56.962+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:56.972 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:56.973 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:56.981 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:29:57.065 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:57.064+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:57.125 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:57.126 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:57.134 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:29:57.165 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:57.165+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:57.247 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:57.248 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:57.250 INFO:teuthology.orchestra.run.smithi071.stdout:leaving image image_client.0-clone20 intact 2024-09-14T05:29:57.257 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:29:57.301 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:57.300+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:57.358 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-14T05:29:57.366 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:57.366+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:57.381 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:57.381 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:57.393 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:29:57.478 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:57.477+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:57.543 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:57.543 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:57.552 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:29:57.569 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:57.568+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:57.604 INFO:teuthology.orchestra.run.smithi071.stdout:2378 read 0x288183 thru 0x28ca8c (0x490a bytes) 2024-09-14T05:29:57.605 INFO:teuthology.orchestra.run.smithi071.stdout:2379 read 0x54bb7eb thru 0x54c834b (0xcb61 bytes) 2024-09-14T05:29:57.606 INFO:teuthology.orchestra.run.smithi071.stdout:2380 punch from 0x5dff7c to 0x5ed1e0, (0xd264 bytes) 2024-09-14T05:29:57.616 INFO:teuthology.orchestra.run.smithi071.stdout:2381 trunc from 0x89adfc5 to 0x31062e9 2024-09-14T05:29:57.662 INFO:teuthology.orchestra.run.smithi071.stdout:2382 punch from 0x23611e2 to 0x23704a9, (0xf2c7 bytes) 2024-09-14T05:29:57.662 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:57.662 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:57.662 INFO:teuthology.orchestra.run.smithi071.stdout:2386 writesame 0xa73838 thru 0xa77968 data_size 0x1(0x4131 bytes) 2024-09-14T05:29:57.669 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:57.668+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:57.671 INFO:teuthology.orchestra.run.smithi071.stdout:2387 punch from 0x2076397 to 0x2082140, (0xbda9 bytes) 2024-09-14T05:29:57.680 INFO:teuthology.orchestra.run.smithi071.stdout:2388 write 0x68d4fd9 thru 0x68dded1 (0x8ef9 bytes) 2024-09-14T05:29:57.687 INFO:teuthology.orchestra.run.smithi071.stdout:2389 trunc from 0x68dded2 to 0x2a7f12f 2024-09-14T05:29:57.696 INFO:teuthology.orchestra.run.smithi071.stdout:2390 punch from 0x1a50610 to 0x1a57b5c, (0x754c bytes) 2024-09-14T05:29:57.696 INFO:teuthology.orchestra.run.smithi071.stdout:2391 read 0x2503742 thru 0x250a4f9 (0x6db8 bytes) 2024-09-14T05:29:57.696 INFO:teuthology.orchestra.run.smithi071.stdout:2392 punch from 0x1188ead to 0x119581a, (0xc96d bytes) 2024-09-14T05:29:57.697 INFO:teuthology.orchestra.run.smithi071.stdout:2394 read 0x1152d13 thru 0x1156757 (0x3a45 bytes) 2024-09-14T05:29:57.697 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:29:57.699 INFO:teuthology.orchestra.run.smithi071.stdout:2396 trunc from 0x2a7f12f to 0xd6192e 2024-09-14T05:29:57.715 INFO:teuthology.orchestra.run.smithi071.stdout:2397 trunc from 0xd6192e to 0x1ee12f0 2024-09-14T05:29:57.715 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:57.715 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:57.725 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:29:57.726 INFO:teuthology.orchestra.run.smithi071.stdout:2398 write 0x1ee164f thru 0x1eee7c7 (0xd179 bytes) 2024-09-14T05:29:57.744 INFO:teuthology.orchestra.run.smithi071.stdout:2399 write 0xa2ccef4 thru 0xa2d857f (0xb68c bytes) 2024-09-14T05:29:57.746 INFO:teuthology.orchestra.run.smithi071.stdout:2400 write 0x88ccd42 thru 0x88d04ac (0x376b bytes) 2024-09-14T05:29:57.749 INFO:teuthology.orchestra.run.smithi071.stdout:2402 trunc from 0xa2d8580 to 0xafb2245 2024-09-14T05:29:57.754 INFO:teuthology.orchestra.run.smithi071.stdout:2403 read 0x81ac425 thru 0x81b33b3 (0x6f8f bytes) 2024-09-14T05:29:57.754 INFO:teuthology.orchestra.run.smithi071.stdout:2404 write 0x8b2b60f thru 0x8b363ad (0xad9f bytes) 2024-09-14T05:29:57.756 INFO:teuthology.orchestra.run.smithi071.stdout:2406 punch from 0x3aec1ec to 0x3afb042, (0xee56 bytes) 2024-09-14T05:29:57.756 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:57.756 INFO:teuthology.orchestra.run.smithi071.stdout:2408 write 0x77e3699 thru 0x77f3015 (0xf97d bytes) 2024-09-14T05:29:57.759 INFO:teuthology.orchestra.run.smithi071.stdout:2410 write 0x9e75906 thru 0x9e84b2d (0xf228 bytes) 2024-09-14T05:29:57.761 INFO:teuthology.orchestra.run.smithi071.stdout:2412 writesame 0x374fdee thru 0x375cf01 data_size 0x2(0xd114 bytes) 2024-09-14T05:29:57.769 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:57.768+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:57.773 INFO:teuthology.orchestra.run.smithi071.stdout:2413 read 0x7e4de01 thru 0x7e52a1e (0x4c1e bytes) 2024-09-14T05:29:57.773 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:57.773 INFO:teuthology.orchestra.run.smithi071.stdout:2417 write 0x12053fd thru 0x120d255 (0x7e59 bytes) 2024-09-14T05:29:57.776 INFO:teuthology.orchestra.run.smithi071.stdout:2419 write 0x41f0527 thru 0x41f9c19 (0x96f3 bytes) 2024-09-14T05:29:57.783 INFO:teuthology.orchestra.run.smithi071.stdout:2420 write 0xba8ac33 thru 0xba98dcd (0xe19b bytes) 2024-09-14T05:29:57.787 INFO:teuthology.orchestra.run.smithi071.stdout:2421 punch from 0xac55238 to 0xac5ec7f, (0x9a47 bytes) 2024-09-14T05:29:57.787 INFO:teuthology.orchestra.run.smithi071.stdout:2423 punch from 0x2229c08 to 0x222fede, (0x62d6 bytes) 2024-09-14T05:29:57.787 INFO:teuthology.orchestra.run.smithi071.stdout:2424 read 0x3a8e3af thru 0x3a99022 (0xac74 bytes) 2024-09-14T05:29:57.799 INFO:teuthology.orchestra.run.smithi071.stdout:2425 trunc from 0xba98dce to 0x1888a2b 2024-09-14T05:29:57.830 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:29:57.831 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:29:57.834 INFO:teuthology.orchestra.run.smithi071.stdout:2426 writesame 0x90f4de5 thru 0x9100ce4 data_size 0x80(0xbf00 bytes) 2024-09-14T05:29:57.835 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:57.844 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:29:57.844 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:29:57.845 INFO:teuthology.orchestra.run.smithi071.stdout:2428 write 0xbb0c384 thru 0xbb1adee (0xea6b bytes) 2024-09-14T05:29:57.847 INFO:teuthology.orchestra.run.smithi071.stdout:2429 write 0x6761ca3 thru 0x67708d1 (0xec2f bytes) 2024-09-14T05:29:57.849 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:29:57.849 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:29:57.851 INFO:teuthology.orchestra.run.smithi071.stdout:2430 write 0x4eb8c81 thru 0x4eb9433 (0x7b3 bytes) 2024-09-14T05:29:57.852 INFO:teuthology.orchestra.run.smithi071.stdout:2431 write 0x6b6838a thru 0x6b6e425 (0x609c bytes) 2024-09-14T05:29:57.854 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:29:57.855 INFO:teuthology.orchestra.run.smithi071.stdout:2432 punch from 0x53d707f to 0x53dda67, (0x69e8 bytes) 2024-09-14T05:29:57.856 INFO:teuthology.orchestra.run.smithi071.stdout:2434 read 0xab538fd thru 0xab5b9ae (0x80b2 bytes) 2024-09-14T05:29:57.856 INFO:teuthology.orchestra.run.smithi071.stdout:2435 write 0x7b2e846 thru 0x7b350c5 (0x6880 bytes) 2024-09-14T05:29:57.857 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:57.857 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:57.858 INFO:teuthology.orchestra.run.smithi071.stdout:2438 write 0x9ba716a thru 0x9bb1c0e (0xaaa5 bytes) 2024-09-14T05:29:57.858 INFO:teuthology.orchestra.run.smithi071.stdout:2439 read 0x3986c0f thru 0x398cc11 (0x6003 bytes) 2024-09-14T05:29:57.859 INFO:teuthology.orchestra.run.smithi071.stdout:2441 read 0x904b7da thru 0x9059f38 (0xe75f bytes) 2024-09-14T05:29:57.859 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:57.859 INFO:teuthology.orchestra.run.smithi071.stdout:truncating to largest ever: 0xee2e54a 2024-09-14T05:29:57.863 INFO:teuthology.orchestra.run.smithi071.stdout:2443 trunc from 0xbb1adef to 0xee2e54a 2024-09-14T05:29:57.867 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:29:57.867 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:29:57.868 INFO:teuthology.orchestra.run.smithi071.stdout:2445 write 0x1d1473 thru 0x1d41b8 (0x2d46 bytes) 2024-09-14T05:29:57.869 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:57.869+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:57.870 INFO:teuthology.orchestra.run.smithi071.stdout:2447 write 0x9deb3bc thru 0x9dee4a3 (0x30e8 bytes) 2024-09-14T05:29:57.871 INFO:teuthology.orchestra.run.smithi071.stdout:2448 writesame 0xb789293 thru 0xb794549 data_size 0x1(0xb2b7 bytes) 2024-09-14T05:29:57.885 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:29:57.885 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:29:57.890 INFO:teuthology.orchestra.run.smithi071.stdout:2449 write 0xd4efb91 thru 0xd4f48b7 (0x4d27 bytes) 2024-09-14T05:29:57.892 INFO:teuthology.orchestra.run.smithi071.stdout:2450 writesame 0x6c0850c thru 0x6c0e5c3 data_size 0xa(0x60b8 bytes) 2024-09-14T05:29:57.894 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:57.894 INFO:teuthology.orchestra.run.smithi071.stdout:2452 read 0x6a0e7af thru 0x6a17b08 (0x935a bytes) 2024-09-14T05:29:57.895 INFO:teuthology.orchestra.run.smithi071.stdout:2453 read 0x588e8d1 thru 0x589436e (0x5a9e bytes) 2024-09-14T05:29:57.895 INFO:teuthology.orchestra.run.smithi071.stdout:2454 writesame 0x6bcaa32 thru 0x6bceee3 data_size 0x9(0x44b2 bytes) 2024-09-14T05:29:57.897 INFO:teuthology.orchestra.run.smithi071.stdout:2455 read 0x7320823 thru 0x7329758 (0x8f36 bytes) 2024-09-14T05:29:57.897 INFO:teuthology.orchestra.run.smithi071.stdout:2456 write 0xca50bd3 thru 0xca5506f (0x449d bytes) 2024-09-14T05:29:57.899 INFO:teuthology.orchestra.run.smithi071.stdout:2457 write 0x280c34b thru 0x281769d (0xb353 bytes) 2024-09-14T05:29:57.901 INFO:teuthology.orchestra.run.smithi071.stdout:2458 write 0x74f10b2 thru 0x74f911a (0x8069 bytes) 2024-09-14T05:29:57.902 INFO:teuthology.orchestra.run.smithi071.stdout:2459 writesame 0x31a2691 thru 0x31b0c92 data_size 0x3b(0xe602 bytes) 2024-09-14T05:29:57.903 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:29:57.903 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:29:57.904 INFO:teuthology.orchestra.run.smithi071.stdout:2460 read 0xba35f94 thru 0xba3cea5 (0x6f12 bytes) 2024-09-14T05:29:57.904 INFO:teuthology.orchestra.run.smithi071.stdout:2461 punch from 0x711f770 to 0x7124d28, (0x55b8 bytes) 2024-09-14T05:29:57.911 INFO:teuthology.orchestra.run.smithi071.stdout:2463 trunc from 0xee2e54a to 0x96b796c 2024-09-14T05:29:57.919 INFO:teuthology.orchestra.run.smithi071.stdout:2465 read 0x8201e36 thru 0x8210b65 (0xed30 bytes) 2024-09-14T05:29:57.921 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:29:57.922 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:29:57.928 INFO:teuthology.orchestra.run.smithi071.stdout:2466 write 0xbe5c8d8 thru 0xbe6a414 (0xdb3d bytes) 2024-09-14T05:29:57.930 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:57.930 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:57.930 INFO:teuthology.orchestra.run.smithi071.stdout:2470 read 0x6ed5420 thru 0x6ee1ccc (0xc8ad bytes) 2024-09-14T05:29:57.931 INFO:teuthology.orchestra.run.smithi071.stdout:2471 punch from 0x13d75d5 to 0x13e6a7b, (0xf4a6 bytes) 2024-09-14T05:29:57.931 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:57.931 INFO:teuthology.orchestra.run.smithi071.stdout:2473 read 0x9155ce6 thru 0x9159ad4 (0x3def bytes) 2024-09-14T05:29:57.931 INFO:teuthology.orchestra.run.smithi071.stdout:2474 read 0xb4bdba4 thru 0xb4c0003 (0x2460 bytes) 2024-09-14T05:29:57.931 INFO:teuthology.orchestra.run.smithi071.stdout:2475 writesame 0x82efa2 thru 0x83ea18 data_size 0x21(0xfa77 bytes) 2024-09-14T05:29:57.934 INFO:teuthology.orchestra.run.smithi071.stdout:2477 writesame 0x232a780 thru 0x23330f0 data_size 0x1f(0x8971 bytes) 2024-09-14T05:29:57.936 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:57.936 INFO:teuthology.orchestra.run.smithi071.stdout:2479 punch from 0xb0923c8 to 0xb09a27b, (0x7eb3 bytes) 2024-09-14T05:29:57.936 INFO:teuthology.orchestra.run.smithi071.stdout:2480 read 0x2d3aa17 thru 0x2d48504 (0xdaee bytes) 2024-09-14T05:29:57.936 INFO:teuthology.orchestra.run.smithi071.stdout:2482 read 0x7656852 thru 0x765d425 (0x6bd4 bytes) 2024-09-14T05:29:57.940 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:29:57.941 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:29:57.953 INFO:teuthology.orchestra.run.smithi071.stdout:2483 trunc from 0xbe6a415 to 0x15d246 2024-09-14T05:29:57.969 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:29:57.969 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:29:57.970 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:57.969+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:57.973 INFO:teuthology.orchestra.run.smithi071.stdout:2484 read 0x8dfe9 thru 0x930cc (0x50e4 bytes) 2024-09-14T05:29:57.974 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:57.974 INFO:teuthology.orchestra.run.smithi071.stdout:2486 punch from 0xbc8ee to 0xc3cbb, (0x73cd bytes) 2024-09-14T05:29:57.986 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:57.986 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:57.986 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:29:57.986 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:57.996 INFO:teuthology.orchestra.run.smithi071.stdout:2487 writesame 0xd7b8ccf thru 0xd7c02dc data_size 0x49(0x760e bytes) 2024-09-14T05:29:57.996 INFO:teuthology.orchestra.run.smithi071.stdout:2489 read 0xb1d4409 thru 0xb1e0437 (0xc02f bytes) 2024-09-14T05:29:57.997 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:29:58.001 INFO:teuthology.orchestra.run.smithi071.stdout:2490 write 0xe2dd98f thru 0xe2e8bed (0xb25f bytes) 2024-09-14T05:29:58.006 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.006 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.006 INFO:teuthology.orchestra.run.smithi071.stdout:2493 write 0x6d64ae5 thru 0x6d73055 (0xe571 bytes) 2024-09-14T05:29:58.009 INFO:teuthology.orchestra.run.smithi071.stdout:2494 read 0x63fac6b thru 0x6405d9a (0xb130 bytes) 2024-09-14T05:29:58.009 INFO:teuthology.orchestra.run.smithi071.stdout:2495 read 0xc8c1afc thru 0xc8cb104 (0x9609 bytes) 2024-09-14T05:29:58.019 INFO:teuthology.orchestra.run.smithi071.stdout:2497 trunc from 0xe2e8bee to 0x68ccfe0 2024-09-14T05:29:58.025 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.025 INFO:teuthology.orchestra.run.smithi071.stdout:2499 write 0x5cf2a1 thru 0x5da499 (0xb1f9 bytes) 2024-09-14T05:29:58.027 INFO:teuthology.orchestra.run.smithi071.stdout:2500 read 0x1f39ddf thru 0x1f46756 (0xc978 bytes) 2024-09-14T05:29:58.027 INFO:teuthology.orchestra.run.smithi071.stdout:2501 read 0x33386ac thru 0x333ee13 (0x6768 bytes) 2024-09-14T05:29:58.028 INFO:teuthology.orchestra.run.smithi071.stdout:2503 trunc from 0x68ccfe0 to 0x5d94334 2024-09-14T05:29:58.033 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.045 INFO:teuthology.orchestra.run.smithi071.stdout:2505 write 0xaf6eb74 thru 0xaf706c5 (0x1b52 bytes) 2024-09-14T05:29:58.046 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.046 INFO:teuthology.orchestra.run.smithi071.stdout:2508 read 0x9253874 thru 0x9253df0 (0x57d bytes) 2024-09-14T05:29:58.046 INFO:teuthology.orchestra.run.smithi071.stdout:2511 writesame 0x796e59f thru 0x797ca39 data_size 0x1(0xe49b bytes) 2024-09-14T05:29:58.071 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:29:58.070+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:29:58.075 INFO:teuthology.orchestra.run.smithi071.stdout:2512 write 0x1e9bd73 thru 0x1e9ef00 (0x318e bytes) 2024-09-14T05:29:58.084 INFO:teuthology.orchestra.run.smithi071.stdout:2513 write 0xbd95689 thru 0xbda0a36 (0xb3ae bytes) 2024-09-14T05:29:58.085 INFO:teuthology.orchestra.run.smithi071.stdout:2514 write 0x5578354 thru 0x557b5d5 (0x3282 bytes) 2024-09-14T05:29:58.088 INFO:teuthology.orchestra.run.smithi071.stdout:2516 read 0x52487c1 thru 0x5253cfc (0xb53c bytes) 2024-09-14T05:29:58.089 INFO:teuthology.orchestra.run.smithi071.stdout:2517 write 0x4ba4e3 thru 0x4bd2e7 (0x2e05 bytes) 2024-09-14T05:29:58.090 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.103 INFO:teuthology.orchestra.run.smithi071.stdout:2519 trunc from 0xbda0a37 to 0x11f0ccb 2024-09-14T05:29:58.116 INFO:teuthology.orchestra.run.smithi071.stdout:2521 punch from 0x12792d to 0x12fb3f, (0x8212 bytes) 2024-09-14T05:29:58.116 INFO:teuthology.orchestra.run.smithi071.stdout:2522 read 0x3ecca thru 0x44ac1 (0x5df8 bytes) 2024-09-14T05:29:58.119 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:58.119 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:29:58.119 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:29:58.119 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:29:58.119 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:58.140 INFO:teuthology.orchestra.run.smithi071.stdout:2523 write 0xdefd199 thru 0xdf08893 (0xb6fb bytes) 2024-09-14T05:29:58.142 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:29:58.143 INFO:teuthology.orchestra.run.smithi071.stdout:2525 write 0xbcf9371 thru 0xbcfc2ed (0x2f7d bytes) 2024-09-14T05:29:58.146 INFO:teuthology.orchestra.run.smithi071.stdout:2526 read 0x71ef625 thru 0x71fdeb2 (0xe88e bytes) 2024-09-14T05:29:58.146 INFO:teuthology.orchestra.run.smithi071.stdout:2527 read 0x2bb657d thru 0x2bbe917 (0x839b bytes) 2024-09-14T05:29:58.146 INFO:teuthology.orchestra.run.smithi071.stdout:2528 write 0xaf694ed thru 0xaf7816e (0xec82 bytes) 2024-09-14T05:29:58.156 INFO:teuthology.orchestra.run.smithi071.stdout:2529 writesame 0xedb37d6 thru 0xedc073d data_size 0x1(0xcf68 bytes) 2024-09-14T05:29:58.157 INFO:teuthology.orchestra.run.smithi071.stdout:2530 punch from 0x527b042 to 0x5285eb3, (0xae71 bytes) 2024-09-14T05:29:58.157 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.161 INFO:teuthology.orchestra.run.smithi071.stdout:2536 trunc from 0xedc073e to 0xb05d5de 2024-09-14T05:29:58.169 INFO:teuthology.orchestra.run.smithi071.stdout:2537 punch from 0x498ca88 to 0x4998442, (0xb9ba bytes) 2024-09-14T05:29:58.169 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.169 INFO:teuthology.orchestra.run.smithi071.stdout:2539 write 0x3325d91 thru 0x3332573 (0xc7e3 bytes) 2024-09-14T05:29:58.171 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:58.170+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:58.172 INFO:teuthology.orchestra.run.smithi071.stdout:2541 punch from 0x7a9d082 to 0x7aa99d1, (0xc94f bytes) 2024-09-14T05:29:58.172 INFO:teuthology.orchestra.run.smithi071.stdout:2544 read 0x5dd6715 thru 0x5ddc2b3 (0x5b9f bytes) 2024-09-14T05:29:58.172 INFO:teuthology.orchestra.run.smithi071.stdout:2547 write 0xa823987 thru 0xa82c88e (0x8f08 bytes) 2024-09-14T05:29:58.173 INFO:teuthology.orchestra.run.smithi071.stdout:2548 punch from 0x63db710 to 0x63e04c3, (0x4db3 bytes) 2024-09-14T05:29:58.174 INFO:teuthology.orchestra.run.smithi071.stdout:2549 writesame 0xa533532 thru 0xa53de4b data_size 0x2d(0xa91a bytes) 2024-09-14T05:29:58.176 INFO:teuthology.orchestra.run.smithi071.stdout:2552 write 0x76e56 thru 0x863ee (0xf599 bytes) 2024-09-14T05:29:58.182 INFO:teuthology.orchestra.run.smithi071.stdout:2553 read 0x75b04ef thru 0x75bf483 (0xef95 bytes) 2024-09-14T05:29:58.182 INFO:teuthology.orchestra.run.smithi071.stdout:2554 read 0x7092f64 thru 0x70932b2 (0x34f bytes) 2024-09-14T05:29:58.182 INFO:teuthology.orchestra.run.smithi071.stdout:2555 write 0x2f90316 thru 0x2f91821 (0x150c bytes) 2024-09-14T05:29:58.182 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.182 INFO:teuthology.orchestra.run.smithi071.stdout:2559 read 0x417824d thru 0x41871d6 (0xef8a bytes) 2024-09-14T05:29:58.182 INFO:teuthology.orchestra.run.smithi071.stdout:2561 punch from 0x8709a8e to 0x870af94, (0x1506 bytes) 2024-09-14T05:29:58.182 INFO:teuthology.orchestra.run.smithi071.stdout:2562 punch from 0x68f2f9d to 0x68fadde, (0x7e41 bytes) 2024-09-14T05:29:58.183 INFO:teuthology.orchestra.run.smithi071.stdout:2564 write 0x7aa3276 thru 0x7aadcfc (0xaa87 bytes) 2024-09-14T05:29:58.185 INFO:teuthology.orchestra.run.smithi071.stdout:2565 trunc from 0xb05d5de to 0x8baf8b7 2024-09-14T05:29:58.191 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T05:29:58.192 INFO:teuthology.orchestra.run.smithi071.stdout:2566 write 0x8a82fe thru 0x8ac0bb (0x3dbe bytes) 2024-09-14T05:29:58.194 INFO:teuthology.orchestra.run.smithi071.stdout:2567 read 0x23dabee thru 0x23db682 (0xa95 bytes) 2024-09-14T05:29:58.194 INFO:teuthology.orchestra.run.smithi071.stdout:2569 punch from 0x2fb7ca6 to 0x2fb8297, (0x5f1 bytes) 2024-09-14T05:29:58.195 INFO:teuthology.orchestra.run.smithi071.stdout:2571 write 0x73011fa thru 0x73103fc (0xf203 bytes) 2024-09-14T05:29:58.196 INFO:teuthology.orchestra.run.smithi071.stdout:2574 punch from 0x2757b15 to 0x276772f, (0xfc1a bytes) 2024-09-14T05:29:58.199 INFO:teuthology.orchestra.run.smithi071.stdout:2575 trunc from 0x8baf8b7 to 0xb5b11f0 2024-09-14T05:29:58.205 INFO:teuthology.orchestra.run.smithi071.stdout:2576 write 0x21b2d60 thru 0x21bf058 (0xc2f9 bytes) 2024-09-14T05:29:58.207 INFO:teuthology.orchestra.run.smithi071.stdout:2577 trunc from 0xb5b11f0 to 0xacee862 2024-09-14T05:29:58.224 INFO:teuthology.orchestra.run.smithi071.stdout:2578 trunc from 0xacee862 to 0x1d9a3e6 2024-09-14T05:29:58.242 INFO:teuthology.orchestra.run.smithi071.stdout:2581 punch from 0x1aa1bda to 0x1aa3819, (0x1c3f bytes) 2024-09-14T05:29:58.242 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.257 INFO:teuthology.orchestra.run.smithi071.stdout:2583 writesame 0x9b6c535 thru 0x9b753f3 data_size 0xd(0x8ebf bytes) 2024-09-14T05:29:58.258 INFO:teuthology.orchestra.run.smithi071.stdout:2584 read 0x33ce5c0 thru 0x33cf627 (0x1068 bytes) 2024-09-14T05:29:58.258 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.258 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.258 INFO:teuthology.orchestra.run.smithi071.stdout:2587 read 0x504e52a thru 0x504e691 (0x168 bytes) 2024-09-14T05:29:58.258 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.259 INFO:teuthology.orchestra.run.smithi071.stdout:2590 read 0x543e7c5 thru 0x54404cb (0x1d07 bytes) 2024-09-14T05:29:58.259 INFO:teuthology.orchestra.run.smithi071.stdout:2591 write 0x39a0e80 thru 0x39a851c (0x769d bytes) 2024-09-14T05:29:58.261 INFO:teuthology.orchestra.run.smithi071.stdout:2592 read 0x4c3fff6 thru 0x4c4c678 (0xc683 bytes) 2024-09-14T05:29:58.262 INFO:teuthology.orchestra.run.smithi071.stdout:2593 punch from 0x738f096 to 0x739e7ae, (0xf718 bytes) 2024-09-14T05:29:58.262 INFO:teuthology.orchestra.run.smithi071.stdout:2595 read 0x8924348 thru 0x8928bec (0x48a5 bytes) 2024-09-14T05:29:58.262 INFO:teuthology.orchestra.run.smithi071.stdout:2596 read 0x5fb7c95 thru 0x5fbf1fd (0x7569 bytes) 2024-09-14T05:29:58.263 INFO:teuthology.orchestra.run.smithi071.stdout:2598 writesame 0x87b7f19 thru 0x87bfb3e data_size 0x3(0x7c26 bytes) 2024-09-14T05:29:58.264 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:29:58.264 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:29:58.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:29:58.264 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:29:58.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.264 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4235.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-14T05:29:58.264 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:29:58.264 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 44.422249018999999, 2024-09-14T05:29:58.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12887758399999999, 2024-09-14T05:29:58.264 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.265 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.265 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:29:58.265 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.265 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4235", 2024-09-14T05:29:58.265 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/3211559243", 2024-09-14T05:29:58.265 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 2 2024-09-14T05:29:58.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.265 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.265 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.265 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:29:58.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.266 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:29:58.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.266 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839833+0000", 2024-09-14T05:29:58.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:29:58.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.266 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839838+0000", 2024-09-14T05:29:58.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2440000000000001e-06 2024-09-14T05:29:58.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839848+0000", 2024-09-14T05:29:58.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0030000000000001e-05 2024-09-14T05:29:58.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839854+0000", 2024-09-14T05:29:58.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3750000000000002e-06 2024-09-14T05:29:58.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839908+0000", 2024-09-14T05:29:58.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3965999999999998e-05 2024-09-14T05:29:58.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:29:58.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839914+0000", 2024-09-14T05:29:58.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6139999999999997e-06 2024-09-14T05:29:58.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968558+0000", 2024-09-14T05:29:58.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12864361199999999 2024-09-14T05:29:58.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.269 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968710+0000", 2024-09-14T05:29:58.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015177300000000001 2024-09-14T05:29:58.269 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.269 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.269 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.269 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27764 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 188416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.269 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:29:58.269 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.050922636999999, 2024-09-14T05:29:58.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080826920999999996, 2024-09-14T05:29:58.269 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.270 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.270 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.270 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.270 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.270 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.270 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27764 2024-09-14T05:29:58.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.270 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.270 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.270 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:29:58.270 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.271 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.271 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:29:58.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.271 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.271 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211159+0000", 2024-09-14T05:29:58.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1800000000000001e-07 2024-09-14T05:29:58.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.272 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211161+0000", 2024-09-14T05:29:58.272 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.968e-06 2024-09-14T05:29:58.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.272 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211169+0000", 2024-09-14T05:29:58.272 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4800000000000001e-06 2024-09-14T05:29:58.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.272 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211174+0000", 2024-09-14T05:29:58.272 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1880000000000001e-06 2024-09-14T05:29:58.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291670+0000", 2024-09-14T05:29:58.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080495199000000003 2024-09-14T05:29:58.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291775+0000", 2024-09-14T05:29:58.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000105375 2024-09-14T05:29:58.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291985+0000", 2024-09-14T05:29:58.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021029299999999999 2024-09-14T05:29:58.274 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.274 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.274 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.274 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27765 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.274 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:29:58.274 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.050899887, 2024-09-14T05:29:58.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081115197999999999, 2024-09-14T05:29:58.274 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.274 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.274 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.275 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.275 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.275 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.275 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27765 2024-09-14T05:29:58.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.275 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.275 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.275 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:29:58.275 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.275 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.275 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:29:58.276 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.276 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.276 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211182+0000", 2024-09-14T05:29:58.276 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3599999999999999e-07 2024-09-14T05:29:58.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.276 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.276 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211184+0000", 2024-09-14T05:29:58.276 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.03e-06 2024-09-14T05:29:58.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.276 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211192+0000", 2024-09-14T05:29:58.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2309999999999996e-06 2024-09-14T05:29:58.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211197+0000", 2024-09-14T05:29:58.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4199999999999998e-06 2024-09-14T05:29:58.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292017+0000", 2024-09-14T05:29:58.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080819516999999994 2024-09-14T05:29:58.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292087+0000", 2024-09-14T05:29:58.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0056999999999997e-05 2024-09-14T05:29:58.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292296+0000", 2024-09-14T05:29:58.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000209507 2024-09-14T05:29:58.278 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.278 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.278 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.278 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27766 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.278 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:29:58.279 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.050876672999999, 2024-09-14T05:29:58.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081359077000000002, 2024-09-14T05:29:58.279 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.279 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.279 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.279 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.279 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.279 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.279 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27766 2024-09-14T05:29:58.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.279 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.279 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.279 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:29:58.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.280 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.280 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:29:58.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.280 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.280 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211205+0000", 2024-09-14T05:29:58.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7e-07 2024-09-14T05:29:58.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.281 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.281 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211207+0000", 2024-09-14T05:29:58.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4540000000000001e-06 2024-09-14T05:29:58.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.281 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.281 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211216+0000", 2024-09-14T05:29:58.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3680000000000002e-06 2024-09-14T05:29:58.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.281 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.281 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211221+0000", 2024-09-14T05:29:58.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3499999999999996e-06 2024-09-14T05:29:58.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292329+0000", 2024-09-14T05:29:58.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081108240999999998 2024-09-14T05:29:58.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292370+0000", 2024-09-14T05:29:58.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.121e-05 2024-09-14T05:29:58.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292563+0000", 2024-09-14T05:29:58.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019298399999999999 2024-09-14T05:29:58.283 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.283 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.283 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.283 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27767 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.283 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:29:58.283 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.050853159000001, 2024-09-14T05:29:58.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081623119999999993, 2024-09-14T05:29:58.283 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.284 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.284 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.284 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.284 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.284 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.284 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27767 2024-09-14T05:29:58.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.284 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:29:58.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:29:58.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:29:58.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0499999999999999e-07 2024-09-14T05:29:58.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211230+0000", 2024-09-14T05:29:58.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0700000000000001e-06 2024-09-14T05:29:58.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211236+0000", 2024-09-14T05:29:58.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8429999999999998e-06 2024-09-14T05:29:58.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211241+0000", 2024-09-14T05:29:58.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:29:58.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292594+0000", 2024-09-14T05:29:58.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081352868999999994 2024-09-14T05:29:58.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292631+0000", 2024-09-14T05:29:58.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7083999999999999e-05 2024-09-14T05:29:58.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292851+0000", 2024-09-14T05:29:58.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022011700000000001 2024-09-14T05:29:58.287 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.287 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.287 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.288 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27768 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 274432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.288 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:29:58.288 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.050834623, 2024-09-14T05:29:58.288 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081875032, 2024-09-14T05:29:58.288 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.288 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.288 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.288 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.288 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.288 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.288 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27768 2024-09-14T05:29:58.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.289 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:29:58.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:29:58.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211247+0000", 2024-09-14T05:29:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.46e-07 2024-09-14T05:29:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211248+0000", 2024-09-14T05:29:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7349999999999999e-06 2024-09-14T05:29:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211253+0000", 2024-09-14T05:29:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.899e-06 2024-09-14T05:29:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211258+0000", 2024-09-14T05:29:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2810000000000002e-06 2024-09-14T05:29:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292888+0000", 2024-09-14T05:29:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081629955000000004 2024-09-14T05:29:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292926+0000", 2024-09-14T05:29:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7947000000000001e-05 2024-09-14T05:29:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293121+0000", 2024-09-14T05:29:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019586899999999999 2024-09-14T05:29:58.292 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.292 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.292 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.292 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27769 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 294912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.292 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:29:58.292 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.050816534000001, 2024-09-14T05:29:58.292 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082181899000000003, 2024-09-14T05:29:58.292 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.292 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.293 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.293 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.293 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.293 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.293 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27769 2024-09-14T05:29:58.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.293 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.294 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.294 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:29:58.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.294 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.294 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:29:58.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.294 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.294 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211265+0000", 2024-09-14T05:29:58.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2199999999999999e-07 2024-09-14T05:29:58.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211267+0000", 2024-09-14T05:29:58.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2239999999999998e-06 2024-09-14T05:29:58.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211273+0000", 2024-09-14T05:29:58.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6540000000000004e-06 2024-09-14T05:29:58.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211277+0000", 2024-09-14T05:29:58.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6299999999999997e-06 2024-09-14T05:29:58.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293150+0000", 2024-09-14T05:29:58.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081872575000000003 2024-09-14T05:29:58.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293245+0000", 2024-09-14T05:29:58.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5327000000000004e-05 2024-09-14T05:29:58.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.297 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.297 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293446+0000", 2024-09-14T05:29:58.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000201067 2024-09-14T05:29:58.297 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.297 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.297 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.297 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27770 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.297 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:29:58.297 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.050796372000001, 2024-09-14T05:29:58.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082663707000000003, 2024-09-14T05:29:58.297 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.298 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.298 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.298 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.298 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.298 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.298 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27770 2024-09-14T05:29:58.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.298 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.298 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.298 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:29:58.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.298 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:29:58.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.299 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:29:58.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1199999999999998e-07 2024-09-14T05:29:58.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.299 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211292+0000", 2024-09-14T05:29:58.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1609999999999997e-06 2024-09-14T05:29:58.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211299+0000", 2024-09-14T05:29:58.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3300000000000004e-06 2024-09-14T05:29:58.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211303+0000", 2024-09-14T05:29:58.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7859999999999996e-06 2024-09-14T05:29:58.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293628+0000", 2024-09-14T05:29:58.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082324283999999998 2024-09-14T05:29:58.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293694+0000", 2024-09-14T05:29:58.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6277000000000003e-05 2024-09-14T05:29:58.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293948+0000", 2024-09-14T05:29:58.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025445699999999998 2024-09-14T05:29:58.301 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.301 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.301 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.302 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27790 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.302 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:29:58.302 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.050769968000001, 2024-09-14T05:29:58.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082461006000000003, 2024-09-14T05:29:58.302 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.302 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.302 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.302 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.302 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.302 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.302 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27790 2024-09-14T05:29:58.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.302 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.303 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.303 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:29:58.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.303 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.303 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:29:58.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.303 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.303 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211312+0000", 2024-09-14T05:29:58.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9299999999999998e-07 2024-09-14T05:29:58.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.304 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.304 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211314+0000", 2024-09-14T05:29:58.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000002e-06 2024-09-14T05:29:58.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.304 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.304 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211320+0000", 2024-09-14T05:29:58.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1589999999999998e-06 2024-09-14T05:29:58.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211325+0000", 2024-09-14T05:29:58.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9609999999999996e-06 2024-09-14T05:29:58.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293474+0000", 2024-09-14T05:29:58.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082149080999999999 2024-09-14T05:29:58.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293542+0000", 2024-09-14T05:29:58.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8497000000000005e-05 2024-09-14T05:29:58.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293772+0000", 2024-09-14T05:29:58.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022969900000000001 2024-09-14T05:29:58.306 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.306 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.306 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.306 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27791 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.306 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:29:58.306 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.050749654000001, 2024-09-14T05:29:58.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082700838999999998, 2024-09-14T05:29:58.307 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.307 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.307 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.307 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.307 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.307 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.307 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27791 2024-09-14T05:29:58.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.307 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.307 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:29:58.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.308 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.308 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:29:58.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.308 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.308 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211332+0000", 2024-09-14T05:29:58.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7399999999999999e-07 2024-09-14T05:29:58.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211334+0000", 2024-09-14T05:29:58.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8500000000000001e-06 2024-09-14T05:29:58.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211340+0000", 2024-09-14T05:29:58.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3010000000000003e-06 2024-09-14T05:29:58.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211345+0000", 2024-09-14T05:29:58.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.938e-06 2024-09-14T05:29:58.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293797+0000", 2024-09-14T05:29:58.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082452152000000001 2024-09-14T05:29:58.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293869+0000", 2024-09-14T05:29:58.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2236000000000004e-05 2024-09-14T05:29:58.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294032+0000", 2024-09-14T05:29:58.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016298800000000001 2024-09-14T05:29:58.311 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.311 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.311 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.311 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27792 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.312 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:29:58.312 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.050730122999999, 2024-09-14T05:29:58.312 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082897304000000005, 2024-09-14T05:29:58.312 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.312 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.312 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.312 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.312 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.312 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.312 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27792 2024-09-14T05:29:58.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.312 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.313 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.313 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:29:58.313 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.313 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.313 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:29:58.313 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.313 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.313 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:29:58.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2000000000000001e-07 2024-09-14T05:29:58.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.314 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211353+0000", 2024-09-14T05:29:58.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.761e-06 2024-09-14T05:29:58.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.314 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211359+0000", 2024-09-14T05:29:58.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2430000000000002e-06 2024-09-14T05:29:58.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211364+0000", 2024-09-14T05:29:58.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0239999999999997e-06 2024-09-14T05:29:58.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294059+0000", 2024-09-14T05:29:58.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082694834999999994 2024-09-14T05:29:58.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294114+0000", 2024-09-14T05:29:58.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4823e-05 2024-09-14T05:29:58.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.316 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294248+0000", 2024-09-14T05:29:58.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013429800000000001 2024-09-14T05:29:58.316 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.316 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.316 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.316 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27793 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.317 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:29:58.317 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.050710324000001, 2024-09-14T05:29:58.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083153362999999994, 2024-09-14T05:29:58.317 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.317 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.317 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.317 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.317 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.317 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.317 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27793 2024-09-14T05:29:58.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.318 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.318 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.318 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:29:58.318 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.318 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.318 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:29:58.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:29:58.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5100000000000001e-07 2024-09-14T05:29:58.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211373+0000", 2024-09-14T05:29:58.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.001e-06 2024-09-14T05:29:58.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211379+0000", 2024-09-14T05:29:58.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4559999999999997e-06 2024-09-14T05:29:58.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211384+0000", 2024-09-14T05:29:58.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9749999999999998e-06 2024-09-14T05:29:58.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294279+0000", 2024-09-14T05:29:58.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082894282999999999 2024-09-14T05:29:58.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294327+0000", 2024-09-14T05:29:58.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8208000000000003e-05 2024-09-14T05:29:58.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294524+0000", 2024-09-14T05:29:58.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019708900000000001 2024-09-14T05:29:58.322 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.322 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.322 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.322 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27794 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.322 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:29:58.322 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.050689999999999, 2024-09-14T05:29:58.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588590000000004, 2024-09-14T05:29:58.322 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.322 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.322 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.323 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.323 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.323 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.323 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27794 2024-09-14T05:29:58.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.323 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:29:58.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:29:58.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:29:58.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4499999999999997e-07 2024-09-14T05:29:58.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211393+0000", 2024-09-14T05:29:58.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.815e-06 2024-09-14T05:29:58.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211400+0000", 2024-09-14T05:29:58.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2899999999999999e-06 2024-09-14T05:29:58.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211404+0000", 2024-09-14T05:29:58.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:29:58.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294560+0000", 2024-09-14T05:29:58.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083155534000000003 2024-09-14T05:29:58.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.326 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294631+0000", 2024-09-14T05:29:58.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1378999999999995e-05 2024-09-14T05:29:58.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.326 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294980+0000", 2024-09-14T05:29:58.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00034844900000000002 2024-09-14T05:29:58.326 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.326 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.326 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.327 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27795 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.327 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:29:58.327 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.050670642, 2024-09-14T05:29:58.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083970367000000004, 2024-09-14T05:29:58.327 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.327 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.327 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.327 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.327 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.327 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.327 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27795 2024-09-14T05:29:58.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.328 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.328 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:29:58.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.328 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:29:58.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211411+0000", 2024-09-14T05:29:58.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5499999999999999e-07 2024-09-14T05:29:58.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211413+0000", 2024-09-14T05:29:58.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:29:58.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211419+0000", 2024-09-14T05:29:58.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1720000000000004e-06 2024-09-14T05:29:58.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211424+0000", 2024-09-14T05:29:58.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0690000000000001e-06 2024-09-14T05:29:58.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295013+0000", 2024-09-14T05:29:58.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588898999999994 2024-09-14T05:29:58.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295102+0000", 2024-09-14T05:29:58.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9462000000000005e-05 2024-09-14T05:29:58.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295381+0000", 2024-09-14T05:29:58.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000278566 2024-09-14T05:29:58.331 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.331 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.331 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.331 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27796 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.331 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:29:58.332 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.050650913, 2024-09-14T05:29:58.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084379107999999994, 2024-09-14T05:29:58.332 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.332 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.332 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.332 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.332 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.332 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.332 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27796 2024-09-14T05:29:58.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.332 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.333 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.333 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:29:58.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.333 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.333 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:29:58.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.333 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:29:58.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2800000000000002e-07 2024-09-14T05:29:58.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211432+0000", 2024-09-14T05:29:58.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:29:58.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211439+0000", 2024-09-14T05:29:58.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5239999999999997e-06 2024-09-14T05:29:58.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211444+0000", 2024-09-14T05:29:58.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.95e-06 2024-09-14T05:29:58.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295602+0000", 2024-09-14T05:29:58.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084158017000000002 2024-09-14T05:29:58.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295674+0000", 2024-09-14T05:29:58.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2385999999999994e-05 2024-09-14T05:29:58.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295809+0000", 2024-09-14T05:29:58.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013495900000000001 2024-09-14T05:29:58.336 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.336 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.336 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.336 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27797 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 266240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.336 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:29:58.337 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.050630931000001, 2024-09-14T05:29:58.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084898597000000006, 2024-09-14T05:29:58.337 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.337 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.337 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.337 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.337 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.337 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.337 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27797 2024-09-14T05:29:58.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.337 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:29:58.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:29:58.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:29:58.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:29:58.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211452+0000", 2024-09-14T05:29:58.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.708e-06 2024-09-14T05:29:58.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211458+0000", 2024-09-14T05:29:58.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3489999999999997e-06 2024-09-14T05:29:58.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211464+0000", 2024-09-14T05:29:58.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3820000000000003e-06 2024-09-14T05:29:58.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.340 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.340 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295839+0000", 2024-09-14T05:29:58.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084375055000000004 2024-09-14T05:29:58.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.340 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.340 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295918+0000", 2024-09-14T05:29:58.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9253999999999997e-05 2024-09-14T05:29:58.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.341 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296349+0000", 2024-09-14T05:29:58.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043051 2024-09-14T05:29:58.341 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.341 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.341 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.341 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27798 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.341 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:29:58.341 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.050611216, 2024-09-14T05:29:58.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085300275999999994, 2024-09-14T05:29:58.341 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.342 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.342 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.342 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.342 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.342 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.342 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27798 2024-09-14T05:29:58.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.342 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:29:58.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:29:58.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:29:58.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:29:58.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211472+0000", 2024-09-14T05:29:58.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7579999999999999e-06 2024-09-14T05:29:58.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211478+0000", 2024-09-14T05:29:58.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0310000000000004e-06 2024-09-14T05:29:58.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211483+0000", 2024-09-14T05:29:58.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8110000000000002e-06 2024-09-14T05:29:58.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.345 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296469+0000", 2024-09-14T05:29:58.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084986575999999994 2024-09-14T05:29:58.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.345 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296545+0000", 2024-09-14T05:29:58.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5749999999999998e-05 2024-09-14T05:29:58.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.345 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.345 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296770+0000", 2024-09-14T05:29:58.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022498900000000001 2024-09-14T05:29:58.346 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.346 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.346 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.346 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27799 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 307200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.346 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:29:58.346 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.050592303, 2024-09-14T05:29:58.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085718461999999995, 2024-09-14T05:29:58.346 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.346 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.346 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.346 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.347 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.347 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.347 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27799 2024-09-14T05:29:58.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.347 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.347 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.347 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:29:58.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.347 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.347 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:29:58.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.348 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:29:58.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8000000000000001e-07 2024-09-14T05:29:58.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.348 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211491+0000", 2024-09-14T05:29:58.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-14T05:29:58.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.348 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211497+0000", 2024-09-14T05:29:58.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.857e-06 2024-09-14T05:29:58.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211502+0000", 2024-09-14T05:29:58.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9790000000000001e-06 2024-09-14T05:29:58.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296800+0000", 2024-09-14T05:29:58.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085298099000000002 2024-09-14T05:29:58.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296871+0000", 2024-09-14T05:29:58.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0879999999999999e-05 2024-09-14T05:29:58.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297207+0000", 2024-09-14T05:29:58.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033651699999999999 2024-09-14T05:29:58.350 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.350 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.350 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.350 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27826 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.351 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:29:58.351 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.048877182, 2024-09-14T05:29:58.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084306729999999996, 2024-09-14T05:29:58.351 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.351 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.351 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.351 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.351 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.351 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.351 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27826 2024-09-14T05:29:58.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.351 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:29:58.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:29:58.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213205+0000", 2024-09-14T05:29:58.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.277e-06 2024-09-14T05:29:58.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213210+0000", 2024-09-14T05:29:58.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3120000000000001e-06 2024-09-14T05:29:58.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213226+0000", 2024-09-14T05:29:58.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5279000000000001e-05 2024-09-14T05:29:58.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213234+0000", 2024-09-14T05:29:58.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7020000000000007e-06 2024-09-14T05:29:58.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297239+0000", 2024-09-14T05:29:58.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084004876000000006 2024-09-14T05:29:58.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297311+0000", 2024-09-14T05:29:58.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1336999999999998e-05 2024-09-14T05:29:58.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.354 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.354 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297511+0000", 2024-09-14T05:29:58.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019994699999999999 2024-09-14T05:29:58.355 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.355 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.355 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.355 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27827 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:29:58.355 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:29:58.355 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.048831995, 2024-09-14T05:29:58.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084520537000000007, 2024-09-14T05:29:58.355 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:29:58.355 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:29:58.355 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:29:58.355 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:29:58.355 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:29:58.356 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.356 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27827 2024-09-14T05:29:58.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.356 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:29:58.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:29:58.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:29:58.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:29:58.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:29:58.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:29:58.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:29:58.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213250+0000", 2024-09-14T05:29:58.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1099999999999995e-07 2024-09-14T05:29:58.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:29:58.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213252+0000", 2024-09-14T05:29:58.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6699999999999998e-06 2024-09-14T05:29:58.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:29:58.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213259+0000", 2024-09-14T05:29:58.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8469999999999998e-06 2024-09-14T05:29:58.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213265+0000", 2024-09-14T05:29:58.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4940000000000003e-06 2024-09-14T05:29:58.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.358 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:29:58.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297541+0000", 2024-09-14T05:29:58.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084276731999999993 2024-09-14T05:29:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:29:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297579+0000", 2024-09-14T05:29:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8108999999999997e-05 2024-09-14T05:29:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:29:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:29:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:29:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297770+0000", 2024-09-14T05:29:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019007400000000001 2024-09-14T05:29:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:29:58.360 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:29:58.360 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:29:58.361 INFO:teuthology.orchestra.run.smithi071.stdout:2599 write 0x15daa68 thru 0x15e6e72 (0xc40b bytes) 2024-09-14T05:29:58.361 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:58.360+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:58.362 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:29:58.364 INFO:teuthology.orchestra.run.smithi071.stdout:2601 trunc from 0x9b753f4 to 0xd2fec72 2024-09-14T05:29:58.364 INFO:teuthology.orchestra.run.smithi071.stdout:2602 read 0x998c7c1 thru 0x9998cac (0xc4ec bytes) 2024-09-14T05:29:58.364 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.364 INFO:teuthology.orchestra.run.smithi071.stdout:2604 punch from 0x3874790 to 0x387f7d2, (0xb042 bytes) 2024-09-14T05:29:58.364 INFO:teuthology.orchestra.run.smithi071.stdout:2610 read 0xc704f7b thru 0xc70ce25 (0x7eab bytes) 2024-09-14T05:29:58.364 INFO:teuthology.orchestra.run.smithi071.stdout:2612 writesame 0x62dd8bc thru 0x62ea65e data_size 0x1(0xcda3 bytes) 2024-09-14T05:29:58.364 INFO:teuthology.orchestra.run.smithi071.stdout:2613 write 0xb3ce76b thru 0xb3dd0c1 (0xe957 bytes) 2024-09-14T05:29:58.364 INFO:teuthology.orchestra.run.smithi071.stdout:2614 writesame 0xcaeec4b thru 0xcaf80d9 data_size 0x7(0x948f bytes) 2024-09-14T05:29:58.364 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.364 INFO:teuthology.orchestra.run.smithi071.stdout:2617 read 0x3a8dfb9 thru 0x3a8ef90 (0xfd8 bytes) 2024-09-14T05:29:58.365 INFO:teuthology.orchestra.run.smithi071.stdout:2620 read 0x41cae4 thru 0x42b63a (0xeb57 bytes) 2024-09-14T05:29:58.365 INFO:teuthology.orchestra.run.smithi071.stdout:2623 write 0x4f11a9e thru 0x4f1f7ea (0xdd4d bytes) 2024-09-14T05:29:58.365 INFO:teuthology.orchestra.run.smithi071.stdout:2624 read 0x4b7b895 thru 0x4b8707c (0xb7e8 bytes) 2024-09-14T05:29:58.365 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.365 INFO:teuthology.orchestra.run.smithi071.stdout:2626 writesame 0xde002f6 thru 0xde06626 data_size 0x1(0x6331 bytes) 2024-09-14T05:29:58.365 INFO:teuthology.orchestra.run.smithi071.stdout:2627 write 0xc6b728 thru 0xc6ca60 (0x1339 bytes) 2024-09-14T05:29:58.365 INFO:teuthology.orchestra.run.smithi071.stdout:2629 writesame 0xad65666 thru 0xad6cf32 data_size 0x5(0x78cd bytes) 2024-09-14T05:29:58.365 INFO:teuthology.orchestra.run.smithi071.stdout:2631 write 0x70131d9 thru 0x7021980 (0xe7a8 bytes) 2024-09-14T05:29:58.365 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.365 INFO:teuthology.orchestra.run.smithi071.stdout:2633 punch from 0x5ed86f2 to 0x5edbd75, (0x3683 bytes) 2024-09-14T05:29:58.365 INFO:teuthology.orchestra.run.smithi071.stdout:2634 trunc from 0xde06627 to 0xc9422d5 2024-09-14T05:29:58.365 INFO:teuthology.orchestra.run.smithi071.stdout:2636 trunc from 0xc9422d5 to 0x20352b7 2024-09-14T05:29:58.368 INFO:teuthology.orchestra.run.smithi071.stdout:2637 trunc from 0x20352b7 to 0x27bade7 2024-09-14T05:29:58.384 INFO:teuthology.orchestra.run.smithi071.stdout:2638 write 0x7a8e727 thru 0x7a9a64e (0xbf28 bytes) 2024-09-14T05:29:58.387 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.387 INFO:teuthology.orchestra.run.smithi071.stdout:2640 write 0x5a07290 thru 0x5a121c4 (0xaf35 bytes) 2024-09-14T05:29:58.401 INFO:teuthology.orchestra.run.smithi071.stdout:2641 writesame 0xd421eda thru 0xd4225f2 data_size 0x1(0x719 bytes) 2024-09-14T05:29:58.402 INFO:teuthology.orchestra.run.smithi071.stdout:2643 read 0xb96574a thru 0xb967b97 (0x244e bytes) 2024-09-14T05:29:58.403 INFO:teuthology.orchestra.run.smithi071.stdout:2644 punch from 0xbe93a30 to 0xbe93b2b, (0xfb bytes) 2024-09-14T05:29:58.403 INFO:teuthology.orchestra.run.smithi071.stdout:2650 write 0x53b3f1d thru 0x53c1f4e (0xe032 bytes) 2024-09-14T05:29:58.411 INFO:teuthology.orchestra.run.smithi071.stdout:2651 write 0xdc31fbe thru 0xdc3e04c (0xc08f bytes) 2024-09-14T05:29:58.428 INFO:teuthology.orchestra.run.smithi071.stdout:2653 trunc from 0xdc3e04d to 0x62d180 2024-09-14T05:29:58.443 INFO:teuthology.orchestra.run.smithi071.stdout:2657 read 0x4bc9c4 thru 0x4bd294 (0x8d1 bytes) 2024-09-14T05:29:58.450 INFO:teuthology.orchestra.run.smithi071.stdout:2658 write 0xcbceaa thru 0xcc5300 (0x8457 bytes) 2024-09-14T05:29:58.453 INFO:teuthology.orchestra.run.smithi071.stdout:2660 read 0x9f2925 thru 0x9fe9e9 (0xc0c5 bytes) 2024-09-14T05:29:58.453 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.461 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:58.461+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:58.473 INFO:teuthology.orchestra.run.smithi071.stdout:2665 write 0xc0a5114 thru 0xc0a53f8 (0x2e5 bytes) 2024-09-14T05:29:58.474 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.474 INFO:teuthology.orchestra.run.smithi071.stdout:2670 write 0x32c16fc thru 0x32c844a (0x6d4f bytes) 2024-09-14T05:29:58.476 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:58.476 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:58.476 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:29:58.476 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:58.477 INFO:teuthology.orchestra.run.smithi071.stdout:2671 write 0x286e3ab thru 0x2878733 (0xa389 bytes) 2024-09-14T05:29:58.482 INFO:teuthology.orchestra.run.smithi071.stdout:2672 trunc from 0xc0a53f9 to 0xaa65420 2024-09-14T05:29:58.486 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:29:58.488 INFO:teuthology.orchestra.run.smithi071.stdout:2673 read 0x1cdd325 thru 0x1ce3969 (0x6645 bytes) 2024-09-14T05:29:58.492 INFO:teuthology.orchestra.run.smithi071.stdout:2674 trunc from 0xaa65420 to 0xd8e930d 2024-09-14T05:29:58.509 INFO:teuthology.orchestra.run.smithi071.stdout:2675 trunc from 0xd8e930d to 0x32cf972 2024-09-14T05:29:58.516 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.516 INFO:teuthology.orchestra.run.smithi071.stdout:2678 writesame 0x3002cf thru 0x3010d9 data_size 0x1(0xe0b bytes) 2024-09-14T05:29:58.529 INFO:teuthology.orchestra.run.smithi071.stdout:2679 write 0x79deef1 thru 0x79e7986 (0x8a96 bytes) 2024-09-14T05:29:58.532 INFO:teuthology.orchestra.run.smithi071.stdout:2680 read 0x45f8de7 thru 0x4607162 (0xe37c bytes) 2024-09-14T05:29:58.532 INFO:teuthology.orchestra.run.smithi071.stdout:2681 punch from 0x78fdc23 to 0x790d5ca, (0xf9a7 bytes) 2024-09-14T05:29:58.534 INFO:teuthology.orchestra.run.smithi071.stdout:2682 trunc from 0x79e7987 to 0x649bf71 2024-09-14T05:29:58.543 INFO:teuthology.orchestra.run.smithi071.stdout:2684 read 0x4dd3332 thru 0x4dd78a2 (0x4571 bytes) 2024-09-14T05:29:58.543 INFO:teuthology.orchestra.run.smithi071.stdout:2685 read 0x5de4415 thru 0x5dee21c (0x9e08 bytes) 2024-09-14T05:29:58.543 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.543 INFO:teuthology.orchestra.run.smithi071.stdout:2688 punch from 0xca6c77 to 0xcaa31e, (0x36a7 bytes) 2024-09-14T05:29:58.543 INFO:teuthology.orchestra.run.smithi071.stdout:2689 read 0x3c5140 thru 0x3cfa14 (0xa8d5 bytes) 2024-09-14T05:29:58.543 INFO:teuthology.orchestra.run.smithi071.stdout:2690 read 0x1780877 thru 0x178cb1f (0xc2a9 bytes) 2024-09-14T05:29:58.544 INFO:teuthology.orchestra.run.smithi071.stdout:2691 read 0x1448a47 thru 0x144ec71 (0x622b bytes) 2024-09-14T05:29:58.544 INFO:teuthology.orchestra.run.smithi071.stdout:2692 punch from 0x37d2e20 to 0x37e109c, (0xe27c bytes) 2024-09-14T05:29:58.544 INFO:teuthology.orchestra.run.smithi071.stdout:2693 read 0x3294c83 thru 0x32a471d (0xfa9b bytes) 2024-09-14T05:29:58.551 INFO:teuthology.orchestra.run.smithi071.stdout:2694 writesame 0x7cd37d3 thru 0x7cd42c0 data_size 0x1(0xaee bytes) 2024-09-14T05:29:58.552 INFO:teuthology.orchestra.run.smithi071.stdout:2695 read 0x4d8a4fe thru 0x4d939b7 (0x94ba bytes) 2024-09-14T05:29:58.553 INFO:teuthology.orchestra.run.smithi071.stdout:2696 read 0x4a5cd6d thru 0x4a69d22 (0xcfb6 bytes) 2024-09-14T05:29:58.553 INFO:teuthology.orchestra.run.smithi071.stdout:2697 read 0x2649835 thru 0x2650335 (0x6b01 bytes) 2024-09-14T05:29:58.566 INFO:teuthology.orchestra.run.smithi071.stdout:2698 write 0xdba56e5 thru 0xdbb3c92 (0xe5ae bytes) 2024-09-14T05:29:58.569 INFO:teuthology.orchestra.run.smithi071.stdout:2699 write 0xb9e8cf4 thru 0xb9f81be (0xf4cb bytes) 2024-09-14T05:29:58.571 INFO:teuthology.orchestra.run.smithi071.stdout:2700 read 0x20c9da8 thru 0x20d7a70 (0xdcc9 bytes) 2024-09-14T05:29:58.571 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.571 INFO:teuthology.orchestra.run.smithi071.stdout:2706 punch from 0x39e2776 to 0x39f260f, (0xfe99 bytes) 2024-09-14T05:29:58.571 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.571 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.571 INFO:teuthology.orchestra.run.smithi071.stdout:2709 punch from 0x7358b08 to 0x735a0cc, (0x15c4 bytes) 2024-09-14T05:29:58.571 INFO:teuthology.orchestra.run.smithi071.stdout:2710 writesame 0x23935fc thru 0x239e5e8 data_size 0x1(0xafed bytes) 2024-09-14T05:29:58.571 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:58.570+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:58.595 INFO:teuthology.orchestra.run.smithi071.stdout:2711 write 0x418018f thru 0x418ab91 (0xaa03 bytes) 2024-09-14T05:29:58.597 INFO:teuthology.orchestra.run.smithi071.stdout:2713 read 0x32eb127 thru 0x32facba (0xfb94 bytes) 2024-09-14T05:29:58.597 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.597 INFO:teuthology.orchestra.run.smithi071.stdout:2715 write 0x4d1c4e8 thru 0x4d241a6 (0x7cbf bytes) 2024-09-14T05:29:58.599 INFO:teuthology.orchestra.run.smithi071.stdout:2716 read 0x25dfb85 thru 0x25eaf05 (0xb381 bytes) 2024-09-14T05:29:58.599 INFO:teuthology.orchestra.run.smithi071.stdout:2718 writesame 0x895eb6 thru 0x8a5567 data_size 0x5b(0xf6b2 bytes) 2024-09-14T05:29:58.601 INFO:teuthology.orchestra.run.smithi071.stdout:2719 read 0x5fbdba3 thru 0x5fbe580 (0x9de bytes) 2024-09-14T05:29:58.601 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.608 INFO:teuthology.orchestra.run.smithi071.stdout:2721 write 0xe1fb0b3 thru 0xe204a87 (0x99d5 bytes) 2024-09-14T05:29:58.610 INFO:teuthology.orchestra.run.smithi071.stdout:2722 write 0xe07b9bb thru 0xe084886 (0x8ecc bytes) 2024-09-14T05:29:58.611 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:58.611 INFO:teuthology.orchestra.run.smithi071.stdout:2724 punch from 0x2af5250 to 0x2b0274c, (0xd4fc bytes) 2024-09-14T05:29:58.611 INFO:teuthology.orchestra.run.smithi071.stdout:2725 write 0x5328455 thru 0x532cf1f (0x4acb bytes) 2024-09-14T05:29:58.614 INFO:teuthology.orchestra.run.smithi071.stdout:2726 punch from 0xb5a623 to 0xb5ec9c, (0x4679 bytes) 2024-09-14T05:29:58.614 INFO:teuthology.orchestra.run.smithi071.stdout:2727 writesame 0xd0ff8c6 thru 0xd10835f data_size 0x2(0x8a9a bytes) 2024-09-14T05:29:58.623 INFO:teuthology.orchestra.run.smithi071.stdout:2728 clone 22 order 24 su 524288 sc 10 2024-09-14T05:29:58.636 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:58.636 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:58.636 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:29:58.636 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:29:58.636 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:58.645 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:29:58.686 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:58.685+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:58.764 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:29:58.763+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:29:58.789 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:58.789 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:29:58.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:29:58.789 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:29:58.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.789 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3523 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6062080~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.789 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:29:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.273561607000001, 2024-09-14T05:29:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096911030999999995, 2024-09-14T05:29:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3523 2024-09-14T05:29:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:29:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:29:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514359+0000", 2024-09-14T05:29:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9299999999999999e-07 2024-09-14T05:29:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514361+0000", 2024-09-14T05:29:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:29:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514366+0000", 2024-09-14T05:29:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1660000000000002e-06 2024-09-14T05:29:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514371+0000", 2024-09-14T05:29:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.648e-06 2024-09-14T05:29:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.610954+0000", 2024-09-14T05:29:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096582170999999994 2024-09-14T05:29:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611002+0000", 2024-09-14T05:29:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7923999999999997e-05 2024-09-14T05:29:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611269+0000", 2024-09-14T05:29:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026789499999999998 2024-09-14T05:29:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3524 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6553600~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:29:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.273542415000001, 2024-09-14T05:29:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097263347999999999, 2024-09-14T05:29:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3524 2024-09-14T05:29:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:29:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:29:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:29:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:29:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514380+0000", 2024-09-14T05:29:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6500000000000001e-06 2024-09-14T05:29:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514386+0000", 2024-09-14T05:29:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7360000000000004e-06 2024-09-14T05:29:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514391+0000", 2024-09-14T05:29:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2830000000000003e-06 2024-09-14T05:29:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611295+0000", 2024-09-14T05:29:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096904814000000006 2024-09-14T05:29:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611343+0000", 2024-09-14T05:29:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7506999999999998e-05 2024-09-14T05:29:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611641+0000", 2024-09-14T05:29:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029798800000000001 2024-09-14T05:29:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3525 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7045120~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:29:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.273520527999999, 2024-09-14T05:29:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097644422999999994, 2024-09-14T05:29:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3525 2024-09-14T05:29:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:29:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:29:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514400+0000", 2024-09-14T05:29:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.77e-07 2024-09-14T05:29:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514402+0000", 2024-09-14T05:29:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6530000000000001e-06 2024-09-14T05:29:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514411+0000", 2024-09-14T05:29:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7429999999999997e-06 2024-09-14T05:29:58.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514417+0000", 2024-09-14T05:29:58.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2249999999999997e-06 2024-09-14T05:29:58.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611668+0000", 2024-09-14T05:29:58.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097250681000000005 2024-09-14T05:29:58.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611716+0000", 2024-09-14T05:29:58.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7336000000000002e-05 2024-09-14T05:29:58.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612044+0000", 2024-09-14T05:29:58.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032840799999999998 2024-09-14T05:29:58.803 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.803 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.803 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.803 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3526 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7536640~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:29:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.273496044000002, 2024-09-14T05:29:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097943884999999994, 2024-09-14T05:29:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3526 2024-09-14T05:29:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:29:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:29:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:29:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0400000000000002e-07 2024-09-14T05:29:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514426+0000", 2024-09-14T05:29:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:29:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514432+0000", 2024-09-14T05:29:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.105e-06 2024-09-14T05:29:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514438+0000", 2024-09-14T05:29:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5149999999999997e-06 2024-09-14T05:29:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612072+0000", 2024-09-14T05:29:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097633947999999998 2024-09-14T05:29:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612120+0000", 2024-09-14T05:29:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8458999999999998e-05 2024-09-14T05:29:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612368+0000", 2024-09-14T05:29:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024761999999999999 2024-09-14T05:29:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3527 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 8028160~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:29:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.273475331, 2024-09-14T05:29:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098244955999999994, 2024-09-14T05:29:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.809 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3527 2024-09-14T05:29:58.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.812 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:29:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:29:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519842+0000", 2024-09-14T05:29:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053975230000000004 2024-09-14T05:29:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519851+0000", 2024-09-14T05:29:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5599999999999994e-06 2024-09-14T05:29:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519858+0000", 2024-09-14T05:29:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2409999999999998e-06 2024-09-14T05:29:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519863+0000", 2024-09-14T05:29:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:29:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612395+0000", 2024-09-14T05:29:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092532552000000004 2024-09-14T05:29:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612442+0000", 2024-09-14T05:29:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6486000000000002e-05 2024-09-14T05:29:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612690+0000", 2024-09-14T05:29:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024791599999999999 2024-09-14T05:29:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3603 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5177344~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:29:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.267922886000001, 2024-09-14T05:29:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096796436, 2024-09-14T05:29:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3603 2024-09-14T05:29:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:29:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:29:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:29:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1899999999999999e-07 2024-09-14T05:29:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519998+0000", 2024-09-14T05:29:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5000000000000001e-07 2024-09-14T05:29:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520002+0000", 2024-09-14T05:29:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7299999999999999e-06 2024-09-14T05:29:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520004+0000", 2024-09-14T05:29:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.429e-06 2024-09-14T05:29:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616493+0000", 2024-09-14T05:29:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096488345000000003 2024-09-14T05:29:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616544+0000", 2024-09-14T05:29:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1724000000000001e-05 2024-09-14T05:29:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616794+0000", 2024-09-14T05:29:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000249139 2024-09-14T05:29:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3604 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5668864~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:29:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.267908248000001, 2024-09-14T05:29:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108222999999994, 2024-09-14T05:29:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.825 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3604 2024-09-14T05:29:58.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:29:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:29:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:29:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7000000000000001e-07 2024-09-14T05:29:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520013+0000", 2024-09-14T05:29:58.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.389e-06 2024-09-14T05:29:58.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520018+0000", 2024-09-14T05:29:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3019999999999996e-06 2024-09-14T05:29:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520026+0000", 2024-09-14T05:29:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6170000000000007e-06 2024-09-14T05:29:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616816+0000", 2024-09-14T05:29:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096789766999999999 2024-09-14T05:29:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616871+0000", 2024-09-14T05:29:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4811000000000001e-05 2024-09-14T05:29:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617120+0000", 2024-09-14T05:29:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024906700000000001 2024-09-14T05:29:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3605 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6160384~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:29:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.267889788000002, 2024-09-14T05:29:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097423945999999997, 2024-09-14T05:29:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3605 2024-09-14T05:29:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:29:58.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:29:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:29:58.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3999999999999998e-07 2024-09-14T05:29:58.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520032+0000", 2024-09-14T05:29:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.083e-06 2024-09-14T05:29:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520036+0000", 2024-09-14T05:29:58.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.019e-06 2024-09-14T05:29:58.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520038+0000", 2024-09-14T05:29:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4849999999999999e-06 2024-09-14T05:29:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617147+0000", 2024-09-14T05:29:58.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108665999999996 2024-09-14T05:29:58.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617199+0000", 2024-09-14T05:29:58.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2558000000000001e-05 2024-09-14T05:29:58.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617454+0000", 2024-09-14T05:29:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025489499999999999 2024-09-14T05:29:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3606 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6651904~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:29:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.267879065999999, 2024-09-14T05:29:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097745705000000002, 2024-09-14T05:29:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3606 2024-09-14T05:29:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:29:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:29:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:29:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-14T05:29:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520042+0000", 2024-09-14T05:29:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4799999999999997e-07 2024-09-14T05:29:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520046+0000", 2024-09-14T05:29:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8149999999999999e-06 2024-09-14T05:29:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520049+0000", 2024-09-14T05:29:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.576e-06 2024-09-14T05:29:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617476+0000", 2024-09-14T05:29:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097427558999999997 2024-09-14T05:29:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617522+0000", 2024-09-14T05:29:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5927999999999998e-05 2024-09-14T05:29:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617787+0000", 2024-09-14T05:29:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000264663 2024-09-14T05:29:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3607 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7143424~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:29:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.267868538999998, 2024-09-14T05:29:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097871125000000003, 2024-09-14T05:29:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3607 2024-09-14T05:29:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:29:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:29:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520052+0000", 2024-09-14T05:29:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.03e-07 2024-09-14T05:29:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520053+0000", 2024-09-14T05:29:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.85e-07 2024-09-14T05:29:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520056+0000", 2024-09-14T05:29:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9140000000000001e-06 2024-09-14T05:29:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520059+0000", 2024-09-14T05:29:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4600000000000002e-06 2024-09-14T05:29:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617809+0000", 2024-09-14T05:29:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097750216000000001 2024-09-14T05:29:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617864+0000", 2024-09-14T05:29:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5211999999999999e-05 2024-09-14T05:29:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617923+0000", 2024-09-14T05:29:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8235000000000001e-05 2024-09-14T05:29:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3608 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7634944~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:29:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.267858231000002, 2024-09-14T05:29:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098199212999999994, 2024-09-14T05:29:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3608 2024-09-14T05:29:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:29:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:29:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525953+0000", 2024-09-14T05:29:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058911850000000002 2024-09-14T05:29:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525957+0000", 2024-09-14T05:29:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2379999999999999e-06 2024-09-14T05:29:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525967+0000", 2024-09-14T05:29:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5389999999999996e-06 2024-09-14T05:29:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525972+0000", 2024-09-14T05:29:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.541e-06 2024-09-14T05:29:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617945+0000", 2024-09-14T05:29:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091972597000000003 2024-09-14T05:29:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617992+0000", 2024-09-14T05:29:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7397e-05 2024-09-14T05:29:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.618261+0000", 2024-09-14T05:29:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026871600000000001 2024-09-14T05:29:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3640 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6586368~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:29:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.261750388999999, 2024-09-14T05:29:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097068580000000002, 2024-09-14T05:29:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3640 2024-09-14T05:29:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:29:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:29:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:29:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.23e-07 2024-09-14T05:29:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526171+0000", 2024-09-14T05:29:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:29:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526175+0000", 2024-09-14T05:29:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8369999999999999e-06 2024-09-14T05:29:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526177+0000", 2024-09-14T05:29:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4959999999999999e-06 2024-09-14T05:29:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622847+0000", 2024-09-14T05:29:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096669790000000005 2024-09-14T05:29:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622902+0000", 2024-09-14T05:29:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5157000000000001e-05 2024-09-14T05:29:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623238+0000", 2024-09-14T05:29:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033603399999999997 2024-09-14T05:29:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3641 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7077888~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:29:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.261739879, 2024-09-14T05:29:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097371141999999994, 2024-09-14T05:29:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3641 2024-09-14T05:29:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:29:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:29:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531851+0000", 2024-09-14T05:29:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005671265 2024-09-14T05:29:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531855+0000", 2024-09-14T05:29:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0849999999999999e-06 2024-09-14T05:29:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531867+0000", 2024-09-14T05:29:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1028e-05 2024-09-14T05:29:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531872+0000", 2024-09-14T05:29:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000003e-06 2024-09-14T05:29:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623251+0000", 2024-09-14T05:29:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091378980999999998 2024-09-14T05:29:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623295+0000", 2024-09-14T05:29:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3881999999999998e-05 2024-09-14T05:29:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623551+0000", 2024-09-14T05:29:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025636099999999998 2024-09-14T05:29:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3721 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:29:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.255881603999999, 2024-09-14T05:29:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096806472000000005, 2024-09-14T05:29:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3721 2024-09-14T05:29:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:29:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:29:58.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532039+0000", 2024-09-14T05:29:58.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.05e-07 2024-09-14T05:29:58.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532040+0000", 2024-09-14T05:29:58.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9099999999999991e-07 2024-09-14T05:29:58.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532043+0000", 2024-09-14T05:29:58.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6610000000000002e-06 2024-09-14T05:29:58.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532046+0000", 2024-09-14T05:29:58.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4080000000000001e-06 2024-09-14T05:29:58.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628545+0000", 2024-09-14T05:29:58.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096499615999999996 2024-09-14T05:29:58.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628592+0000", 2024-09-14T05:29:58.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6907000000000003e-05 2024-09-14T05:29:58.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628845+0000", 2024-09-14T05:29:58.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025268399999999998 2024-09-14T05:29:58.876 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.876 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.877 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.877 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3722 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6193152~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.877 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:29:58.877 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.255871524, 2024-09-14T05:29:58.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097146587000000006, 2024-09-14T05:29:58.877 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.877 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.877 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.877 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.877 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.877 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.878 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3722 2024-09-14T05:29:58.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.878 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:29:58.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:29:58.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538191+0000", 2024-09-14T05:29:58.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061429969999999999 2024-09-14T05:29:58.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538200+0000", 2024-09-14T05:29:58.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7069999999999998e-06 2024-09-14T05:29:58.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538212+0000", 2024-09-14T05:29:58.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1504e-05 2024-09-14T05:29:58.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538219+0000", 2024-09-14T05:29:58.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7500000000000003e-06 2024-09-14T05:29:58.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628869+0000", 2024-09-14T05:29:58.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090649224 2024-09-14T05:29:58.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628919+0000", 2024-09-14T05:29:58.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0144e-05 2024-09-14T05:29:58.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.629195+0000", 2024-09-14T05:29:58.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027626099999999998 2024-09-14T05:29:58.881 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.881 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.881 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.881 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3758 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7110656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.881 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:29:58.881 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.249277308, 2024-09-14T05:29:58.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096977544999999998, 2024-09-14T05:29:58.882 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.882 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.882 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.882 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3758 2024-09-14T05:29:58.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.882 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:29:58.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:29:58.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:29:58.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:29:58.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538645+0000", 2024-09-14T05:29:58.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.843e-06 2024-09-14T05:29:58.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538651+0000", 2024-09-14T05:29:58.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6169999999999998e-06 2024-09-14T05:29:58.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538655+0000", 2024-09-14T05:29:58.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5399999999999997e-06 2024-09-14T05:29:58.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635225+0000", 2024-09-14T05:29:58.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096570149999999993 2024-09-14T05:29:58.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635272+0000", 2024-09-14T05:29:58.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7199e-05 2024-09-14T05:29:58.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635620+0000", 2024-09-14T05:29:58.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034782600000000003 2024-09-14T05:29:58.886 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.886 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.886 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.886 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3759 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.887 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:29:58.887 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.249259396999999, 2024-09-14T05:29:58.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097335331999999997, 2024-09-14T05:29:58.887 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.887 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.887 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.887 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.887 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.887 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.887 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3759 2024-09-14T05:29:58.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.887 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:29:58.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:29:58.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544013+0000", 2024-09-14T05:29:58.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053520829999999997 2024-09-14T05:29:58.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544016+0000", 2024-09-14T05:29:58.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6660000000000001e-06 2024-09-14T05:29:58.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544026+0000", 2024-09-14T05:29:58.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7089999999999997e-06 2024-09-14T05:29:58.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544043+0000", 2024-09-14T05:29:58.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833999999999998e-05 2024-09-14T05:29:58.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635646+0000", 2024-09-14T05:29:58.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091603544999999995 2024-09-14T05:29:58.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635696+0000", 2024-09-14T05:29:58.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9307000000000001e-05 2024-09-14T05:29:58.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635996+0000", 2024-09-14T05:29:58.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030018800000000001 2024-09-14T05:29:58.891 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.891 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.891 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.891 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3838 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6225920~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.891 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:29:58.892 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.243597717, 2024-09-14T05:29:58.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096894235999999995, 2024-09-14T05:29:58.892 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.892 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.892 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.892 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.892 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.892 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.892 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3838 2024-09-14T05:29:58.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.892 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:29:58.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:29:58.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544323+0000", 2024-09-14T05:29:58.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-14T05:29:58.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544325+0000", 2024-09-14T05:29:58.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.75e-06 2024-09-14T05:29:58.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544331+0000", 2024-09-14T05:29:58.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5019999999999998e-06 2024-09-14T05:29:58.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544335+0000", 2024-09-14T05:29:58.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2989999999999998e-06 2024-09-14T05:29:58.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640869+0000", 2024-09-14T05:29:58.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096533351000000003 2024-09-14T05:29:58.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640919+0000", 2024-09-14T05:29:58.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999999e-05 2024-09-14T05:29:58.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641217+0000", 2024-09-14T05:29:58.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029717600000000002 2024-09-14T05:29:58.896 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.896 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.896 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.896 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3839 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.896 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:29:58.896 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.243578779, 2024-09-14T05:29:58.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097240339999999995, 2024-09-14T05:29:58.897 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.897 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.897 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.897 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.897 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.897 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.897 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3839 2024-09-14T05:29:58.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.897 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:29:58.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:29:58.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549834+0000", 2024-09-14T05:29:58.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054927819999999999 2024-09-14T05:29:58.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549838+0000", 2024-09-14T05:29:58.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8689999999999997e-06 2024-09-14T05:29:58.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549848+0000", 2024-09-14T05:29:58.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0154e-05 2024-09-14T05:29:58.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549853+0000", 2024-09-14T05:29:58.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9740000000000001e-06 2024-09-14T05:29:58.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641245+0000", 2024-09-14T05:29:58.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091392245999999996 2024-09-14T05:29:58.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641295+0000", 2024-09-14T05:29:58.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0018999999999997e-05 2024-09-14T05:29:58.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641582+0000", 2024-09-14T05:29:58.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028629600000000001 2024-09-14T05:29:58.900 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.901 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.901 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.901 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3868 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:58.901 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:29:58.901 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.237916201000001, 2024-09-14T05:29:58.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096859555999999999, 2024-09-14T05:29:58.901 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:58.901 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:58.901 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:58.901 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:58.901 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:58.901 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:58.902 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3868 2024-09-14T05:29:58.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.902 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:58.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:58.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:29:58.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:58.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:29:58.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:58.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:58.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556173+0000", 2024-09-14T05:29:58.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006169038 2024-09-14T05:29:58.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:58.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556177+0000", 2024-09-14T05:29:58.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0010000000000003e-06 2024-09-14T05:29:58.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:58.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556188+0000", 2024-09-14T05:29:58.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0675e-05 2024-09-14T05:29:58.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:58.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556199+0000", 2024-09-14T05:29:58.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.155e-05 2024-09-14T05:29:58.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:58.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646511+0000", 2024-09-14T05:29:58.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090312187000000002 2024-09-14T05:29:58.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:58.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646563+0000", 2024-09-14T05:29:58.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2219000000000003e-05 2024-09-14T05:29:58.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:58.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:58.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:58.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646863+0000", 2024-09-14T05:29:58.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029988600000000002 2024-09-14T05:29:58.905 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.905 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.905 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.905 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:58.905 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:58.905 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:58.907 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:29:58.907 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:29:58.906+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:29:59.007 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:59.007+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:59.051 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:59.051 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:59.051 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:29:59.051 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:59.060 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:29:59.108 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:59.107+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:59.162 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:29:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:29:59.163 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:59.171 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:29:59.191 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-14T05:29:59.209 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:59.208+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:59.258 INFO:teuthology.orchestra.run.smithi071.stdout:flattening image image_client.0-clone21 2024-09-14T05:29:59.310 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:29:59.309+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:29:59.317 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:29:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:29:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:29:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1989 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 16326080~54496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:29:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.75510118, 2024-09-14T05:29:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096664322999999996, 2024-09-14T05:29:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1989 2024-09-14T05:29:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:29:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:29:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561455+0000", 2024-09-14T05:29:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1699999999999997e-07 2024-09-14T05:29:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590703+0000", 2024-09-14T05:29:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029248225999999999 2024-09-14T05:29:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590732+0000", 2024-09-14T05:29:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8679000000000001e-05 2024-09-14T05:29:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590759+0000", 2024-09-14T05:29:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7319999999999999e-05 2024-09-14T05:29:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590835+0000", 2024-09-14T05:29:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5543999999999994e-05 2024-09-14T05:29:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590956+0000", 2024-09-14T05:29:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012168 2024-09-14T05:29:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 1", 2024-09-14T05:29:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.591261+0000", 2024-09-14T05:29:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030484799999999998 2024-09-14T05:29:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.637682+0000", 2024-09-14T05:29:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046420550999999997 2024-09-14T05:29:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.657912+0000", 2024-09-14T05:29:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020230011999999999 2024-09-14T05:29:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658007+0000", 2024-09-14T05:29:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5814000000000001e-05 2024-09-14T05:29:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658118+0000", 2024-09-14T05:29:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110832 2024-09-14T05:29:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2033 3.1e 3:7e47f6eb:::rbd_data.112d1f36d097.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30853661~63531] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:29:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.643826514000001, 2024-09-14T05:29:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065167630000000004, 2024-09-14T05:29:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2033 2024-09-14T05:29:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:29:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:29:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672730+0000", 2024-09-14T05:29:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.393e-06 2024-09-14T05:29:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691289+0000", 2024-09-14T05:29:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018558991 2024-09-14T05:29:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691316+0000", 2024-09-14T05:29:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7498999999999999e-05 2024-09-14T05:29:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691331+0000", 2024-09-14T05:29:59.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4652e-05 2024-09-14T05:29:59.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691397+0000", 2024-09-14T05:29:59.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6112000000000007e-05 2024-09-14T05:29:59.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691509+0000", 2024-09-14T05:29:59.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111599 2024-09-14T05:29:59.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:59.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.692253+0000", 2024-09-14T05:29:59.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074456200000000004 2024-09-14T05:29:59.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.719140+0000", 2024-09-14T05:29:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026886542999999999 2024-09-14T05:29:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737789+0000", 2024-09-14T05:29:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018649203999999999 2024-09-14T05:29:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737846+0000", 2024-09-14T05:29:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6501e-05 2024-09-14T05:29:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737896+0000", 2024-09-14T05:29:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0574000000000001e-05 2024-09-14T05:29:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.335 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.335 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3008 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13729792~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:29:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.012902644, 2024-09-14T05:29:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061007651000000003, 2024-09-14T05:29:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3008 2024-09-14T05:29:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:29:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:29:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303653+0000", 2024-09-14T05:29:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0699999999999999e-07 2024-09-14T05:29:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303654+0000", 2024-09-14T05:29:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-07 2024-09-14T05:29:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303657+0000", 2024-09-14T05:29:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2890000000000002e-06 2024-09-14T05:29:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303661+0000", 2024-09-14T05:29:59.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3290000000000004e-06 2024-09-14T05:29:59.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364233+0000", 2024-09-14T05:29:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060572134 2024-09-14T05:29:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364315+0000", 2024-09-14T05:29:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1549000000000001e-05 2024-09-14T05:29:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364660+0000", 2024-09-14T05:29:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034525300000000001 2024-09-14T05:29:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3009 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14221312~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:29:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.012888554, 2024-09-14T05:29:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061445063000000001, 2024-09-14T05:29:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3009 2024-09-14T05:29:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:29:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:29:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:29:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:29:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303669+0000", 2024-09-14T05:29:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7150000000000001e-06 2024-09-14T05:29:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303675+0000", 2024-09-14T05:29:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5339999999999996e-06 2024-09-14T05:29:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303681+0000", 2024-09-14T05:29:59.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3639999999999998e-06 2024-09-14T05:29:59.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364705+0000", 2024-09-14T05:29:59.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061024123 2024-09-14T05:29:59.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364786+0000", 2024-09-14T05:29:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1378999999999995e-05 2024-09-14T05:29:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365112+0000", 2024-09-14T05:29:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032558700000000001 2024-09-14T05:29:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3010 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14712832~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:29:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.01286769, 2024-09-14T05:29:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061834630000000002, 2024-09-14T05:29:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3010 2024-09-14T05:29:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:29:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:29:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303688+0000", 2024-09-14T05:29:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:29:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303690+0000", 2024-09-14T05:29:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9460000000000001e-06 2024-09-14T05:29:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303696+0000", 2024-09-14T05:29:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2230000000000004e-06 2024-09-14T05:29:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303700+0000", 2024-09-14T05:29:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7290000000000002e-06 2024-09-14T05:29:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365163+0000", 2024-09-14T05:29:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061463093000000003 2024-09-14T05:29:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365214+0000", 2024-09-14T05:29:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0977999999999999e-05 2024-09-14T05:29:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365522+0000", 2024-09-14T05:29:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000308227 2024-09-14T05:29:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3241 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:29:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.987193267999999, 2024-09-14T05:29:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061041892, 2024-09-14T05:29:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3241 2024-09-14T05:29:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:29:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:29:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329496+0000", 2024-09-14T05:29:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013421799999999999 2024-09-14T05:29:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329499+0000", 2024-09-14T05:29:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8459999999999999e-06 2024-09-14T05:29:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329505+0000", 2024-09-14T05:29:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4320000000000004e-06 2024-09-14T05:29:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329509+0000", 2024-09-14T05:29:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0339999999999998e-06 2024-09-14T05:29:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.389947+0000", 2024-09-14T05:29:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060438093999999998 2024-09-14T05:29:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390038+0000", 2024-09-14T05:29:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0253999999999993e-05 2024-09-14T05:29:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390404+0000", 2024-09-14T05:29:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036601400000000001 2024-09-14T05:29:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3242 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:29:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.98703836, 2024-09-14T05:29:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061282426000000001, 2024-09-14T05:29:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3242 2024-09-14T05:29:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:29:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:29:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:29:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9900000000000001e-07 2024-09-14T05:29:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329518+0000", 2024-09-14T05:29:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.328e-06 2024-09-14T05:29:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329522+0000", 2024-09-14T05:29:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.563e-06 2024-09-14T05:29:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329525+0000", 2024-09-14T05:29:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7379999999999999e-06 2024-09-14T05:29:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390437+0000", 2024-09-14T05:29:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060912463 2024-09-14T05:29:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390498+0000", 2024-09-14T05:29:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0615999999999997e-05 2024-09-14T05:29:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390799+0000", 2024-09-14T05:29:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030131899999999998 2024-09-14T05:29:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3243 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:29:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.987026574000001, 2024-09-14T05:29:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061728980000000003, 2024-09-14T05:29:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3243 2024-09-14T05:29:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:29:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:29:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329692+0000", 2024-09-14T05:29:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000163948 2024-09-14T05:29:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329696+0000", 2024-09-14T05:29:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5540000000000002e-06 2024-09-14T05:29:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329706+0000", 2024-09-14T05:29:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7960000000000007e-06 2024-09-14T05:29:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329712+0000", 2024-09-14T05:29:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6109999999999999e-06 2024-09-14T05:29:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390861+0000", 2024-09-14T05:29:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061148767 2024-09-14T05:29:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390942+0000", 2024-09-14T05:29:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1266000000000004e-05 2024-09-14T05:29:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391258+0000", 2024-09-14T05:29:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031503800000000002 2024-09-14T05:29:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3244 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14778368~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:29:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.986831526, 2024-09-14T05:29:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062050594000000001, 2024-09-14T05:29:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3244 2024-09-14T05:29:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:29:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:29:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329916+0000", 2024-09-14T05:29:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019290799999999999 2024-09-14T05:29:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329919+0000", 2024-09-14T05:29:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9019999999999999e-06 2024-09-14T05:29:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329926+0000", 2024-09-14T05:29:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9e-06 2024-09-14T05:29:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329932+0000", 2024-09-14T05:29:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.088e-06 2024-09-14T05:29:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391299+0000", 2024-09-14T05:29:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061366136000000002 2024-09-14T05:29:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391391+0000", 2024-09-14T05:29:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2204e-05 2024-09-14T05:29:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391774+0000", 2024-09-14T05:29:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038345600000000001 2024-09-14T05:29:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3921 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 5308416~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:29:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.915287716000002, 2024-09-14T05:29:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060842694000000003, 2024-09-14T05:29:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3921 2024-09-14T05:29:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:29:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:29:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401705+0000", 2024-09-14T05:29:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768499999999999 2024-09-14T05:29:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401709+0000", 2024-09-14T05:29:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0319999999999997e-06 2024-09-14T05:29:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401720+0000", 2024-09-14T05:29:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1005000000000001e-05 2024-09-14T05:29:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401727+0000", 2024-09-14T05:29:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1879999999999996e-06 2024-09-14T05:29:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461748+0000", 2024-09-14T05:29:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060021199999999997 2024-09-14T05:29:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461793+0000", 2024-09-14T05:29:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4530999999999998e-05 2024-09-14T05:29:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462110+0000", 2024-09-14T05:29:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031705299999999997 2024-09-14T05:29:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3923 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 6291456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:29:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.914641688, 2024-09-14T05:29:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060891273000000003, 2024-09-14T05:29:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3923 2024-09-14T05:29:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:29:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:29:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407054+0000", 2024-09-14T05:29:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005140787 2024-09-14T05:29:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407059+0000", 2024-09-14T05:29:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.04e-06 2024-09-14T05:29:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407069+0000", 2024-09-14T05:29:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4429999999999992e-06 2024-09-14T05:29:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407076+0000", 2024-09-14T05:29:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1679999999999998e-06 2024-09-14T05:29:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462514+0000", 2024-09-14T05:29:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055438551000000003 2024-09-14T05:29:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462562+0000", 2024-09-14T05:29:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7649999999999999e-05 2024-09-14T05:29:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462805+0000", 2024-09-14T05:29:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024263400000000001 2024-09-14T05:29:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4052 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 12550144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:29:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.899069679, 2024-09-14T05:29:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060872904999999998, 2024-09-14T05:29:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4052 2024-09-14T05:29:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:29:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:29:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417486+0000", 2024-09-14T05:29:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-14T05:29:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417487+0000", 2024-09-14T05:29:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4499999999999995e-07 2024-09-14T05:29:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417489+0000", 2024-09-14T05:29:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8899999999999999e-06 2024-09-14T05:29:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417492+0000", 2024-09-14T05:29:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4389999999999999e-06 2024-09-14T05:29:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478027+0000", 2024-09-14T05:29:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060534959999999999 2024-09-14T05:29:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478079+0000", 2024-09-14T05:29:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2104999999999997e-05 2024-09-14T05:29:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478358+0000", 2024-09-14T05:29:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000279351 2024-09-14T05:29:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4053 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13041664~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:29:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.899060294999998, 2024-09-14T05:29:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061258866000000002, 2024-09-14T05:29:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4053 2024-09-14T05:29:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:29:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:29:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:29:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.17e-07 2024-09-14T05:29:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417496+0000", 2024-09-14T05:29:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0890000000000001e-06 2024-09-14T05:29:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417499+0000", 2024-09-14T05:29:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8629999999999999e-06 2024-09-14T05:29:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417502+0000", 2024-09-14T05:29:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6290000000000002e-06 2024-09-14T05:29:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478387+0000", 2024-09-14T05:29:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060885433000000003 2024-09-14T05:29:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478435+0000", 2024-09-14T05:29:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8167999999999997e-05 2024-09-14T05:29:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478754+0000", 2024-09-14T05:29:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031846700000000001 2024-09-14T05:29:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4054 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13533184~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:29:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.899050411000001, 2024-09-14T05:29:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061604694000000002, 2024-09-14T05:29:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4054 2024-09-14T05:29:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:29:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:29:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:29:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.04e-07 2024-09-14T05:29:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417506+0000", 2024-09-14T05:29:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8299999999999995e-07 2024-09-14T05:29:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417509+0000", 2024-09-14T05:29:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.88e-06 2024-09-14T05:29:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417511+0000", 2024-09-14T05:29:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5950000000000001e-06 2024-09-14T05:29:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478784+0000", 2024-09-14T05:29:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061272971000000002 2024-09-14T05:29:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478841+0000", 2024-09-14T05:29:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-14T05:29:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479109+0000", 2024-09-14T05:29:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026794600000000003 2024-09-14T05:29:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4055 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14024704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:29:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.899040709000001, 2024-09-14T05:29:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061932299000000003, 2024-09-14T05:29:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4055 2024-09-14T05:29:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:29:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:29:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417515+0000", 2024-09-14T05:29:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-14T05:29:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417516+0000", 2024-09-14T05:29:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7199999999999997e-07 2024-09-14T05:29:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417519+0000", 2024-09-14T05:29:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.937e-06 2024-09-14T05:29:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417522+0000", 2024-09-14T05:29:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1420000000000002e-06 2024-09-14T05:29:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479140+0000", 2024-09-14T05:29:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061617933999999999 2024-09-14T05:29:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479188+0000", 2024-09-14T05:29:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8717000000000001e-05 2024-09-14T05:29:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479447+0000", 2024-09-14T05:29:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025838800000000002 2024-09-14T05:29:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4056 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14516224~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:29:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.899030536000001, 2024-09-14T05:29:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062263185999999998, 2024-09-14T05:29:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4056 2024-09-14T05:29:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:29:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:29:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:29:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8700000000000002e-07 2024-09-14T05:29:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417528+0000", 2024-09-14T05:29:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.579e-06 2024-09-14T05:29:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417533+0000", 2024-09-14T05:29:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2400000000000001e-06 2024-09-14T05:29:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417536+0000", 2024-09-14T05:29:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0309999999999999e-06 2024-09-14T05:29:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479477+0000", 2024-09-14T05:29:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061940967 2024-09-14T05:29:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479523+0000", 2024-09-14T05:29:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6335000000000003e-05 2024-09-14T05:29:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479788+0000", 2024-09-14T05:29:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026474700000000002 2024-09-14T05:29:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4057 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15007744~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:29:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.899016074999999, 2024-09-14T05:29:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062589928000000003, 2024-09-14T05:29:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4057 2024-09-14T05:29:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:29:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:29:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422838+0000", 2024-09-14T05:29:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052990349999999997 2024-09-14T05:29:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422854+0000", 2024-09-14T05:29:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5523999999999999e-05 2024-09-14T05:29:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422864+0000", 2024-09-14T05:29:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0084e-05 2024-09-14T05:29:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422872+0000", 2024-09-14T05:29:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-14T05:29:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479813+0000", 2024-09-14T05:29:59.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056941389000000002 2024-09-14T05:29:59.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479868+0000", 2024-09-14T05:29:59.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4973999999999999e-05 2024-09-14T05:29:59.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.480129+0000", 2024-09-14T05:29:59.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026084200000000001 2024-09-14T05:29:59.409 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.409 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.410 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.410 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4171 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14057472~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.410 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:29:59.410 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.887018515000001, 2024-09-14T05:29:59.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061184929999999998, 2024-09-14T05:29:59.410 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.410 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.410 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.411 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.411 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4171 2024-09-14T05:29:59.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.411 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:29:59.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:29:59.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:29:59.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.39e-07 2024-09-14T05:29:59.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429541+0000", 2024-09-14T05:29:59.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:29:59.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429547+0000", 2024-09-14T05:29:59.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3280000000000002e-06 2024-09-14T05:29:59.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429552+0000", 2024-09-14T05:29:59.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4859999999999997e-06 2024-09-14T05:29:59.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490379+0000", 2024-09-14T05:29:59.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060826681 2024-09-14T05:29:59.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490434+0000", 2024-09-14T05:29:59.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4577999999999998e-05 2024-09-14T05:29:59.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490722+0000", 2024-09-14T05:29:59.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028781400000000001 2024-09-14T05:29:59.414 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.414 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.414 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.414 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4172 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14548992~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.414 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:29:59.415 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.886996506999999, 2024-09-14T05:29:59.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061498256000000001, 2024-09-14T05:29:59.415 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.415 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.415 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.415 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.415 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.415 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.415 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4172 2024-09-14T05:29:59.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.415 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:29:59.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:29:59.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:29:59.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.72e-07 2024-09-14T05:29:59.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429561+0000", 2024-09-14T05:29:59.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.717e-06 2024-09-14T05:29:59.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429566+0000", 2024-09-14T05:29:59.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6529999999999999e-06 2024-09-14T05:29:59.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429571+0000", 2024-09-14T05:29:59.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5979999999999999e-06 2024-09-14T05:29:59.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490751+0000", 2024-09-14T05:29:59.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061180336000000002 2024-09-14T05:29:59.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490798+0000", 2024-09-14T05:29:59.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6875000000000001e-05 2024-09-14T05:29:59.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491057+0000", 2024-09-14T05:29:59.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025870500000000002 2024-09-14T05:29:59.419 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.419 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.419 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.419 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4173 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15040512~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.419 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:29:59.419 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.886978881000001, 2024-09-14T05:29:59.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061894820000000003, 2024-09-14T05:29:59.419 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.419 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.419 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.420 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4173 2024-09-14T05:29:59.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.420 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:29:59.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:29:59.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429577+0000", 2024-09-14T05:29:59.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-14T05:29:59.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429578+0000", 2024-09-14T05:29:59.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6980000000000001e-06 2024-09-14T05:29:59.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429585+0000", 2024-09-14T05:29:59.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.725e-06 2024-09-14T05:29:59.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429590+0000", 2024-09-14T05:29:59.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.938e-06 2024-09-14T05:29:59.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491091+0000", 2024-09-14T05:29:59.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061500773000000002 2024-09-14T05:29:59.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491136+0000", 2024-09-14T05:29:59.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5284000000000002e-05 2024-09-14T05:29:59.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491471+0000", 2024-09-14T05:29:59.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033485699999999998 2024-09-14T05:29:59.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.423 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.423 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.423 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:59.424 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-14T05:29:59.425 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:29:59.426 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:29:59.424+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:29:59.526 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:29:59.525+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:29:59.532 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:59.532 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:59.532 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:29:59.533 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:59.541 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:29:59.576 INFO:teuthology.orchestra.run.smithi071.stdout:2730 trunc from 0xe204a88 to 0xac64ee6 2024-09-14T05:29:59.601 INFO:teuthology.orchestra.run.smithi071.stdout:2731 read 0xa4dc433 thru 0xa4eaa44 (0xe612 bytes) 2024-09-14T05:29:59.601 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:59.603 INFO:teuthology.orchestra.run.smithi071.stdout:2734 trunc from 0xac64ee6 to 0xc88f3da 2024-09-14T05:29:59.607 INFO:teuthology.orchestra.run.smithi071.stdout:2736 write 0x18cd65e thru 0x18cfec6 (0x2869 bytes) 2024-09-14T05:29:59.613 INFO:teuthology.orchestra.run.smithi071.stdout:2737 writesame 0x36bea92 thru 0x36c8a23 data_size 0x32(0x9f92 bytes) 2024-09-14T05:29:59.627 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:59.626+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:59.629 INFO:teuthology.orchestra.run.smithi071.stdout:2738 trunc from 0xc88f3da to 0x8b7895 2024-09-14T05:29:59.691 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:59.691 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:29:59.691 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:29:59.691 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:29:59.691 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:59.692 INFO:teuthology.orchestra.run.smithi071.stdout:2739 writesame 0xee47e49 thru 0xee4bfea data_size 0x1f(0x41a2 bytes) 2024-09-14T05:29:59.692 INFO:teuthology.orchestra.run.smithi071.stdout:2740 write 0xe891ae1 thru 0xe89961c (0x7b3c bytes) 2024-09-14T05:29:59.694 INFO:teuthology.orchestra.run.smithi071.stdout:2741 write 0x42e4a7f thru 0x42f14b0 (0xca32 bytes) 2024-09-14T05:29:59.696 INFO:teuthology.orchestra.run.smithi071.stdout:2742 punch from 0x3f0214d to 0x3f0dbe6, (0xba99 bytes) 2024-09-14T05:29:59.696 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:29:59.696 INFO:teuthology.orchestra.run.smithi071.stdout:2744 write 0x3bd7e9b thru 0x3be7b60 (0xfcc6 bytes) 2024-09-14T05:29:59.697 INFO:teuthology.orchestra.run.smithi071.stdout:2745 clone 23 order 22 su 131072 sc 10 2024-09-14T05:29:59.700 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:29:59.729 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:29:59.728+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:29:59.804 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:29:59.804 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:29:59.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:29:59.804 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:29:59.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.805 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:1134 3.7 e30/24 3:e1ef7f87:::rbd_data.112dd25fb4c.000000000000005f:head v 30'40, mlcod=30'36)", 2024-09-14T05:29:59.805 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.166405+0000", 2024-09-14T05:29:59.805 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.635863328999999, 2024-09-14T05:29:59.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061353650000000003, 2024-09-14T05:29:59.805 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.805 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.806 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.806 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166405+0000", 2024-09-14T05:29:59.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166405+0000", 2024-09-14T05:29:59.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166406+0000", 2024-09-14T05:29:59.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.85e-07 2024-09-14T05:29:59.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166413+0000", 2024-09-14T05:29:59.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1029999999999996e-06 2024-09-14T05:29:59.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166425+0000", 2024-09-14T05:29:59.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.258e-05 2024-09-14T05:29:59.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166433+0000", 2024-09-14T05:29:59.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2910000000000002e-06 2024-09-14T05:29:59.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166492+0000", 2024-09-14T05:29:59.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9107999999999997e-05 2024-09-14T05:29:59.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166513+0000", 2024-09-14T05:29:59.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.128e-05 2024-09-14T05:29:59.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.172500+0000", 2024-09-14T05:29:59.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059870519999999997 2024-09-14T05:29:59.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.172540+0000", 2024-09-14T05:29:59.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0166000000000001e-05 2024-09-14T05:29:59.812 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.812 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.812 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.813 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1205 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:59.813 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:29:59.813 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.604378461, 2024-09-14T05:29:59.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0086364279999999998, 2024-09-14T05:29:59.813 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.814 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.814 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.814 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.814 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.814 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.814 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1205 2024-09-14T05:29:59.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.814 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:29:59.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:29:59.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197891+0000", 2024-09-14T05:29:59.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.364e-06 2024-09-14T05:29:59.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197896+0000", 2024-09-14T05:29:59.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3000000000000001e-06 2024-09-14T05:29:59.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197910+0000", 2024-09-14T05:29:59.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3339000000000001e-05 2024-09-14T05:29:59.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197919+0000", 2024-09-14T05:29:59.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8869999999999998e-06 2024-09-14T05:29:59.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197974+0000", 2024-09-14T05:29:59.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5720000000000002e-05 2024-09-14T05:29:59.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.198094+0000", 2024-09-14T05:29:59.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119739 2024-09-14T05:29:59.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206526+0000", 2024-09-14T05:29:59.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084320790000000003 2024-09-14T05:29:59.820 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.820 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.820 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.821 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1220 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:59.821 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:29:59.821 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.595622097, 2024-09-14T05:29:59.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012822337, 2024-09-14T05:29:59.821 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.821 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.821 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.821 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.822 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1220 2024-09-14T05:29:59.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.822 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:29:59.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:29:59.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206647+0000", 2024-09-14T05:29:59.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4600000000000003e-07 2024-09-14T05:29:59.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206652+0000", 2024-09-14T05:29:59.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6619999999999996e-06 2024-09-14T05:29:59.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206662+0000", 2024-09-14T05:29:59.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.995e-06 2024-09-14T05:29:59.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206667+0000", 2024-09-14T05:29:59.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-14T05:29:59.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206706+0000", 2024-09-14T05:29:59.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9523000000000001e-05 2024-09-14T05:29:59.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206755+0000", 2024-09-14T05:29:59.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000002e-05 2024-09-14T05:29:59.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:59.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206877+0000", 2024-09-14T05:29:59.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121938 2024-09-14T05:29:59.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:59.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.207310+0000", 2024-09-14T05:29:59.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000432452 2024-09-14T05:29:59.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:59.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219364+0000", 2024-09-14T05:29:59.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012054481000000001 2024-09-14T05:29:59.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219425+0000", 2024-09-14T05:29:59.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0618000000000001e-05 2024-09-14T05:29:59.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219468+0000", 2024-09-14T05:29:59.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3739000000000003e-05 2024-09-14T05:29:59.830 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.830 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.830 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.831 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1243 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:59.831 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:29:59.831 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.582656870000001, 2024-09-14T05:29:59.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078660580000000004, 2024-09-14T05:29:59.831 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.831 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.831 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.832 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.832 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.832 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.832 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1243 2024-09-14T05:29:59.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.832 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:29:59.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:29:59.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219612+0000", 2024-09-14T05:29:59.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.09e-07 2024-09-14T05:29:59.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219616+0000", 2024-09-14T05:29:59.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.216e-06 2024-09-14T05:29:59.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219630+0000", 2024-09-14T05:29:59.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3064999999999999e-05 2024-09-14T05:29:59.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219637+0000", 2024-09-14T05:29:59.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7409999999999992e-06 2024-09-14T05:29:59.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219697+0000", 2024-09-14T05:29:59.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.02e-05 2024-09-14T05:29:59.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219833+0000", 2024-09-14T05:29:59.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013595600000000001 2024-09-14T05:29:59.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227477+0000", 2024-09-14T05:29:59.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076439710000000003 2024-09-14T05:29:59.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.841 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.842 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1264 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:59.842 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:29:59.842 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.574646077000001, 2024-09-14T05:29:59.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080867990000000004, 2024-09-14T05:29:59.842 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.842 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.842 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.842 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.842 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.843 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.843 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1264 2024-09-14T05:29:59.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.843 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:29:59.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:29:59.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227623+0000", 2024-09-14T05:29:59.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:29:59.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227627+0000", 2024-09-14T05:29:59.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.194e-06 2024-09-14T05:29:59.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227639+0000", 2024-09-14T05:29:59.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1296e-05 2024-09-14T05:29:59.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227648+0000", 2024-09-14T05:29:59.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2280000000000004e-06 2024-09-14T05:29:59.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227689+0000", 2024-09-14T05:29:59.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1329999999999999e-05 2024-09-14T05:29:59.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227741+0000", 2024-09-14T05:29:59.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1641000000000002e-05 2024-09-14T05:29:59.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:29:59.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227914+0000", 2024-09-14T05:29:59.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000173215 2024-09-14T05:29:59.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:59.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228474+0000", 2024-09-14T05:29:59.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055980199999999998 2024-09-14T05:29:59.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:59.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235608+0000", 2024-09-14T05:29:59.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0071343830000000002 2024-09-14T05:29:59.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235669+0000", 2024-09-14T05:29:59.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1068999999999994e-05 2024-09-14T05:29:59.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235709+0000", 2024-09-14T05:29:59.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9671e-05 2024-09-14T05:29:59.851 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.851 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.851 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.852 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1520 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:59.852 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:29:59.852 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.479638458, 2024-09-14T05:29:59.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061963629999999999, 2024-09-14T05:29:59.852 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.852 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.852 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:29:59.852 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.853 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.853 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.853 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1520 2024-09-14T05:29:59.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.853 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:29:59.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:29:59.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322631+0000", 2024-09-14T05:29:59.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.54e-07 2024-09-14T05:29:59.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322635+0000", 2024-09-14T05:29:59.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0450000000000002e-06 2024-09-14T05:29:59.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322646+0000", 2024-09-14T05:29:59.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1919999999999999e-05 2024-09-14T05:29:59.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322654+0000", 2024-09-14T05:29:59.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3139999999999998e-06 2024-09-14T05:29:59.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322704+0000", 2024-09-14T05:29:59.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0420000000000002e-05 2024-09-14T05:29:59.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322794+0000", 2024-09-14T05:29:59.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0037000000000005e-05 2024-09-14T05:29:59.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328826+0000", 2024-09-14T05:29:59.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060318730000000001 2024-09-14T05:29:59.859 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.859 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.859 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.860 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1526 3.4 3:2695cbe4:::rbd_data.112dd25fb4c.0000000000000119:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:59.860 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:29:59.860 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.476105514, 2024-09-14T05:29:59.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0079537359999999994, 2024-09-14T05:29:59.860 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.860 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.861 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.861 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.861 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.861 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.861 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1526 2024-09-14T05:29:59.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.861 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:29:59.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:29:59.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:29:59.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3600000000000003e-07 2024-09-14T05:29:59.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326168+0000", 2024-09-14T05:29:59.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3460000000000004e-06 2024-09-14T05:29:59.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326178+0000", 2024-09-14T05:29:59.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0291e-05 2024-09-14T05:29:59.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326183+0000", 2024-09-14T05:29:59.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-14T05:29:59.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326230+0000", 2024-09-14T05:29:59.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6650999999999998e-05 2024-09-14T05:29:59.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326295+0000", 2024-09-14T05:29:59.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5158000000000002e-05 2024-09-14T05:29:59.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:59.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326408+0000", 2024-09-14T05:29:59.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113039 2024-09-14T05:29:59.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:59.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.331648+0000", 2024-09-14T05:29:59.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052404929999999997 2024-09-14T05:29:59.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:59.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334011+0000", 2024-09-14T05:29:59.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002362669 2024-09-14T05:29:59.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334064+0000", 2024-09-14T05:29:59.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3198999999999997e-05 2024-09-14T05:29:59.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334116+0000", 2024-09-14T05:29:59.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2133999999999999e-05 2024-09-14T05:29:59.870 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.870 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.870 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.870 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1529 3.1 3:809746f7:::rbd_data.112dd25fb4c.0000000000000118:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:59.870 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:29:59.870 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.474340964, 2024-09-14T05:29:59.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061981060000000001, 2024-09-14T05:29:59.871 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.871 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.871 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.871 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.871 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.871 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.871 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1529 2024-09-14T05:29:59.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.872 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:29:59.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:29:59.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327928+0000", 2024-09-14T05:29:59.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0100000000000004e-07 2024-09-14T05:29:59.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327933+0000", 2024-09-14T05:29:59.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4689999999999999e-06 2024-09-14T05:29:59.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327944+0000", 2024-09-14T05:29:59.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1623999999999999e-05 2024-09-14T05:29:59.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327951+0000", 2024-09-14T05:29:59.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9759999999999998e-06 2024-09-14T05:29:59.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328003+0000", 2024-09-14T05:29:59.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1521999999999999e-05 2024-09-14T05:29:59.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328087+0000", 2024-09-14T05:29:59.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3988999999999998e-05 2024-09-14T05:29:59.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:59.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328304+0000", 2024-09-14T05:29:59.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000217254 2024-09-14T05:29:59.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:59.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.333363+0000", 2024-09-14T05:29:59.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050586499999999996 2024-09-14T05:29:59.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:59.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334026+0000", 2024-09-14T05:29:59.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066303499999999997 2024-09-14T05:29:59.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334080+0000", 2024-09-14T05:29:59.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4261000000000001e-05 2024-09-14T05:29:59.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334125+0000", 2024-09-14T05:29:59.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5525000000000002e-05 2024-09-14T05:29:59.878 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.878 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.878 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.878 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1531 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:29:59.878 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:29:59.878 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.473287347999999, 2024-09-14T05:29:59.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082114309999999999, 2024-09-14T05:29:59.879 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.879 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.879 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.879 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1531 2024-09-14T05:29:59.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.879 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:29:59.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:29:59.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328982+0000", 2024-09-14T05:29:59.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:29:59.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328985+0000", 2024-09-14T05:29:59.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7900000000000001e-06 2024-09-14T05:29:59.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328996+0000", 2024-09-14T05:29:59.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.028e-05 2024-09-14T05:29:59.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329001+0000", 2024-09-14T05:29:59.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0610000000000003e-06 2024-09-14T05:29:59.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329035+0000", 2024-09-14T05:29:59.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4366999999999998e-05 2024-09-14T05:29:59.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329082+0000", 2024-09-14T05:29:59.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6646e-05 2024-09-14T05:29:59.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:59.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329195+0000", 2024-09-14T05:29:59.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113279 2024-09-14T05:29:59.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:59.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334293+0000", 2024-09-14T05:29:59.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050980399999999999 2024-09-14T05:29:59.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:59.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337101+0000", 2024-09-14T05:29:59.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0028076170000000001 2024-09-14T05:29:59.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337154+0000", 2024-09-14T05:29:59.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3641999999999999e-05 2024-09-14T05:29:59.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337192+0000", 2024-09-14T05:29:59.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7932000000000002e-05 2024-09-14T05:29:59.884 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.884 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.884 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.884 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:2131 3.b e30/24 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head v 30'36, mlcod=30'34)", 2024-09-14T05:29:59.885 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:29:59.885 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.009939993, 2024-09-14T05:29:59.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031871709999999998, 2024-09-14T05:29:59.885 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.885 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.885 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.885 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:29:59.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:29:59.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792331+0000", 2024-09-14T05:29:59.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6900000000000001e-06 2024-09-14T05:29:59.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797319+0000", 2024-09-14T05:29:59.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049876349999999998 2024-09-14T05:29:59.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797349+0000", 2024-09-14T05:29:59.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0046e-05 2024-09-14T05:29:59.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797359+0000", 2024-09-14T05:29:59.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0037999999999999e-05 2024-09-14T05:29:59.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797445+0000", 2024-09-14T05:29:59.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6132999999999997e-05 2024-09-14T05:29:59.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797472+0000", 2024-09-14T05:29:59.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7478999999999999e-05 2024-09-14T05:29:59.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824161+0000", 2024-09-14T05:29:59.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026688714999999998 2024-09-14T05:29:59.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824200+0000", 2024-09-14T05:29:59.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8974000000000003e-05 2024-09-14T05:29:59.889 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.889 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.889 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.889 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4259 3.4 3:25f21dc9:::rbd_data.112d3d611e54.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 294912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.889 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:29:59.889 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.410767378999999, 2024-09-14T05:29:59.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010043553, 2024-09-14T05:29:59.890 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.890 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.890 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.890 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4259 2024-09-14T05:29:59.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.890 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:29:59.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:29:59.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391503+0000", 2024-09-14T05:29:59.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9099999999999999e-06 2024-09-14T05:29:59.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391821+0000", 2024-09-14T05:29:59.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031871800000000001 2024-09-14T05:29:59.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391844+0000", 2024-09-14T05:29:59.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2045e-05 2024-09-14T05:29:59.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391853+0000", 2024-09-14T05:29:59.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7249999999999992e-06 2024-09-14T05:29:59.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391917+0000", 2024-09-14T05:29:59.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3458000000000001e-05 2024-09-14T05:29:59.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392037+0000", 2024-09-14T05:29:59.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120449 2024-09-14T05:29:59.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:59.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392305+0000", 2024-09-14T05:29:59.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026739299999999999 2024-09-14T05:29:59.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:59.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.393609+0000", 2024-09-14T05:29:59.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013042450000000001 2024-09-14T05:29:59.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:59.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401430+0000", 2024-09-14T05:29:59.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078211670000000004 2024-09-14T05:29:59.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401499+0000", 2024-09-14T05:29:59.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9374000000000004e-05 2024-09-14T05:29:59.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401544+0000", 2024-09-14T05:29:59.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5068999999999998e-05 2024-09-14T05:29:59.895 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.895 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.895 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.895 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4452 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.895 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:29:59.896 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.919954771, 2024-09-14T05:29:59.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066061380000000001, 2024-09-14T05:29:59.896 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.896 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.896 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.896 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.896 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.896 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.896 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4452 2024-09-14T05:29:59.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.896 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:29:59.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:29:59.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882314+0000", 2024-09-14T05:29:59.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5899999999999998e-07 2024-09-14T05:29:59.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882315+0000", 2024-09-14T05:29:59.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6169999999999999e-06 2024-09-14T05:29:59.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882319+0000", 2024-09-14T05:29:59.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3629999999999998e-06 2024-09-14T05:29:59.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882324+0000", 2024-09-14T05:29:59.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1370000000000001e-06 2024-09-14T05:29:59.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883369+0000", 2024-09-14T05:29:59.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010448320000000001 2024-09-14T05:29:59.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883421+0000", 2024-09-14T05:29:59.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2401000000000002e-05 2024-09-14T05:29:59.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:59.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883638+0000", 2024-09-14T05:29:59.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021725600000000001 2024-09-14T05:29:59.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:59.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.884464+0000", 2024-09-14T05:29:59.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00082569800000000004 2024-09-14T05:29:59.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:59.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888860+0000", 2024-09-14T05:29:59.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0043963370000000002 2024-09-14T05:29:59.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888896+0000", 2024-09-14T05:29:59.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6022999999999997e-05 2024-09-14T05:29:59.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888920+0000", 2024-09-14T05:29:59.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3215000000000001e-05 2024-09-14T05:29:59.901 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.901 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.901 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.902 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4467 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.902 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:29:59.902 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.901586321, 2024-09-14T05:29:59.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0065700189999999999, 2024-09-14T05:29:59.902 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.902 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.902 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.902 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.902 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.902 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.902 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4467 2024-09-14T05:29:59.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.902 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:29:59.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:29:59.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:29:59.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-14T05:29:59.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900684+0000", 2024-09-14T05:29:59.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.629e-06 2024-09-14T05:29:59.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900688+0000", 2024-09-14T05:29:59.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8469999999999998e-06 2024-09-14T05:29:59.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900691+0000", 2024-09-14T05:29:59.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-14T05:29:59.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.901775+0000", 2024-09-14T05:29:59.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001084177 2024-09-14T05:29:59.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.901862+0000", 2024-09-14T05:29:59.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7167000000000001e-05 2024-09-14T05:29:59.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:59.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.902223+0000", 2024-09-14T05:29:59.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036068800000000001 2024-09-14T05:29:59.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:59.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.902843+0000", 2024-09-14T05:29:59.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000620489 2024-09-14T05:29:59.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:59.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907167+0000", 2024-09-14T05:29:59.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0043242649999999999 2024-09-14T05:29:59.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907217+0000", 2024-09-14T05:29:59.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9648000000000003e-05 2024-09-14T05:29:59.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907252+0000", 2024-09-14T05:29:59.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4780000000000002e-05 2024-09-14T05:29:59.908 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.908 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.908 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.908 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4554 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:29:59.908 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:29:59.908 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.817866567999999, 2024-09-14T05:29:59.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069998999999999999, 2024-09-14T05:29:59.908 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.908 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.908 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.908 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.908 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.909 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4554 2024-09-14T05:29:59.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.909 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:29:59.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:29:59.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:29:59.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9299999999999999e-07 2024-09-14T05:29:59.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984404+0000", 2024-09-14T05:29:59.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.855e-06 2024-09-14T05:29:59.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984407+0000", 2024-09-14T05:29:59.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-14T05:29:59.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984410+0000", 2024-09-14T05:29:59.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8159999999999998e-06 2024-09-14T05:29:59.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985511+0000", 2024-09-14T05:29:59.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00110053 2024-09-14T05:29:59.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985561+0000", 2024-09-14T05:29:59.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0389000000000002e-05 2024-09-14T05:29:59.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:59.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985772+0000", 2024-09-14T05:29:59.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000210571 2024-09-14T05:29:59.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:59.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.987075+0000", 2024-09-14T05:29:59.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001303128 2024-09-14T05:29:59.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:59.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991303+0000", 2024-09-14T05:29:59.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042283310000000001 2024-09-14T05:29:59.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991359+0000", 2024-09-14T05:29:59.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6335999999999997e-05 2024-09-14T05:29:59.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991402+0000", 2024-09-14T05:29:59.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2074999999999999e-05 2024-09-14T05:29:59.913 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.914 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.914 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.914 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4711 3.4 3:226df663:::rbd_data.112d3d611e54.000000000000002d:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:29:59.914 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:29:59.914 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.635195412000002, 2024-09-14T05:29:59.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011304193000000001, 2024-09-14T05:29:59.914 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.914 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.914 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.915 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4711 2024-09-14T05:29:59.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.915 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:29:59.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:29:59.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167074+0000", 2024-09-14T05:29:59.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0999999999999997e-07 2024-09-14T05:29:59.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167078+0000", 2024-09-14T05:29:59.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.878e-06 2024-09-14T05:29:59.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167087+0000", 2024-09-14T05:29:59.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6940000000000007e-06 2024-09-14T05:29:59.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167093+0000", 2024-09-14T05:29:59.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7740000000000001e-06 2024-09-14T05:29:59.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167128+0000", 2024-09-14T05:29:59.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5231999999999997e-05 2024-09-14T05:29:59.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167207+0000", 2024-09-14T05:29:59.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9036999999999996e-05 2024-09-14T05:29:59.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:29:59.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.169140+0000", 2024-09-14T05:29:59.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001932897 2024-09-14T05:29:59.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:59.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178258+0000", 2024-09-14T05:29:59.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091180819999999996 2024-09-14T05:29:59.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:59.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178314+0000", 2024-09-14T05:29:59.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5571e-05 2024-09-14T05:29:59.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178348+0000", 2024-09-14T05:29:59.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3747000000000001e-05 2024-09-14T05:29:59.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178377+0000", 2024-09-14T05:29:59.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9471e-05 2024-09-14T05:29:59.920 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.920 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.920 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.920 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4721 3.1 3:86302a6e:::rbd_data.112d3d611e54.0000000000000011:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:29:59.920 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:29:59.920 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.603209926000002, 2024-09-14T05:29:59.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066831750000000004, 2024-09-14T05:29:59.920 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.920 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.920 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.921 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4721 2024-09-14T05:29:59.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.921 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:29:59.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:29:59.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199059+0000", 2024-09-14T05:29:59.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.06e-07 2024-09-14T05:29:59.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199062+0000", 2024-09-14T05:29:59.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2279999999999998e-06 2024-09-14T05:29:59.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199072+0000", 2024-09-14T05:29:59.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5999999999999996e-06 2024-09-14T05:29:59.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199078+0000", 2024-09-14T05:29:59.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9109999999999998e-06 2024-09-14T05:29:59.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199123+0000", 2024-09-14T05:29:59.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5278000000000003e-05 2024-09-14T05:29:59.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199202+0000", 2024-09-14T05:29:59.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8521000000000003e-05 2024-09-14T05:29:59.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:29:59.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199271+0000", 2024-09-14T05:29:59.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9857999999999994e-05 2024-09-14T05:29:59.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:59.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199941+0000", 2024-09-14T05:29:59.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066984599999999998 2024-09-14T05:29:59.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:59.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205653+0000", 2024-09-14T05:29:59.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057117319999999997 2024-09-14T05:29:59.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205701+0000", 2024-09-14T05:29:59.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8254000000000003e-05 2024-09-14T05:29:59.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205741+0000", 2024-09-14T05:29:59.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0241000000000003e-05 2024-09-14T05:29:59.926 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.926 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.926 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.926 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4819 3.12 3:4e8efced:::rbd_data.112d3d611e54.000000000000000e:head [stat,truncate 1114112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:29:59.926 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:29:59.926 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.517895766999999, 2024-09-14T05:29:59.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062422770000000001, 2024-09-14T05:29:59.926 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.926 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.926 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:29:59.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:29:59.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:29:59.927 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4819 2024-09-14T05:29:59.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.927 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:29:59.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:29:59.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284373+0000", 2024-09-14T05:29:59.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3399999999999998e-07 2024-09-14T05:29:59.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284377+0000", 2024-09-14T05:29:59.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8539999999999999e-06 2024-09-14T05:29:59.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284388+0000", 2024-09-14T05:29:59.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1328999999999999e-05 2024-09-14T05:29:59.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284395+0000", 2024-09-14T05:29:59.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9759999999999998e-06 2024-09-14T05:29:59.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284723+0000", 2024-09-14T05:29:59.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032775799999999999 2024-09-14T05:29:59.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284801+0000", 2024-09-14T05:29:59.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7562e-05 2024-09-14T05:29:59.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:29:59.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284933+0000", 2024-09-14T05:29:59.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000131989 2024-09-14T05:29:59.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:29:59.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.287450+0000", 2024-09-14T05:29:59.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025177699999999999 2024-09-14T05:29:59.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:29:59.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290531+0000", 2024-09-14T05:29:59.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0030809959999999999 2024-09-14T05:29:59.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290588+0000", 2024-09-14T05:29:59.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6177000000000001e-05 2024-09-14T05:29:59.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290615+0000", 2024-09-14T05:29:59.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7132e-05 2024-09-14T05:29:59.932 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.932 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.932 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.932 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:7067 3.5 e35/24 3:a65d0cff:::rbd_data.112df7f2fe0a.0000000000000028:head v 35'89, mlcod=34'86)", 2024-09-14T05:29:59.932 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:29:59.932 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.659502496000002, 2024-09-14T05:29:59.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0064513080000000002, 2024-09-14T05:29:59.932 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.932 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.932 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.933 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:29:59.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:29:59.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:29:59.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:29:59.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142771+0000", 2024-09-14T05:29:59.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8949999999999997e-06 2024-09-14T05:29:59.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142779+0000", 2024-09-14T05:29:59.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8699999999999992e-06 2024-09-14T05:29:59.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142785+0000", 2024-09-14T05:29:59.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4899999999999997e-06 2024-09-14T05:29:59.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142843+0000", 2024-09-14T05:29:59.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7377000000000003e-05 2024-09-14T05:29:59.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142871+0000", 2024-09-14T05:29:59.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8305000000000001e-05 2024-09-14T05:29:59.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149177+0000", 2024-09-14T05:29:59.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063058639999999996 2024-09-14T05:29:59.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149217+0000", 2024-09-14T05:29:59.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0080999999999998e-05 2024-09-14T05:29:59.936 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.936 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.936 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.936 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:72582 3.b e47/24 3:d5072567:::rbd_data.112d4a90444c.0000000000000119:head v 47'386, mlcod=46'383)", 2024-09-14T05:29:59.937 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:29:59.937 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.277241559, 2024-09-14T05:29:59.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0067720940000000002, 2024-09-14T05:29:59.937 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.937 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.937 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.937 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:29:59.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:29:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:29:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1100000000000002e-07 2024-09-14T05:29:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525031+0000", 2024-09-14T05:29:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0740000000000003e-06 2024-09-14T05:29:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525040+0000", 2024-09-14T05:29:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5329999999999994e-06 2024-09-14T05:29:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525045+0000", 2024-09-14T05:29:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:29:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525090+0000", 2024-09-14T05:29:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4743000000000002e-05 2024-09-14T05:29:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525102+0000", 2024-09-14T05:29:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2067e-05 2024-09-14T05:29:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531757+0000", 2024-09-14T05:29:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066550380000000003 2024-09-14T05:29:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531799+0000", 2024-09-14T05:29:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1640000000000001e-05 2024-09-14T05:29:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:79671 3.15 e47/24 3:ab572765:::rbd_data.112d956b542f.0000000000000013:head v 47'120, mlcod=47'118)", 2024-09-14T05:29:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:29:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.6512613409999997, 2024-09-14T05:29:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066700609999999997, 2024-09-14T05:29:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:29:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:29:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:29:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:29:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:29:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:29:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:29:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:29:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:29:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:29:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151008+0000", 2024-09-14T05:29:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:29:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:29:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151264+0000", 2024-09-14T05:29:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025656099999999999 2024-09-14T05:29:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:29:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151278+0000", 2024-09-14T05:29:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3288999999999999e-05 2024-09-14T05:29:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:29:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151287+0000", 2024-09-14T05:29:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7909999999999993e-06 2024-09-14T05:29:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:29:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151361+0000", 2024-09-14T05:29:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3979000000000004e-05 2024-09-14T05:29:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:29:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151383+0000", 2024-09-14T05:29:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2039999999999999e-05 2024-09-14T05:29:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:29:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.157542+0000", 2024-09-14T05:29:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006158915 2024-09-14T05:29:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:29:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:29:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:29:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.157677+0000", 2024-09-14T05:29:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013544300000000001 2024-09-14T05:29:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:29:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:29:59.945 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:29:59.947 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:29:59.947 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:29:59.946+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:29:59.988 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:30:00.048 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:00.047+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:00.096 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:00.097 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:00.097 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:00.097 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:00.106 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:30:00.149 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:00.149+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:00.153 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:00.153 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:00.170 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:00.170 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:00.186 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:00.186 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:00.204 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:00.205 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:00.210 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:00.210 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:00.210 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:00.210 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:00.210 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:00.219 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:30:00.226 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:00.227 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:00.246 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:00.247 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:00.254 INFO:teuthology.orchestra.run.smithi071.stdout:flattening image image_client.0-clone22 2024-09-14T05:30:00.260 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:00.259+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:00.265 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:00.265 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:00.282 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:00.282 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:00.350 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:00.350+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:00.361 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:00.362 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:00.370 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:30:00.410 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-14T05:30:00.452 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:00.451+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:00.486 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:00.486 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:00.498 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:30:00.582 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:00.581+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:00.645 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:00.645 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:00.653 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:30:00.654 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:00.652+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:00.661 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:00.661 INFO:teuthology.orchestra.run.smithi071.stdout:2748 write 0xb7f8801 thru 0xb808490 (0xfc90 bytes) 2024-09-14T05:30:00.671 INFO:teuthology.orchestra.run.smithi071.stdout:2750 read 0xde66ff3 thru 0xde71287 (0xa295 bytes) 2024-09-14T05:30:00.681 INFO:teuthology.orchestra.run.smithi071.stdout:2751 trunc from 0xee4bfeb to 0x73db01c 2024-09-14T05:30:00.712 INFO:teuthology.orchestra.run.smithi071.stdout:2753 read 0x529c277 thru 0x52a6a3e (0xa7c8 bytes) 2024-09-14T05:30:00.713 INFO:teuthology.orchestra.run.smithi071.stdout:2754 read 0x4ac7e4e thru 0x4acdebc (0x606f bytes) 2024-09-14T05:30:00.713 INFO:teuthology.orchestra.run.smithi071.stdout:2755 punch from 0x6f58213 to 0x6f62547, (0xa334 bytes) 2024-09-14T05:30:00.713 INFO:teuthology.orchestra.run.smithi071.stdout:2757 write 0x3dd687c thru 0x3dd9f33 (0x36b8 bytes) 2024-09-14T05:30:00.733 INFO:teuthology.orchestra.run.smithi071.stdout:2758 write 0xd94b541 thru 0xd956c12 (0xb6d2 bytes) 2024-09-14T05:30:00.742 INFO:teuthology.orchestra.run.smithi071.stdout:2759 writesame 0xe782d99 thru 0xe788679 data_size 0x1(0x58e1 bytes) 2024-09-14T05:30:00.742 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:00.743 INFO:teuthology.orchestra.run.smithi071.stdout:2761 write 0x7769fed thru 0x7773c03 (0x9c17 bytes) 2024-09-14T05:30:00.744 INFO:teuthology.orchestra.run.smithi071.stdout:2762 write 0xe1ae5b7 thru 0xe1b6429 (0x7e73 bytes) 2024-09-14T05:30:00.748 INFO:teuthology.orchestra.run.smithi071.stdout:2765 read 0x2671a80 thru 0x26738de (0x1e5f bytes) 2024-09-14T05:30:00.748 INFO:teuthology.orchestra.run.smithi071.stdout:2768 read 0x52853c3 thru 0x52891c2 (0x3e00 bytes) 2024-09-14T05:30:00.749 INFO:teuthology.orchestra.run.smithi071.stdout:2769 write 0xdde739e thru 0xddf241d (0xb080 bytes) 2024-09-14T05:30:00.750 INFO:teuthology.orchestra.run.smithi071.stdout:2771 write 0x789cbc7 thru 0x78a56a9 (0x8ae3 bytes) 2024-09-14T05:30:00.753 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:00.753+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:00.754 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:00.754 INFO:teuthology.orchestra.run.smithi071.stdout:2775 punch from 0x1c595bd to 0x1c643a4, (0xade7 bytes) 2024-09-14T05:30:00.754 INFO:teuthology.orchestra.run.smithi071.stdout:2776 punch from 0xb374182 to 0xb375736, (0x15b4 bytes) 2024-09-14T05:30:00.754 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:00.754 INFO:teuthology.orchestra.run.smithi071.stdout:2778 punch from 0x590b90e to 0x5916e4b, (0xb53d bytes) 2024-09-14T05:30:00.754 INFO:teuthology.orchestra.run.smithi071.stdout:2779 write 0xcdadc71 thru 0xcdbca5d (0xeded bytes) 2024-09-14T05:30:00.756 INFO:teuthology.orchestra.run.smithi071.stdout:2780 writesame 0x33cc518 thru 0x33d001b data_size 0x6(0x3b04 bytes) 2024-09-14T05:30:00.757 INFO:teuthology.orchestra.run.smithi071.stdout:2781 read 0x1113aaf thru 0x1118352 (0x48a4 bytes) 2024-09-14T05:30:00.758 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:00.758 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:00.758 INFO:teuthology.orchestra.run.smithi071.stdout:2785 read 0x72e2b6d thru 0x72e6344 (0x37d8 bytes) 2024-09-14T05:30:00.758 INFO:teuthology.orchestra.run.smithi071.stdout:2786 read 0x2c932b0 thru 0x2ca05b1 (0xd302 bytes) 2024-09-14T05:30:00.759 INFO:teuthology.orchestra.run.smithi071.stdout:2788 read 0xb6f36d9 thru 0xb6f5e04 (0x272c bytes) 2024-09-14T05:30:00.759 INFO:teuthology.orchestra.run.smithi071.stdout:2789 read 0x52e9ecf thru 0x52f2ea2 (0x8fd4 bytes) 2024-09-14T05:30:00.759 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:00.760 INFO:teuthology.orchestra.run.smithi071.stdout:2792 write 0xbac4e62 thru 0xbad338b (0xe52a bytes) 2024-09-14T05:30:00.763 INFO:teuthology.orchestra.run.smithi071.stdout:2793 read 0x78f16ed thru 0x78f27e8 (0x10fc bytes) 2024-09-14T05:30:00.763 INFO:teuthology.orchestra.run.smithi071.stdout:2794 read 0x7a177ac thru 0x7a1ca56 (0x52ab bytes) 2024-09-14T05:30:00.763 INFO:teuthology.orchestra.run.smithi071.stdout:2796 read 0x28d5fe2 thru 0x28e3f92 (0xdfb1 bytes) 2024-09-14T05:30:00.764 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:00.764 INFO:teuthology.orchestra.run.smithi071.stdout:2798 write 0x61b553 thru 0x6273f5 (0xbea3 bytes) 2024-09-14T05:30:00.773 INFO:teuthology.orchestra.run.smithi071.stdout:2800 write 0x4e9fa45 thru 0x4ea5265 (0x5821 bytes) 2024-09-14T05:30:00.781 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:00.782 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:00.782 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:00.782 INFO:teuthology.orchestra.run.smithi071.stdout:2806 write 0x84c76da thru 0x84d6f32 (0xf859 bytes) 2024-09-14T05:30:00.797 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:00.797 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:00.799 INFO:teuthology.orchestra.run.smithi071.stdout:2807 trunc from 0xe78867a to 0x39bdeba 2024-09-14T05:30:00.812 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:30:00.833 INFO:teuthology.orchestra.run.smithi071.stdout:2808 read 0x322f1f8 thru 0x323b8da (0xc6e3 bytes) 2024-09-14T05:30:00.833 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:00.834 INFO:teuthology.orchestra.run.smithi071.stdout:2811 read 0x86dddd thru 0x872921 (0x4b45 bytes) 2024-09-14T05:30:00.844 INFO:teuthology.orchestra.run.smithi071.stdout:2812 write 0x7772d4b thru 0x7781b2c (0xede2 bytes) 2024-09-14T05:30:00.851 INFO:teuthology.orchestra.run.smithi071.stdout:2813 read 0x519f7b3 thru 0x51a40e7 (0x4935 bytes) 2024-09-14T05:30:00.851 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:00.854 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:00.854+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:00.860 INFO:teuthology.orchestra.run.smithi071.stdout:2815 trunc from 0x7781b2d to 0x7dff6 2024-09-14T05:30:00.906 INFO:teuthology.orchestra.run.smithi071.stdout:2816 write 0xa1a0b10 thru 0xa1a957b (0x8a6c bytes) 2024-09-14T05:30:00.911 INFO:teuthology.orchestra.run.smithi071.stdout:2817 read 0x66b72c1 thru 0x66bc48c (0x51cc bytes) 2024-09-14T05:30:00.913 INFO:teuthology.orchestra.run.smithi071.stdout:2818 trunc from 0xa1a957c to 0xc995585 2024-09-14T05:30:00.918 INFO:teuthology.orchestra.run.smithi071.stdout:2819 punch from 0xc03fd0d to 0xc042c1b, (0x2f0e bytes) 2024-09-14T05:30:00.918 INFO:teuthology.orchestra.run.smithi071.stdout:2820 writesame 0x19049bc thru 0x1906c93 data_size 0xa(0x22d8 bytes) 2024-09-14T05:30:00.920 INFO:teuthology.orchestra.run.smithi071.stdout:2821 read 0xa01a3f4 thru 0xa0233b8 (0x8fc5 bytes) 2024-09-14T05:30:00.920 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:00.920 INFO:teuthology.orchestra.run.smithi071.stdout:2823 punch from 0xa72032b to 0xa728c53, (0x8928 bytes) 2024-09-14T05:30:00.920 INFO:teuthology.orchestra.run.smithi071.stdout:2824 read 0x81f8d41 thru 0x81fe836 (0x5af6 bytes) 2024-09-14T05:30:00.920 INFO:teuthology.orchestra.run.smithi071.stdout:2826 read 0x842d1b thru 0x851439 (0xe71f bytes) 2024-09-14T05:30:00.921 INFO:teuthology.orchestra.run.smithi071.stdout:2827 writesame 0x2a77d7c thru 0x2a7da49 data_size 0x7(0x5cce bytes) 2024-09-14T05:30:00.924 INFO:teuthology.orchestra.run.smithi071.stdout:2831 punch from 0x9916d5f to 0x991cd0a, (0x5fab bytes) 2024-09-14T05:30:00.924 INFO:teuthology.orchestra.run.smithi071.stdout:2833 read 0x2b369c8 thru 0x2b423a0 (0xb9d9 bytes) 2024-09-14T05:30:00.924 INFO:teuthology.orchestra.run.smithi071.stdout:2834 read 0x988243f thru 0x9887f87 (0x5b49 bytes) 2024-09-14T05:30:00.932 INFO:teuthology.orchestra.run.smithi071.stdout:2835 write 0xed2367f thru 0xed2fe09 (0xc78b bytes) 2024-09-14T05:30:00.935 INFO:teuthology.orchestra.run.smithi071.stdout:2836 punch from 0xb7028a to 0xb78b1b, (0x8891 bytes) 2024-09-14T05:30:00.935 INFO:teuthology.orchestra.run.smithi071.stdout:2837 write 0xb7a4533 thru 0xb7ab157 (0x6c25 bytes) 2024-09-14T05:30:00.939 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:00.955 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:00.954+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:00.958 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T05:30:00.958 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T05:30:00.958 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-14T05:30:00.958 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T05:30:00.958 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T05:30:00.959 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-14T05:30:00.960 INFO:teuthology.orchestra.run.smithi071.stdout:2839 trunc from 0xed2fe0a to 0x1a8ac2 2024-09-14T05:30:00.970 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:00.970 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:00.971 INFO:teuthology.orchestra.run.smithi071.stdout:2842 read 0x21462 thru 0x2322d (0x1dcc bytes) 2024-09-14T05:30:00.972 INFO:teuthology.orchestra.run.smithi071.stdout:2843 punch from 0xadad0 to 0xaef6f, (0x149f bytes) 2024-09-14T05:30:00.980 INFO:teuthology.orchestra.run.smithi071.stdout:2846 writesame 0x2efb710 thru 0x2f0810c data_size 0x7(0xc9fd bytes) 2024-09-14T05:30:00.982 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:30:00.983 INFO:teuthology.orchestra.run.smithi071.stdout:2847 trunc from 0x2f0810d to 0x52a20d9 2024-09-14T05:30:00.988 INFO:teuthology.orchestra.run.smithi071.stdout:2848 read 0x51850dd thru 0x518ba0f (0x6933 bytes) 2024-09-14T05:30:00.988 INFO:teuthology.orchestra.run.smithi071.stdout:2849 punch from 0x19c4472 to 0x19cc4f9, (0x8087 bytes) 2024-09-14T05:30:00.994 INFO:teuthology.orchestra.run.smithi071.stdout:2850 trunc from 0x52a20d9 to 0xa91eb6 2024-09-14T05:30:01.000 INFO:teuthology.orchestra.run.smithi071.stdout:2851 read 0x5a64e thru 0x5e35d (0x3d10 bytes) 2024-09-14T05:30:01.008 INFO:teuthology.orchestra.run.smithi071.stdout:2852 trunc from 0xa91eb6 to 0x6a26428 2024-09-14T05:30:01.015 INFO:teuthology.orchestra.run.smithi071.stdout:2853 trunc from 0x6a26428 to 0x7e4586b 2024-09-14T05:30:01.020 INFO:teuthology.orchestra.run.smithi071.stdout:2854 read 0x7c6b94d thru 0x7c70a73 (0x5127 bytes) 2024-09-14T05:30:01.021 INFO:teuthology.orchestra.run.smithi071.stdout:2856 read 0x9bbe30 thru 0x9bf91d (0x3aee bytes) 2024-09-14T05:30:01.021 INFO:teuthology.orchestra.run.smithi071.stdout:2857 write 0x3a4e165 thru 0x3a533c9 (0x5265 bytes) 2024-09-14T05:30:01.023 INFO:teuthology.orchestra.run.smithi071.stdout:2858 read 0x67b1044 thru 0x67bda92 (0xca4f bytes) 2024-09-14T05:30:01.023 INFO:teuthology.orchestra.run.smithi071.stdout:2860 punch from 0x25b3c09 to 0x25c142a, (0xd821 bytes) 2024-09-14T05:30:01.035 INFO:teuthology.orchestra.run.smithi071.stdout:2861 write 0xce4625f thru 0xce4cd9f (0x6b41 bytes) 2024-09-14T05:30:01.039 INFO:teuthology.orchestra.run.smithi071.stdout:2862 read 0xa97ba33 thru 0xa981aff (0x60cd bytes) 2024-09-14T05:30:01.039 INFO:teuthology.orchestra.run.smithi071.stdout:2865 write 0xbb70fa thru 0xbc0399 (0x92a0 bytes) 2024-09-14T05:30:01.042 INFO:teuthology.orchestra.run.smithi071.stdout:2871 punch from 0x78d1506 to 0x78db540, (0xa03a bytes) 2024-09-14T05:30:01.043 INFO:teuthology.orchestra.run.smithi071.stdout:2872 read 0x3c13ea7 thru 0x3c1e3ed (0xa547 bytes) 2024-09-14T05:30:01.043 INFO:teuthology.orchestra.run.smithi071.stdout:2873 write 0xc794bcf thru 0xc79f9d6 (0xae08 bytes) 2024-09-14T05:30:01.045 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:01.045 INFO:teuthology.orchestra.run.smithi071.stdout:2877 write 0xb8cea7f thru 0xb8d3809 (0x4d8b bytes) 2024-09-14T05:30:01.048 INFO:teuthology.orchestra.run.smithi071.stdout:2878 punch from 0x185a5be to 0x1861afa, (0x753c bytes) 2024-09-14T05:30:01.048 INFO:teuthology.orchestra.run.smithi071.stdout:2879 read 0xb37ec2f thru 0xb3853ae (0x6780 bytes) 2024-09-14T05:30:01.048 INFO:teuthology.orchestra.run.smithi071.stdout:2882 read 0x36fcb66 thru 0x36fe00b (0x14a6 bytes) 2024-09-14T05:30:01.049 INFO:teuthology.orchestra.run.smithi071.stdout:2883 write 0x90b7d87 thru 0x90c4d13 (0xcf8d bytes) 2024-09-14T05:30:01.056 INFO:teuthology.orchestra.run.smithi071.stdout:2884 read 0x3491bb2 thru 0x3496ceb (0x513a bytes) 2024-09-14T05:30:01.056 INFO:teuthology.orchestra.run.smithi071.stdout:2885 punch from 0x6360142 to 0x6363369, (0x3227 bytes) 2024-09-14T05:30:01.056 INFO:teuthology.orchestra.run.smithi071.stdout:2886 read 0x9752b33 thru 0x975bc20 (0x90ee bytes) 2024-09-14T05:30:01.056 INFO:teuthology.orchestra.run.smithi071.stdout:2887 read 0x33fc194 thru 0x3400c31 (0x4a9e bytes) 2024-09-14T05:30:01.056 INFO:teuthology.orchestra.run.smithi071.stdout:2888 write 0x44e3f2 thru 0x4537fd (0x540c bytes) 2024-09-14T05:30:01.056 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:01.055+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:01.057 INFO:teuthology.orchestra.run.smithi071.stdout:2889 punch from 0x7eed128 to 0x7eee691, (0x1569 bytes) 2024-09-14T05:30:01.057 INFO:teuthology.orchestra.run.smithi071.stdout:2890 punch from 0xc5858ed to 0xc595196, (0xf8a9 bytes) 2024-09-14T05:30:01.064 INFO:teuthology.orchestra.run.smithi071.stdout:2891 write 0xe90a0f9 thru 0xe90de1e (0x3d26 bytes) 2024-09-14T05:30:01.068 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:01.068 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T05:30:01.069 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:01.069 INFO:teuthology.orchestra.run.smithi071.stdout:2892 punch from 0xb144659 to 0xb147f07, (0x38ae bytes) 2024-09-14T05:30:01.069 INFO:teuthology.orchestra.run.smithi071.stdout:2893 read 0x919a9b1 thru 0x919d410 (0x2a60 bytes) 2024-09-14T05:30:01.069 INFO:teuthology.orchestra.run.smithi071.stdout:2894 write 0xaf940c2 thru 0xafa3464 (0xf3a3 bytes) 2024-09-14T05:30:01.077 INFO:teuthology.orchestra.run.smithi071.stdout:2895 read 0x40b0386 thru 0x40b6d78 (0x69f3 bytes) 2024-09-14T05:30:01.077 INFO:teuthology.orchestra.run.smithi071.stdout:2896 read 0xdcddf1b thru 0xdcecf82 (0xf068 bytes) 2024-09-14T05:30:01.078 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:01.078 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:01.078 INFO:teuthology.orchestra.run.smithi071.stdout:2899 write 0xa41b6d6 thru 0xa41ceb3 (0x17de bytes) 2024-09-14T05:30:01.080 INFO:teuthology.orchestra.run.smithi071.stdout:2900 read 0x3619a10 thru 0x3622c7a (0x926b bytes) 2024-09-14T05:30:01.093 INFO:teuthology.orchestra.run.smithi071.stdout:2901 trunc from 0xe90de1f to 0x49b9e77 2024-09-14T05:30:01.097 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:01.098 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:01.104 INFO:teuthology.orchestra.run.smithi071.stdout:2903 read 0x3e9b5aa thru 0x3ea1644 (0x609b bytes) 2024-09-14T05:30:01.104 INFO:teuthology.orchestra.run.smithi071.stdout:2905 read 0x40ae8f7 thru 0x40b438a (0x5a94 bytes) 2024-09-14T05:30:01.104 INFO:teuthology.orchestra.run.smithi071.stdout:2906 punch from 0x1dec969 to 0x1df5936, (0x8fcd bytes) 2024-09-14T05:30:01.104 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:01.104 INFO:teuthology.orchestra.run.smithi071.stdout:2908 clone 24 order 21 su 1048576 sc 8 2024-09-14T05:30:01.107 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:30:01.156 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:01.156+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:01.210 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T05:30:01.255 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:01.256 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:01.264 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:30:01.270 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:01.256+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:01.319 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone23 from 0x49b9e77 (overlap 0x7dff6) to 0x4b993 2024-09-14T05:30:01.358 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:01.357+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:01.378 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-14T05:30:01.389 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:01.389 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:01.401 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:30:01.485 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:01.484+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:01.552 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:01.552 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:01.560 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:01.559+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:01.561 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:30:01.637 INFO:teuthology.orchestra.run.smithi071.stdout:2910 read 0x4ff065 thru 0x507de4 (0x8d80 bytes) 2024-09-14T05:30:01.638 INFO:teuthology.orchestra.run.smithi071.stdout:2911 punch from 0x1a39e5e to 0x1a495b2, (0xf754 bytes) 2024-09-14T05:30:01.643 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:01.648 INFO:teuthology.orchestra.run.smithi071.stdout:2913 trunc from 0x49b9e77 to 0x8e9c60e 2024-09-14T05:30:01.653 INFO:teuthology.orchestra.run.smithi071.stdout:2914 write 0x2fca718 thru 0x2fd05c5 (0x5eae bytes) 2024-09-14T05:30:01.656 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:01.657 INFO:teuthology.orchestra.run.smithi071.stdout:2916 read 0x64cfc43 thru 0x64d5f36 (0x62f4 bytes) 2024-09-14T05:30:01.661 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:01.661+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:01.669 INFO:teuthology.orchestra.run.smithi071.stdout:2918 writesame 0xe5692a4 thru 0xe5789f8 data_size 0x1(0xf755 bytes) 2024-09-14T05:30:01.669 INFO:teuthology.orchestra.run.smithi071.stdout:2920 write 0x4dcf36e thru 0x4dd0993 (0x1626 bytes) 2024-09-14T05:30:01.674 INFO:teuthology.orchestra.run.smithi071.stdout:2921 trunc from 0xe5789f9 to 0xc952bac 2024-09-14T05:30:01.679 INFO:teuthology.orchestra.run.smithi071.stdout:2922 clone 25 order 20 su 524288 sc 10 2024-09-14T05:30:01.710 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:01.710 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:01.718 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:30:01.761 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:01.761+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:01.862 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:01.862 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:01.862 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:01.861+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:01.871 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:30:01.963 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:01.963+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:02.014 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:02.015 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:02.023 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:30:02.064 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:02.064+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:02.166 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:02.165+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:02.166 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:02.166 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:02.175 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:30:02.208 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-14T05:30:02.266 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:02.266+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:02.276 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone24 from 0xc952bac (overlap 0x49b9e77) to 0x1d7d8fc 2024-09-14T05:30:02.294 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:02.295 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:02.295 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:30:02.305 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:30:02.367 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:02.366+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:02.371 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-14T05:30:02.374 INFO:teuthology.orchestra.run.smithi071.stdout:2927 flatten 2024-09-14T05:30:02.405 INFO:teuthology.orchestra.run.smithi071.stdout:2928 write 0x89dfd1e thru 0x89e5ab6 (0x5d99 bytes) 2024-09-14T05:30:02.409 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:02.409 INFO:teuthology.orchestra.run.smithi071.stdout:2932 writesame 0x11f811d thru 0x11f900e data_size 0x1(0xef2 bytes) 2024-09-14T05:30:02.414 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:02.414 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:02.415 INFO:teuthology.orchestra.run.smithi071.stdout:2933 trunc from 0xc952bac to 0xe11151e 2024-09-14T05:30:02.420 INFO:teuthology.orchestra.run.smithi071.stdout:2934 read 0x1f45ca2 thru 0x1f49ebe (0x421d bytes) 2024-09-14T05:30:02.420 INFO:teuthology.orchestra.run.smithi071.stdout:2935 read 0x5f7c9ba thru 0x5f8c04c (0xf693 bytes) 2024-09-14T05:30:02.420 INFO:teuthology.orchestra.run.smithi071.stdout:2937 read 0x1174c5e thru 0x1182e98 (0xe23b bytes) 2024-09-14T05:30:02.420 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:02.420 INFO:teuthology.orchestra.run.smithi071.stdout:2939 write 0x6eaa4c0 thru 0x6eada43 (0x3584 bytes) 2024-09-14T05:30:02.423 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:30:02.424 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:02.424 INFO:teuthology.orchestra.run.smithi071.stdout:2941 writesame 0x155c2da thru 0x15679aa data_size 0x1(0xb6d1 bytes) 2024-09-14T05:30:02.432 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:02.432 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:02.444 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:02.444 INFO:teuthology.orchestra.run.smithi071.stdout:2943 read 0x67360bc thru 0x6740379 (0xa2be bytes) 2024-09-14T05:30:02.444 INFO:teuthology.orchestra.run.smithi071.stdout:2945 clone 26 order 20 su 262144 sc 3 2024-09-14T05:30:02.450 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:02.450 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:02.466 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:02.466 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:02.482 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:02.482 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:02.500 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:02.500 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:02.508 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:02.508+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:02.516 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:02.516 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:02.533 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:02.533 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:02.549 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:02.549 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:02.569 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:02.568+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:02.571 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:02.571 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:02.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:02.571 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:02.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.572 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3523 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6062080~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.572 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:02.572 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.055657444000001, 2024-09-14T05:30:02.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096911030999999995, 2024-09-14T05:30:02.572 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.572 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.572 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.572 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.572 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.572 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.572 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3523 2024-09-14T05:30:02.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.572 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:02.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:02.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514359+0000", 2024-09-14T05:30:02.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9299999999999999e-07 2024-09-14T05:30:02.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514361+0000", 2024-09-14T05:30:02.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:02.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514366+0000", 2024-09-14T05:30:02.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1660000000000002e-06 2024-09-14T05:30:02.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514371+0000", 2024-09-14T05:30:02.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.648e-06 2024-09-14T05:30:02.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.610954+0000", 2024-09-14T05:30:02.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096582170999999994 2024-09-14T05:30:02.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611002+0000", 2024-09-14T05:30:02.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7923999999999997e-05 2024-09-14T05:30:02.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611269+0000", 2024-09-14T05:30:02.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026789499999999998 2024-09-14T05:30:02.576 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.576 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.576 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.577 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3524 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6553600~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.577 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:02.577 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.055638252000001, 2024-09-14T05:30:02.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097263347999999999, 2024-09-14T05:30:02.577 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.577 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.577 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.577 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.577 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.577 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.577 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3524 2024-09-14T05:30:02.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.578 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:02.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:02.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:02.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:02.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514380+0000", 2024-09-14T05:30:02.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6500000000000001e-06 2024-09-14T05:30:02.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514386+0000", 2024-09-14T05:30:02.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7360000000000004e-06 2024-09-14T05:30:02.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514391+0000", 2024-09-14T05:30:02.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2830000000000003e-06 2024-09-14T05:30:02.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611295+0000", 2024-09-14T05:30:02.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096904814000000006 2024-09-14T05:30:02.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611343+0000", 2024-09-14T05:30:02.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7506999999999998e-05 2024-09-14T05:30:02.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611641+0000", 2024-09-14T05:30:02.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029798800000000001 2024-09-14T05:30:02.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.581 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.581 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3525 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7045120~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.582 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:02.582 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.055616364999999, 2024-09-14T05:30:02.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097644422999999994, 2024-09-14T05:30:02.582 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.582 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.582 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.582 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3525 2024-09-14T05:30:02.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.582 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:02.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:02.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514400+0000", 2024-09-14T05:30:02.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.77e-07 2024-09-14T05:30:02.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514402+0000", 2024-09-14T05:30:02.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6530000000000001e-06 2024-09-14T05:30:02.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514411+0000", 2024-09-14T05:30:02.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7429999999999997e-06 2024-09-14T05:30:02.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514417+0000", 2024-09-14T05:30:02.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2249999999999997e-06 2024-09-14T05:30:02.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611668+0000", 2024-09-14T05:30:02.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097250681000000005 2024-09-14T05:30:02.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611716+0000", 2024-09-14T05:30:02.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7336000000000002e-05 2024-09-14T05:30:02.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612044+0000", 2024-09-14T05:30:02.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032840799999999998 2024-09-14T05:30:02.586 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.586 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.586 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.586 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3526 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7536640~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.586 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:02.586 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.055591881000002, 2024-09-14T05:30:02.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097943884999999994, 2024-09-14T05:30:02.586 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.587 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.587 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.587 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.587 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.587 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.587 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3526 2024-09-14T05:30:02.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.587 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:02.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:02.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:02.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0400000000000002e-07 2024-09-14T05:30:02.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514426+0000", 2024-09-14T05:30:02.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:02.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514432+0000", 2024-09-14T05:30:02.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.105e-06 2024-09-14T05:30:02.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514438+0000", 2024-09-14T05:30:02.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5149999999999997e-06 2024-09-14T05:30:02.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612072+0000", 2024-09-14T05:30:02.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097633947999999998 2024-09-14T05:30:02.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612120+0000", 2024-09-14T05:30:02.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8458999999999998e-05 2024-09-14T05:30:02.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612368+0000", 2024-09-14T05:30:02.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024761999999999999 2024-09-14T05:30:02.591 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.591 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.591 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.591 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3527 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 8028160~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.591 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:02.591 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.055571168, 2024-09-14T05:30:02.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098244955999999994, 2024-09-14T05:30:02.591 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.591 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.591 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.591 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.591 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.592 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.592 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3527 2024-09-14T05:30:02.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.592 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:02.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:02.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519842+0000", 2024-09-14T05:30:02.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053975230000000004 2024-09-14T05:30:02.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519851+0000", 2024-09-14T05:30:02.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5599999999999994e-06 2024-09-14T05:30:02.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519858+0000", 2024-09-14T05:30:02.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2409999999999998e-06 2024-09-14T05:30:02.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519863+0000", 2024-09-14T05:30:02.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:30:02.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612395+0000", 2024-09-14T05:30:02.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092532552000000004 2024-09-14T05:30:02.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612442+0000", 2024-09-14T05:30:02.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6486000000000002e-05 2024-09-14T05:30:02.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612690+0000", 2024-09-14T05:30:02.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024791599999999999 2024-09-14T05:30:02.595 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.595 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.595 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.596 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3603 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5177344~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.596 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:02.596 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.050018723000001, 2024-09-14T05:30:02.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096796436, 2024-09-14T05:30:02.596 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.596 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.596 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.597 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3603 2024-09-14T05:30:02.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.597 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:02.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:02.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:02.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1899999999999999e-07 2024-09-14T05:30:02.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519998+0000", 2024-09-14T05:30:02.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5000000000000001e-07 2024-09-14T05:30:02.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520002+0000", 2024-09-14T05:30:02.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7299999999999999e-06 2024-09-14T05:30:02.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520004+0000", 2024-09-14T05:30:02.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.429e-06 2024-09-14T05:30:02.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616493+0000", 2024-09-14T05:30:02.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096488345000000003 2024-09-14T05:30:02.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616544+0000", 2024-09-14T05:30:02.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1724000000000001e-05 2024-09-14T05:30:02.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616794+0000", 2024-09-14T05:30:02.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000249139 2024-09-14T05:30:02.600 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.600 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.600 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.600 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3604 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5668864~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.600 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:02.600 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.050004085000001, 2024-09-14T05:30:02.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108222999999994, 2024-09-14T05:30:02.601 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.601 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.601 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.601 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3604 2024-09-14T05:30:02.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.601 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:02.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:02.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:02.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7000000000000001e-07 2024-09-14T05:30:02.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520013+0000", 2024-09-14T05:30:02.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.389e-06 2024-09-14T05:30:02.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520018+0000", 2024-09-14T05:30:02.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3019999999999996e-06 2024-09-14T05:30:02.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520026+0000", 2024-09-14T05:30:02.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6170000000000007e-06 2024-09-14T05:30:02.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616816+0000", 2024-09-14T05:30:02.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096789766999999999 2024-09-14T05:30:02.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616871+0000", 2024-09-14T05:30:02.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4811000000000001e-05 2024-09-14T05:30:02.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617120+0000", 2024-09-14T05:30:02.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024906700000000001 2024-09-14T05:30:02.604 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.605 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.605 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.605 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3605 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6160384~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.605 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:02.605 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.049985625000001, 2024-09-14T05:30:02.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097423945999999997, 2024-09-14T05:30:02.605 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.605 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.605 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.605 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.605 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.606 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3605 2024-09-14T05:30:02.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.606 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:02.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:02.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:02.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3999999999999998e-07 2024-09-14T05:30:02.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520032+0000", 2024-09-14T05:30:02.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.083e-06 2024-09-14T05:30:02.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520036+0000", 2024-09-14T05:30:02.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.019e-06 2024-09-14T05:30:02.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520038+0000", 2024-09-14T05:30:02.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4849999999999999e-06 2024-09-14T05:30:02.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617147+0000", 2024-09-14T05:30:02.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108665999999996 2024-09-14T05:30:02.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617199+0000", 2024-09-14T05:30:02.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2558000000000001e-05 2024-09-14T05:30:02.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617454+0000", 2024-09-14T05:30:02.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025489499999999999 2024-09-14T05:30:02.609 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.609 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.609 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.609 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3606 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6651904~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.610 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:02.610 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.049974902999999, 2024-09-14T05:30:02.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097745705000000002, 2024-09-14T05:30:02.610 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.610 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.610 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.610 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3606 2024-09-14T05:30:02.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.610 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:02.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:02.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:02.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-14T05:30:02.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520042+0000", 2024-09-14T05:30:02.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4799999999999997e-07 2024-09-14T05:30:02.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520046+0000", 2024-09-14T05:30:02.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8149999999999999e-06 2024-09-14T05:30:02.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520049+0000", 2024-09-14T05:30:02.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.576e-06 2024-09-14T05:30:02.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617476+0000", 2024-09-14T05:30:02.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097427558999999997 2024-09-14T05:30:02.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617522+0000", 2024-09-14T05:30:02.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5927999999999998e-05 2024-09-14T05:30:02.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617787+0000", 2024-09-14T05:30:02.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000264663 2024-09-14T05:30:02.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.614 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.614 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3607 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7143424~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.614 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:02.614 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.049964375999998, 2024-09-14T05:30:02.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097871125000000003, 2024-09-14T05:30:02.614 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.615 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.615 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.615 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3607 2024-09-14T05:30:02.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.615 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:02.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:02.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520052+0000", 2024-09-14T05:30:02.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.03e-07 2024-09-14T05:30:02.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520053+0000", 2024-09-14T05:30:02.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.85e-07 2024-09-14T05:30:02.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520056+0000", 2024-09-14T05:30:02.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9140000000000001e-06 2024-09-14T05:30:02.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520059+0000", 2024-09-14T05:30:02.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4600000000000002e-06 2024-09-14T05:30:02.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617809+0000", 2024-09-14T05:30:02.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097750216000000001 2024-09-14T05:30:02.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617864+0000", 2024-09-14T05:30:02.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5211999999999999e-05 2024-09-14T05:30:02.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617923+0000", 2024-09-14T05:30:02.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8235000000000001e-05 2024-09-14T05:30:02.619 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.619 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.619 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.619 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3608 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7634944~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.619 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:02.619 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.049954068000002, 2024-09-14T05:30:02.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098199212999999994, 2024-09-14T05:30:02.619 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.619 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.620 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.620 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.620 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.620 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.620 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3608 2024-09-14T05:30:02.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.620 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:02.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:02.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525953+0000", 2024-09-14T05:30:02.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058911850000000002 2024-09-14T05:30:02.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525957+0000", 2024-09-14T05:30:02.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2379999999999999e-06 2024-09-14T05:30:02.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525967+0000", 2024-09-14T05:30:02.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5389999999999996e-06 2024-09-14T05:30:02.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525972+0000", 2024-09-14T05:30:02.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.541e-06 2024-09-14T05:30:02.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617945+0000", 2024-09-14T05:30:02.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091972597000000003 2024-09-14T05:30:02.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617992+0000", 2024-09-14T05:30:02.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7397e-05 2024-09-14T05:30:02.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.618261+0000", 2024-09-14T05:30:02.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026871600000000001 2024-09-14T05:30:02.623 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.624 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.624 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.624 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3640 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6586368~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.624 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:02.624 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.043846225999999, 2024-09-14T05:30:02.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097068580000000002, 2024-09-14T05:30:02.624 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.624 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.624 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.624 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.624 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.625 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.625 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3640 2024-09-14T05:30:02.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.625 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:02.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:02.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:02.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.23e-07 2024-09-14T05:30:02.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526171+0000", 2024-09-14T05:30:02.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:30:02.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526175+0000", 2024-09-14T05:30:02.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8369999999999999e-06 2024-09-14T05:30:02.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526177+0000", 2024-09-14T05:30:02.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4959999999999999e-06 2024-09-14T05:30:02.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622847+0000", 2024-09-14T05:30:02.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096669790000000005 2024-09-14T05:30:02.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622902+0000", 2024-09-14T05:30:02.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5157000000000001e-05 2024-09-14T05:30:02.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623238+0000", 2024-09-14T05:30:02.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033603399999999997 2024-09-14T05:30:02.628 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.628 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.628 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.628 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3641 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7077888~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.629 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:02.629 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.043835716, 2024-09-14T05:30:02.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097371141999999994, 2024-09-14T05:30:02.629 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.629 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.629 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.629 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.629 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.629 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.629 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3641 2024-09-14T05:30:02.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.629 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:02.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:02.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531851+0000", 2024-09-14T05:30:02.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005671265 2024-09-14T05:30:02.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531855+0000", 2024-09-14T05:30:02.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0849999999999999e-06 2024-09-14T05:30:02.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531867+0000", 2024-09-14T05:30:02.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1028e-05 2024-09-14T05:30:02.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531872+0000", 2024-09-14T05:30:02.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000003e-06 2024-09-14T05:30:02.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623251+0000", 2024-09-14T05:30:02.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091378980999999998 2024-09-14T05:30:02.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623295+0000", 2024-09-14T05:30:02.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3881999999999998e-05 2024-09-14T05:30:02.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623551+0000", 2024-09-14T05:30:02.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025636099999999998 2024-09-14T05:30:02.633 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.633 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.633 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.633 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3721 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.633 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:02.633 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.037977440999999, 2024-09-14T05:30:02.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096806472000000005, 2024-09-14T05:30:02.633 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.633 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.633 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.633 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.634 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.634 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.634 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3721 2024-09-14T05:30:02.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.634 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:02.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:02.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532039+0000", 2024-09-14T05:30:02.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.05e-07 2024-09-14T05:30:02.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532040+0000", 2024-09-14T05:30:02.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9099999999999991e-07 2024-09-14T05:30:02.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532043+0000", 2024-09-14T05:30:02.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6610000000000002e-06 2024-09-14T05:30:02.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532046+0000", 2024-09-14T05:30:02.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4080000000000001e-06 2024-09-14T05:30:02.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628545+0000", 2024-09-14T05:30:02.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096499615999999996 2024-09-14T05:30:02.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628592+0000", 2024-09-14T05:30:02.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6907000000000003e-05 2024-09-14T05:30:02.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628845+0000", 2024-09-14T05:30:02.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025268399999999998 2024-09-14T05:30:02.637 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.637 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.637 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.637 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3722 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6193152~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.638 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:02.638 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.037967361, 2024-09-14T05:30:02.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097146587000000006, 2024-09-14T05:30:02.638 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.638 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.638 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.638 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.638 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.638 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.638 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3722 2024-09-14T05:30:02.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.638 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:02.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:02.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538191+0000", 2024-09-14T05:30:02.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061429969999999999 2024-09-14T05:30:02.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538200+0000", 2024-09-14T05:30:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7069999999999998e-06 2024-09-14T05:30:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538212+0000", 2024-09-14T05:30:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1504e-05 2024-09-14T05:30:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538219+0000", 2024-09-14T05:30:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7500000000000003e-06 2024-09-14T05:30:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628869+0000", 2024-09-14T05:30:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090649224 2024-09-14T05:30:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628919+0000", 2024-09-14T05:30:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0144e-05 2024-09-14T05:30:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.629195+0000", 2024-09-14T05:30:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027626099999999998 2024-09-14T05:30:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3758 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7110656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.031373145, 2024-09-14T05:30:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096977544999999998, 2024-09-14T05:30:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3758 2024-09-14T05:30:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538645+0000", 2024-09-14T05:30:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.843e-06 2024-09-14T05:30:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538651+0000", 2024-09-14T05:30:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6169999999999998e-06 2024-09-14T05:30:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538655+0000", 2024-09-14T05:30:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5399999999999997e-06 2024-09-14T05:30:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635225+0000", 2024-09-14T05:30:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096570149999999993 2024-09-14T05:30:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635272+0000", 2024-09-14T05:30:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7199e-05 2024-09-14T05:30:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635620+0000", 2024-09-14T05:30:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034782600000000003 2024-09-14T05:30:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3759 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.031355233999999, 2024-09-14T05:30:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097335331999999997, 2024-09-14T05:30:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3759 2024-09-14T05:30:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544013+0000", 2024-09-14T05:30:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053520829999999997 2024-09-14T05:30:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544016+0000", 2024-09-14T05:30:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6660000000000001e-06 2024-09-14T05:30:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544026+0000", 2024-09-14T05:30:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7089999999999997e-06 2024-09-14T05:30:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544043+0000", 2024-09-14T05:30:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833999999999998e-05 2024-09-14T05:30:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635646+0000", 2024-09-14T05:30:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091603544999999995 2024-09-14T05:30:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635696+0000", 2024-09-14T05:30:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9307000000000001e-05 2024-09-14T05:30:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635996+0000", 2024-09-14T05:30:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030018800000000001 2024-09-14T05:30:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3838 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6225920~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.025693554, 2024-09-14T05:30:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096894235999999995, 2024-09-14T05:30:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3838 2024-09-14T05:30:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544323+0000", 2024-09-14T05:30:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-14T05:30:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544325+0000", 2024-09-14T05:30:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.75e-06 2024-09-14T05:30:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544331+0000", 2024-09-14T05:30:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5019999999999998e-06 2024-09-14T05:30:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544335+0000", 2024-09-14T05:30:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2989999999999998e-06 2024-09-14T05:30:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640869+0000", 2024-09-14T05:30:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096533351000000003 2024-09-14T05:30:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640919+0000", 2024-09-14T05:30:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999999e-05 2024-09-14T05:30:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641217+0000", 2024-09-14T05:30:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029717600000000002 2024-09-14T05:30:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3839 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.025674616, 2024-09-14T05:30:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097240339999999995, 2024-09-14T05:30:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3839 2024-09-14T05:30:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549834+0000", 2024-09-14T05:30:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054927819999999999 2024-09-14T05:30:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549838+0000", 2024-09-14T05:30:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8689999999999997e-06 2024-09-14T05:30:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549848+0000", 2024-09-14T05:30:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0154e-05 2024-09-14T05:30:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549853+0000", 2024-09-14T05:30:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9740000000000001e-06 2024-09-14T05:30:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641245+0000", 2024-09-14T05:30:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091392245999999996 2024-09-14T05:30:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641295+0000", 2024-09-14T05:30:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0018999999999997e-05 2024-09-14T05:30:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641582+0000", 2024-09-14T05:30:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028629600000000001 2024-09-14T05:30:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3868 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.020012038000001, 2024-09-14T05:30:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096859555999999999, 2024-09-14T05:30:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3868 2024-09-14T05:30:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556173+0000", 2024-09-14T05:30:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006169038 2024-09-14T05:30:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556177+0000", 2024-09-14T05:30:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0010000000000003e-06 2024-09-14T05:30:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556188+0000", 2024-09-14T05:30:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0675e-05 2024-09-14T05:30:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556199+0000", 2024-09-14T05:30:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.155e-05 2024-09-14T05:30:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646511+0000", 2024-09-14T05:30:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090312187000000002 2024-09-14T05:30:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646563+0000", 2024-09-14T05:30:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2219000000000003e-05 2024-09-14T05:30:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646863+0000", 2024-09-14T05:30:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029988600000000002 2024-09-14T05:30:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:02.665 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:02.666 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:30:02.669 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:02.668+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:02.770 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:02.769+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:02.809 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:02.809 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:02.818 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:30:02.870 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:02.870+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:02.920 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:02.921 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:02.921 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:02.921 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:02.921 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:02.929 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:30:02.971 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:02.970+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:03.032 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1989 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 16326080~54496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.469929673000003, 2024-09-14T05:30:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096664322999999996, 2024-09-14T05:30:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1989 2024-09-14T05:30:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:03.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561455+0000", 2024-09-14T05:30:03.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1699999999999997e-07 2024-09-14T05:30:03.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590703+0000", 2024-09-14T05:30:03.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029248225999999999 2024-09-14T05:30:03.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590732+0000", 2024-09-14T05:30:03.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8679000000000001e-05 2024-09-14T05:30:03.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590759+0000", 2024-09-14T05:30:03.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7319999999999999e-05 2024-09-14T05:30:03.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590835+0000", 2024-09-14T05:30:03.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5543999999999994e-05 2024-09-14T05:30:03.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590956+0000", 2024-09-14T05:30:03.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012168 2024-09-14T05:30:03.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 1", 2024-09-14T05:30:03.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.591261+0000", 2024-09-14T05:30:03.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030484799999999998 2024-09-14T05:30:03.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:03.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.637682+0000", 2024-09-14T05:30:03.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046420550999999997 2024-09-14T05:30:03.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:03.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.657912+0000", 2024-09-14T05:30:03.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020230011999999999 2024-09-14T05:30:03.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658007+0000", 2024-09-14T05:30:03.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5814000000000001e-05 2024-09-14T05:30:03.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658118+0000", 2024-09-14T05:30:03.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110832 2024-09-14T05:30:03.039 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.039 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.039 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.039 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2033 3.1e 3:7e47f6eb:::rbd_data.112d1f36d097.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30853661~63531] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:03.039 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:03.039 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.358655007000003, 2024-09-14T05:30:03.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065167630000000004, 2024-09-14T05:30:03.039 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.039 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.040 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.040 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2033 2024-09-14T05:30:03.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.040 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:03.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:03.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672730+0000", 2024-09-14T05:30:03.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.393e-06 2024-09-14T05:30:03.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691289+0000", 2024-09-14T05:30:03.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018558991 2024-09-14T05:30:03.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691316+0000", 2024-09-14T05:30:03.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7498999999999999e-05 2024-09-14T05:30:03.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691331+0000", 2024-09-14T05:30:03.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4652e-05 2024-09-14T05:30:03.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691397+0000", 2024-09-14T05:30:03.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6112000000000007e-05 2024-09-14T05:30:03.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691509+0000", 2024-09-14T05:30:03.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111599 2024-09-14T05:30:03.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:03.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.692253+0000", 2024-09-14T05:30:03.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074456200000000004 2024-09-14T05:30:03.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:03.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.719140+0000", 2024-09-14T05:30:03.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026886542999999999 2024-09-14T05:30:03.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:03.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737789+0000", 2024-09-14T05:30:03.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018649203999999999 2024-09-14T05:30:03.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737846+0000", 2024-09-14T05:30:03.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6501e-05 2024-09-14T05:30:03.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737896+0000", 2024-09-14T05:30:03.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0574000000000001e-05 2024-09-14T05:30:03.045 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.045 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.045 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.045 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3008 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13729792~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.045 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:03.045 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.727731136999999, 2024-09-14T05:30:03.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061007651000000003, 2024-09-14T05:30:03.045 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.046 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.046 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.046 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3008 2024-09-14T05:30:03.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.046 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:03.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:03.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303653+0000", 2024-09-14T05:30:03.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0699999999999999e-07 2024-09-14T05:30:03.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303654+0000", 2024-09-14T05:30:03.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-07 2024-09-14T05:30:03.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303657+0000", 2024-09-14T05:30:03.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2890000000000002e-06 2024-09-14T05:30:03.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303661+0000", 2024-09-14T05:30:03.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3290000000000004e-06 2024-09-14T05:30:03.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364233+0000", 2024-09-14T05:30:03.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060572134 2024-09-14T05:30:03.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364315+0000", 2024-09-14T05:30:03.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1549000000000001e-05 2024-09-14T05:30:03.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364660+0000", 2024-09-14T05:30:03.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034525300000000001 2024-09-14T05:30:03.049 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.049 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.050 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.050 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3009 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14221312~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.050 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:03.050 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.727717046999999, 2024-09-14T05:30:03.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061445063000000001, 2024-09-14T05:30:03.050 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.050 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.050 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.050 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3009 2024-09-14T05:30:03.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.051 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:03.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:03.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:03.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:03.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303669+0000", 2024-09-14T05:30:03.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7150000000000001e-06 2024-09-14T05:30:03.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303675+0000", 2024-09-14T05:30:03.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5339999999999996e-06 2024-09-14T05:30:03.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303681+0000", 2024-09-14T05:30:03.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3639999999999998e-06 2024-09-14T05:30:03.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364705+0000", 2024-09-14T05:30:03.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061024123 2024-09-14T05:30:03.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364786+0000", 2024-09-14T05:30:03.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1378999999999995e-05 2024-09-14T05:30:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365112+0000", 2024-09-14T05:30:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032558700000000001 2024-09-14T05:30:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3010 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14712832~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.727696182999999, 2024-09-14T05:30:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061834630000000002, 2024-09-14T05:30:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3010 2024-09-14T05:30:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:03.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:03.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303688+0000", 2024-09-14T05:30:03.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:30:03.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303690+0000", 2024-09-14T05:30:03.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9460000000000001e-06 2024-09-14T05:30:03.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303696+0000", 2024-09-14T05:30:03.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2230000000000004e-06 2024-09-14T05:30:03.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303700+0000", 2024-09-14T05:30:03.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7290000000000002e-06 2024-09-14T05:30:03.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365163+0000", 2024-09-14T05:30:03.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061463093000000003 2024-09-14T05:30:03.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365214+0000", 2024-09-14T05:30:03.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0977999999999999e-05 2024-09-14T05:30:03.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365522+0000", 2024-09-14T05:30:03.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000308227 2024-09-14T05:30:03.059 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.059 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.059 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.059 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3241 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.059 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:03.059 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.702021761000001, 2024-09-14T05:30:03.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061041892, 2024-09-14T05:30:03.059 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.059 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.060 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.060 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.060 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.060 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.060 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3241 2024-09-14T05:30:03.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.060 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:03.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:03.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329496+0000", 2024-09-14T05:30:03.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013421799999999999 2024-09-14T05:30:03.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329499+0000", 2024-09-14T05:30:03.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8459999999999999e-06 2024-09-14T05:30:03.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329505+0000", 2024-09-14T05:30:03.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4320000000000004e-06 2024-09-14T05:30:03.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329509+0000", 2024-09-14T05:30:03.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0339999999999998e-06 2024-09-14T05:30:03.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.389947+0000", 2024-09-14T05:30:03.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060438093999999998 2024-09-14T05:30:03.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390038+0000", 2024-09-14T05:30:03.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0253999999999993e-05 2024-09-14T05:30:03.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390404+0000", 2024-09-14T05:30:03.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036601400000000001 2024-09-14T05:30:03.063 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.063 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.063 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.064 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3242 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.064 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:03.064 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.701866852999999, 2024-09-14T05:30:03.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061282426000000001, 2024-09-14T05:30:03.064 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.064 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.064 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.064 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.064 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.064 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.064 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3242 2024-09-14T05:30:03.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.065 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:03.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:03.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:03.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9900000000000001e-07 2024-09-14T05:30:03.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329518+0000", 2024-09-14T05:30:03.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.328e-06 2024-09-14T05:30:03.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329522+0000", 2024-09-14T05:30:03.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.563e-06 2024-09-14T05:30:03.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329525+0000", 2024-09-14T05:30:03.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7379999999999999e-06 2024-09-14T05:30:03.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390437+0000", 2024-09-14T05:30:03.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060912463 2024-09-14T05:30:03.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390498+0000", 2024-09-14T05:30:03.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0615999999999997e-05 2024-09-14T05:30:03.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390799+0000", 2024-09-14T05:30:03.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030131899999999998 2024-09-14T05:30:03.068 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.068 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.068 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.068 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3243 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.068 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:03.068 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.701855067, 2024-09-14T05:30:03.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061728980000000003, 2024-09-14T05:30:03.069 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.069 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.069 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.069 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.069 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.069 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.069 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3243 2024-09-14T05:30:03.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.069 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:03.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:03.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329692+0000", 2024-09-14T05:30:03.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000163948 2024-09-14T05:30:03.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329696+0000", 2024-09-14T05:30:03.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5540000000000002e-06 2024-09-14T05:30:03.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329706+0000", 2024-09-14T05:30:03.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7960000000000007e-06 2024-09-14T05:30:03.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329712+0000", 2024-09-14T05:30:03.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6109999999999999e-06 2024-09-14T05:30:03.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390861+0000", 2024-09-14T05:30:03.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061148767 2024-09-14T05:30:03.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390942+0000", 2024-09-14T05:30:03.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1266000000000004e-05 2024-09-14T05:30:03.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391258+0000", 2024-09-14T05:30:03.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031503800000000002 2024-09-14T05:30:03.072 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.072 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.073 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.073 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3244 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14778368~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.073 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:03.073 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.701660018999998, 2024-09-14T05:30:03.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062050594000000001, 2024-09-14T05:30:03.073 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.073 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.073 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.073 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.073 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.073 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.073 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3244 2024-09-14T05:30:03.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.074 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:03.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:03.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329916+0000", 2024-09-14T05:30:03.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019290799999999999 2024-09-14T05:30:03.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329919+0000", 2024-09-14T05:30:03.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9019999999999999e-06 2024-09-14T05:30:03.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329926+0000", 2024-09-14T05:30:03.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9e-06 2024-09-14T05:30:03.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329932+0000", 2024-09-14T05:30:03.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.088e-06 2024-09-14T05:30:03.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391299+0000", 2024-09-14T05:30:03.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061366136000000002 2024-09-14T05:30:03.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391391+0000", 2024-09-14T05:30:03.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2204e-05 2024-09-14T05:30:03.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391774+0000", 2024-09-14T05:30:03.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038345600000000001 2024-09-14T05:30:03.077 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.077 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.077 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.077 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3921 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 5308416~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.077 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:03.077 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.630116209000001, 2024-09-14T05:30:03.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060842694000000003, 2024-09-14T05:30:03.078 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.078 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.078 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.078 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.078 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.078 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.078 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3921 2024-09-14T05:30:03.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.078 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401705+0000", 2024-09-14T05:30:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768499999999999 2024-09-14T05:30:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401709+0000", 2024-09-14T05:30:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0319999999999997e-06 2024-09-14T05:30:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401720+0000", 2024-09-14T05:30:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1005000000000001e-05 2024-09-14T05:30:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401727+0000", 2024-09-14T05:30:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1879999999999996e-06 2024-09-14T05:30:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461748+0000", 2024-09-14T05:30:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060021199999999997 2024-09-14T05:30:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461793+0000", 2024-09-14T05:30:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4530999999999998e-05 2024-09-14T05:30:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462110+0000", 2024-09-14T05:30:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031705299999999997 2024-09-14T05:30:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3923 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 6291456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.629470180999999, 2024-09-14T05:30:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060891273000000003, 2024-09-14T05:30:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3923 2024-09-14T05:30:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407054+0000", 2024-09-14T05:30:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005140787 2024-09-14T05:30:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407059+0000", 2024-09-14T05:30:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.04e-06 2024-09-14T05:30:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407069+0000", 2024-09-14T05:30:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4429999999999992e-06 2024-09-14T05:30:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407076+0000", 2024-09-14T05:30:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1679999999999998e-06 2024-09-14T05:30:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462514+0000", 2024-09-14T05:30:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055438551000000003 2024-09-14T05:30:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462562+0000", 2024-09-14T05:30:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7649999999999999e-05 2024-09-14T05:30:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462805+0000", 2024-09-14T05:30:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024263400000000001 2024-09-14T05:30:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4052 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 12550144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.613898171999999, 2024-09-14T05:30:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060872904999999998, 2024-09-14T05:30:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4052 2024-09-14T05:30:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417486+0000", 2024-09-14T05:30:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-14T05:30:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417487+0000", 2024-09-14T05:30:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4499999999999995e-07 2024-09-14T05:30:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417489+0000", 2024-09-14T05:30:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8899999999999999e-06 2024-09-14T05:30:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417492+0000", 2024-09-14T05:30:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4389999999999999e-06 2024-09-14T05:30:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478027+0000", 2024-09-14T05:30:03.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060534959999999999 2024-09-14T05:30:03.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478079+0000", 2024-09-14T05:30:03.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2104999999999997e-05 2024-09-14T05:30:03.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478358+0000", 2024-09-14T05:30:03.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000279351 2024-09-14T05:30:03.091 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.091 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.091 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.091 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4053 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13041664~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.091 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:03.091 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.613888788000001, 2024-09-14T05:30:03.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061258866000000002, 2024-09-14T05:30:03.091 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.091 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.091 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.091 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.092 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.092 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.092 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4053 2024-09-14T05:30:03.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.092 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:03.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:03.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:03.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.17e-07 2024-09-14T05:30:03.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417496+0000", 2024-09-14T05:30:03.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0890000000000001e-06 2024-09-14T05:30:03.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417499+0000", 2024-09-14T05:30:03.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8629999999999999e-06 2024-09-14T05:30:03.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417502+0000", 2024-09-14T05:30:03.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6290000000000002e-06 2024-09-14T05:30:03.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478387+0000", 2024-09-14T05:30:03.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060885433000000003 2024-09-14T05:30:03.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478435+0000", 2024-09-14T05:30:03.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8167999999999997e-05 2024-09-14T05:30:03.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478754+0000", 2024-09-14T05:30:03.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031846700000000001 2024-09-14T05:30:03.095 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.095 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.095 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.096 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4054 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13533184~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.096 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:03.096 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.613878904, 2024-09-14T05:30:03.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061604694000000002, 2024-09-14T05:30:03.096 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.096 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.096 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.096 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.096 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.096 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.097 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4054 2024-09-14T05:30:03.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.097 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:03.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:03.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:03.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.04e-07 2024-09-14T05:30:03.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417506+0000", 2024-09-14T05:30:03.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8299999999999995e-07 2024-09-14T05:30:03.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417509+0000", 2024-09-14T05:30:03.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.88e-06 2024-09-14T05:30:03.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417511+0000", 2024-09-14T05:30:03.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5950000000000001e-06 2024-09-14T05:30:03.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478784+0000", 2024-09-14T05:30:03.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061272971000000002 2024-09-14T05:30:03.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478841+0000", 2024-09-14T05:30:03.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-14T05:30:03.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479109+0000", 2024-09-14T05:30:03.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026794600000000003 2024-09-14T05:30:03.100 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.100 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.101 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.101 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4055 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14024704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.101 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:03.101 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.613869202, 2024-09-14T05:30:03.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061932299000000003, 2024-09-14T05:30:03.101 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.101 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.101 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.101 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.101 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.101 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.101 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4055 2024-09-14T05:30:03.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.102 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:03.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:03.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417515+0000", 2024-09-14T05:30:03.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-14T05:30:03.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417516+0000", 2024-09-14T05:30:03.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7199999999999997e-07 2024-09-14T05:30:03.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417519+0000", 2024-09-14T05:30:03.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.937e-06 2024-09-14T05:30:03.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417522+0000", 2024-09-14T05:30:03.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1420000000000002e-06 2024-09-14T05:30:03.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479140+0000", 2024-09-14T05:30:03.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061617933999999999 2024-09-14T05:30:03.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479188+0000", 2024-09-14T05:30:03.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8717000000000001e-05 2024-09-14T05:30:03.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479447+0000", 2024-09-14T05:30:03.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025838800000000002 2024-09-14T05:30:03.105 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.105 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.105 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.105 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4056 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14516224~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.105 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:03.106 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.613859029, 2024-09-14T05:30:03.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062263185999999998, 2024-09-14T05:30:03.106 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.106 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.106 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.106 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.106 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.106 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.106 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4056 2024-09-14T05:30:03.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.106 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:03.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:03.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:03.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8700000000000002e-07 2024-09-14T05:30:03.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417528+0000", 2024-09-14T05:30:03.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.579e-06 2024-09-14T05:30:03.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417533+0000", 2024-09-14T05:30:03.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2400000000000001e-06 2024-09-14T05:30:03.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417536+0000", 2024-09-14T05:30:03.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0309999999999999e-06 2024-09-14T05:30:03.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479477+0000", 2024-09-14T05:30:03.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061940967 2024-09-14T05:30:03.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479523+0000", 2024-09-14T05:30:03.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6335000000000003e-05 2024-09-14T05:30:03.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479788+0000", 2024-09-14T05:30:03.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026474700000000002 2024-09-14T05:30:03.109 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.110 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.110 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.110 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4057 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15007744~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.110 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:03.110 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.613844568000001, 2024-09-14T05:30:03.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062589928000000003, 2024-09-14T05:30:03.110 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.110 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.110 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.110 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.110 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.110 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.111 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4057 2024-09-14T05:30:03.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.111 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:03.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:03.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422838+0000", 2024-09-14T05:30:03.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052990349999999997 2024-09-14T05:30:03.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422854+0000", 2024-09-14T05:30:03.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5523999999999999e-05 2024-09-14T05:30:03.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422864+0000", 2024-09-14T05:30:03.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0084e-05 2024-09-14T05:30:03.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422872+0000", 2024-09-14T05:30:03.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-14T05:30:03.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479813+0000", 2024-09-14T05:30:03.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056941389000000002 2024-09-14T05:30:03.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479868+0000", 2024-09-14T05:30:03.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4973999999999999e-05 2024-09-14T05:30:03.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.480129+0000", 2024-09-14T05:30:03.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026084200000000001 2024-09-14T05:30:03.114 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.114 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.114 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.115 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4171 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14057472~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.115 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:03.115 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.601847008, 2024-09-14T05:30:03.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061184929999999998, 2024-09-14T05:30:03.115 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.115 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.115 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.115 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.115 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.115 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.115 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4171 2024-09-14T05:30:03.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.116 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:03.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:03.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:03.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.39e-07 2024-09-14T05:30:03.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429541+0000", 2024-09-14T05:30:03.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:30:03.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429547+0000", 2024-09-14T05:30:03.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3280000000000002e-06 2024-09-14T05:30:03.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429552+0000", 2024-09-14T05:30:03.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4859999999999997e-06 2024-09-14T05:30:03.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490379+0000", 2024-09-14T05:30:03.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060826681 2024-09-14T05:30:03.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490434+0000", 2024-09-14T05:30:03.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4577999999999998e-05 2024-09-14T05:30:03.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490722+0000", 2024-09-14T05:30:03.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028781400000000001 2024-09-14T05:30:03.119 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.119 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.119 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.119 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4172 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14548992~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.119 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:03.119 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.601825000000002, 2024-09-14T05:30:03.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061498256000000001, 2024-09-14T05:30:03.119 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.120 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.120 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.120 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.120 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.120 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.120 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4172 2024-09-14T05:30:03.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.120 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:03.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:03.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:03.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.72e-07 2024-09-14T05:30:03.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429561+0000", 2024-09-14T05:30:03.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.717e-06 2024-09-14T05:30:03.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429566+0000", 2024-09-14T05:30:03.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6529999999999999e-06 2024-09-14T05:30:03.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429571+0000", 2024-09-14T05:30:03.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5979999999999999e-06 2024-09-14T05:30:03.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490751+0000", 2024-09-14T05:30:03.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061180336000000002 2024-09-14T05:30:03.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490798+0000", 2024-09-14T05:30:03.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6875000000000001e-05 2024-09-14T05:30:03.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491057+0000", 2024-09-14T05:30:03.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025870500000000002 2024-09-14T05:30:03.123 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.123 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.123 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.124 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4173 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15040512~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.124 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:03.124 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.601807374, 2024-09-14T05:30:03.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061894820000000003, 2024-09-14T05:30:03.124 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.124 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.124 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.124 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.124 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.124 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.124 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4173 2024-09-14T05:30:03.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.125 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:03.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:03.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429577+0000", 2024-09-14T05:30:03.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-14T05:30:03.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429578+0000", 2024-09-14T05:30:03.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6980000000000001e-06 2024-09-14T05:30:03.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429585+0000", 2024-09-14T05:30:03.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.725e-06 2024-09-14T05:30:03.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429590+0000", 2024-09-14T05:30:03.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.938e-06 2024-09-14T05:30:03.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491091+0000", 2024-09-14T05:30:03.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061500773000000002 2024-09-14T05:30:03.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491136+0000", 2024-09-14T05:30:03.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5284000000000002e-05 2024-09-14T05:30:03.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491471+0000", 2024-09-14T05:30:03.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033485699999999998 2024-09-14T05:30:03.131 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.131 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.131 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.131 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.131 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.131 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:03.132 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:30:03.133 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:03.131+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:03.233 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:03.233+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:03.241 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:03.241 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:03.241 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:03.241 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:03.251 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:30:03.335 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:03.335+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:03.396 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:03.396 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:03.396 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:03.397 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:03.397 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:03.406 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:30:03.421 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-14T05:30:03.436 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:03.435+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:03.482 INFO:teuthology.orchestra.run.smithi071.stdout:2946 trunc from 0xe11151e to 0x2ed1047 2024-09-14T05:30:03.499 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:03.499 INFO:teuthology.orchestra.run.smithi071.stdout:2951 read 0x1d42361 thru 0x1d49dcb (0x7a6b bytes) 2024-09-14T05:30:03.499 INFO:teuthology.orchestra.run.smithi071.stdout:2952 write 0x26ae69e thru 0x26b53d9 (0x6d3c bytes) 2024-09-14T05:30:03.503 INFO:teuthology.orchestra.run.smithi071.stdout:2953 read 0x198cbad thru 0x1999bbe (0xd012 bytes) 2024-09-14T05:30:03.514 INFO:teuthology.orchestra.run.smithi071.stdout:2956 write 0x7a5e56e thru 0x7a6be16 (0xd8a9 bytes) 2024-09-14T05:30:03.518 INFO:teuthology.orchestra.run.smithi071.stdout:2957 read 0x6daa590 thru 0x6dba27b (0xfcec bytes) 2024-09-14T05:30:03.518 INFO:teuthology.orchestra.run.smithi071.stdout:2958 read 0x279dd0e thru 0x27a4316 (0x6609 bytes) 2024-09-14T05:30:03.518 INFO:teuthology.orchestra.run.smithi071.stdout:2960 read 0x6c8ede8 thru 0x6c93a90 (0x4ca9 bytes) 2024-09-14T05:30:03.518 INFO:teuthology.orchestra.run.smithi071.stdout:2961 write 0x6a81f37 thru 0x6a8243a (0x504 bytes) 2024-09-14T05:30:03.521 INFO:teuthology.orchestra.run.smithi071.stdout:2962 read 0x67995e3 thru 0x679dc34 (0x4652 bytes) 2024-09-14T05:30:03.525 INFO:teuthology.orchestra.run.smithi071.stdout:2965 trunc from 0x7a6be17 to 0x465ca71 2024-09-14T05:30:03.534 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:03.536 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:03.536+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:03.551 INFO:teuthology.orchestra.run.smithi071.stdout:2967 write 0xba9bece thru 0xbaa52b1 (0x93e4 bytes) 2024-09-14T05:30:03.554 INFO:teuthology.orchestra.run.smithi071.stdout:2968 write 0x22e60ce thru 0x22ee16c (0x809f bytes) 2024-09-14T05:30:03.557 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:1134 3.7 e30/24 3:e1ef7f87:::rbd_data.112dd25fb4c.000000000000005f:head v 30'40, mlcod=30'36)", 2024-09-14T05:30:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.166405+0000", 2024-09-14T05:30:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.388833323, 2024-09-14T05:30:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061353650000000003, 2024-09-14T05:30:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166405+0000", 2024-09-14T05:30:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166405+0000", 2024-09-14T05:30:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166406+0000", 2024-09-14T05:30:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.85e-07 2024-09-14T05:30:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166413+0000", 2024-09-14T05:30:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1029999999999996e-06 2024-09-14T05:30:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166425+0000", 2024-09-14T05:30:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.258e-05 2024-09-14T05:30:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166433+0000", 2024-09-14T05:30:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2910000000000002e-06 2024-09-14T05:30:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166492+0000", 2024-09-14T05:30:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9107999999999997e-05 2024-09-14T05:30:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166513+0000", 2024-09-14T05:30:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.128e-05 2024-09-14T05:30:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.172500+0000", 2024-09-14T05:30:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059870519999999997 2024-09-14T05:30:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.172540+0000", 2024-09-14T05:30:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0166000000000001e-05 2024-09-14T05:30:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1205 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.357348455, 2024-09-14T05:30:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0086364279999999998, 2024-09-14T05:30:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1205 2024-09-14T05:30:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197891+0000", 2024-09-14T05:30:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.364e-06 2024-09-14T05:30:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197896+0000", 2024-09-14T05:30:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3000000000000001e-06 2024-09-14T05:30:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197910+0000", 2024-09-14T05:30:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3339000000000001e-05 2024-09-14T05:30:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197919+0000", 2024-09-14T05:30:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8869999999999998e-06 2024-09-14T05:30:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197974+0000", 2024-09-14T05:30:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5720000000000002e-05 2024-09-14T05:30:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.198094+0000", 2024-09-14T05:30:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119739 2024-09-14T05:30:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206526+0000", 2024-09-14T05:30:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084320790000000003 2024-09-14T05:30:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1220 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.348592091, 2024-09-14T05:30:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012822337, 2024-09-14T05:30:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1220 2024-09-14T05:30:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206647+0000", 2024-09-14T05:30:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4600000000000003e-07 2024-09-14T05:30:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206652+0000", 2024-09-14T05:30:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6619999999999996e-06 2024-09-14T05:30:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206662+0000", 2024-09-14T05:30:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.995e-06 2024-09-14T05:30:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206667+0000", 2024-09-14T05:30:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-14T05:30:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206706+0000", 2024-09-14T05:30:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9523000000000001e-05 2024-09-14T05:30:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206755+0000", 2024-09-14T05:30:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000002e-05 2024-09-14T05:30:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206877+0000", 2024-09-14T05:30:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121938 2024-09-14T05:30:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.207310+0000", 2024-09-14T05:30:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000432452 2024-09-14T05:30:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219364+0000", 2024-09-14T05:30:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012054481000000001 2024-09-14T05:30:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219425+0000", 2024-09-14T05:30:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0618000000000001e-05 2024-09-14T05:30:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219468+0000", 2024-09-14T05:30:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3739000000000003e-05 2024-09-14T05:30:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1243 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.335626863999998, 2024-09-14T05:30:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078660580000000004, 2024-09-14T05:30:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1243 2024-09-14T05:30:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219612+0000", 2024-09-14T05:30:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.09e-07 2024-09-14T05:30:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219616+0000", 2024-09-14T05:30:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.216e-06 2024-09-14T05:30:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219630+0000", 2024-09-14T05:30:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3064999999999999e-05 2024-09-14T05:30:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219637+0000", 2024-09-14T05:30:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7409999999999992e-06 2024-09-14T05:30:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219697+0000", 2024-09-14T05:30:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.02e-05 2024-09-14T05:30:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219833+0000", 2024-09-14T05:30:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013595600000000001 2024-09-14T05:30:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227477+0000", 2024-09-14T05:30:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076439710000000003 2024-09-14T05:30:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1264 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.327616071000001, 2024-09-14T05:30:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080867990000000004, 2024-09-14T05:30:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1264 2024-09-14T05:30:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227623+0000", 2024-09-14T05:30:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:30:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227627+0000", 2024-09-14T05:30:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.194e-06 2024-09-14T05:30:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227639+0000", 2024-09-14T05:30:03.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1296e-05 2024-09-14T05:30:03.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227648+0000", 2024-09-14T05:30:03.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2280000000000004e-06 2024-09-14T05:30:03.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227689+0000", 2024-09-14T05:30:03.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1329999999999999e-05 2024-09-14T05:30:03.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227741+0000", 2024-09-14T05:30:03.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1641000000000002e-05 2024-09-14T05:30:03.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:30:03.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227914+0000", 2024-09-14T05:30:03.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000173215 2024-09-14T05:30:03.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:03.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228474+0000", 2024-09-14T05:30:03.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055980199999999998 2024-09-14T05:30:03.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:03.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235608+0000", 2024-09-14T05:30:03.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0071343830000000002 2024-09-14T05:30:03.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235669+0000", 2024-09-14T05:30:03.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1068999999999994e-05 2024-09-14T05:30:03.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235709+0000", 2024-09-14T05:30:03.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9671e-05 2024-09-14T05:30:03.592 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.592 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.593 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.593 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1520 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:03.593 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:30:03.593 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.232608452000001, 2024-09-14T05:30:03.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061963629999999999, 2024-09-14T05:30:03.594 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.594 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.594 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:03.594 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.594 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.594 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.595 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1520 2024-09-14T05:30:03.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.595 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:30:03.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:30:03.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322631+0000", 2024-09-14T05:30:03.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.54e-07 2024-09-14T05:30:03.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322635+0000", 2024-09-14T05:30:03.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0450000000000002e-06 2024-09-14T05:30:03.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322646+0000", 2024-09-14T05:30:03.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1919999999999999e-05 2024-09-14T05:30:03.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322654+0000", 2024-09-14T05:30:03.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3139999999999998e-06 2024-09-14T05:30:03.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322704+0000", 2024-09-14T05:30:03.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0420000000000002e-05 2024-09-14T05:30:03.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322794+0000", 2024-09-14T05:30:03.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0037000000000005e-05 2024-09-14T05:30:03.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328826+0000", 2024-09-14T05:30:03.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060318730000000001 2024-09-14T05:30:03.601 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.601 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.602 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.602 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1526 3.4 3:2695cbe4:::rbd_data.112dd25fb4c.0000000000000119:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:03.602 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:03.602 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.229075508000001, 2024-09-14T05:30:03.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0079537359999999994, 2024-09-14T05:30:03.602 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.602 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.602 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.602 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.602 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.603 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.603 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1526 2024-09-14T05:30:03.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.603 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:03.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:03.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:03.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3600000000000003e-07 2024-09-14T05:30:03.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326168+0000", 2024-09-14T05:30:03.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3460000000000004e-06 2024-09-14T05:30:03.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326178+0000", 2024-09-14T05:30:03.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0291e-05 2024-09-14T05:30:03.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326183+0000", 2024-09-14T05:30:03.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-14T05:30:03.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326230+0000", 2024-09-14T05:30:03.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6650999999999998e-05 2024-09-14T05:30:03.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326295+0000", 2024-09-14T05:30:03.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5158000000000002e-05 2024-09-14T05:30:03.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:03.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326408+0000", 2024-09-14T05:30:03.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113039 2024-09-14T05:30:03.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:03.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.331648+0000", 2024-09-14T05:30:03.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052404929999999997 2024-09-14T05:30:03.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:03.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334011+0000", 2024-09-14T05:30:03.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002362669 2024-09-14T05:30:03.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334064+0000", 2024-09-14T05:30:03.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3198999999999997e-05 2024-09-14T05:30:03.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334116+0000", 2024-09-14T05:30:03.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2133999999999999e-05 2024-09-14T05:30:03.609 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.609 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.609 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.609 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1529 3.1 3:809746f7:::rbd_data.112dd25fb4c.0000000000000118:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:03.609 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:30:03.609 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.227310957999997, 2024-09-14T05:30:03.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061981060000000001, 2024-09-14T05:30:03.610 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.610 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.610 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.610 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1529 2024-09-14T05:30:03.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.610 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:30:03.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:30:03.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327928+0000", 2024-09-14T05:30:03.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0100000000000004e-07 2024-09-14T05:30:03.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327933+0000", 2024-09-14T05:30:03.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4689999999999999e-06 2024-09-14T05:30:03.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327944+0000", 2024-09-14T05:30:03.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1623999999999999e-05 2024-09-14T05:30:03.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327951+0000", 2024-09-14T05:30:03.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9759999999999998e-06 2024-09-14T05:30:03.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328003+0000", 2024-09-14T05:30:03.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1521999999999999e-05 2024-09-14T05:30:03.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328087+0000", 2024-09-14T05:30:03.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3988999999999998e-05 2024-09-14T05:30:03.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:03.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328304+0000", 2024-09-14T05:30:03.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000217254 2024-09-14T05:30:03.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:03.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.333363+0000", 2024-09-14T05:30:03.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050586499999999996 2024-09-14T05:30:03.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:03.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334026+0000", 2024-09-14T05:30:03.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066303499999999997 2024-09-14T05:30:03.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334080+0000", 2024-09-14T05:30:03.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4261000000000001e-05 2024-09-14T05:30:03.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334125+0000", 2024-09-14T05:30:03.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5525000000000002e-05 2024-09-14T05:30:03.617 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.617 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.617 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.617 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1531 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:03.617 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:03.618 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.226257341999997, 2024-09-14T05:30:03.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082114309999999999, 2024-09-14T05:30:03.618 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.618 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.618 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.618 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.618 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.618 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.618 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1531 2024-09-14T05:30:03.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.618 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:03.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:03.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328982+0000", 2024-09-14T05:30:03.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:30:03.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328985+0000", 2024-09-14T05:30:03.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7900000000000001e-06 2024-09-14T05:30:03.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328996+0000", 2024-09-14T05:30:03.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.028e-05 2024-09-14T05:30:03.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329001+0000", 2024-09-14T05:30:03.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0610000000000003e-06 2024-09-14T05:30:03.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329035+0000", 2024-09-14T05:30:03.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4366999999999998e-05 2024-09-14T05:30:03.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329082+0000", 2024-09-14T05:30:03.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6646e-05 2024-09-14T05:30:03.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:03.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329195+0000", 2024-09-14T05:30:03.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113279 2024-09-14T05:30:03.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:03.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334293+0000", 2024-09-14T05:30:03.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050980399999999999 2024-09-14T05:30:03.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:03.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337101+0000", 2024-09-14T05:30:03.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0028076170000000001 2024-09-14T05:30:03.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337154+0000", 2024-09-14T05:30:03.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3641999999999999e-05 2024-09-14T05:30:03.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337192+0000", 2024-09-14T05:30:03.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7932000000000002e-05 2024-09-14T05:30:03.625 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.626 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.626 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.626 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:2131 3.b e30/24 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head v 30'36, mlcod=30'34)", 2024-09-14T05:30:03.626 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:03.626 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.762909987, 2024-09-14T05:30:03.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031871709999999998, 2024-09-14T05:30:03.626 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.626 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.626 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.626 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:03.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:03.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792331+0000", 2024-09-14T05:30:03.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6900000000000001e-06 2024-09-14T05:30:03.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797319+0000", 2024-09-14T05:30:03.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049876349999999998 2024-09-14T05:30:03.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797349+0000", 2024-09-14T05:30:03.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0046e-05 2024-09-14T05:30:03.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797359+0000", 2024-09-14T05:30:03.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0037999999999999e-05 2024-09-14T05:30:03.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797445+0000", 2024-09-14T05:30:03.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6132999999999997e-05 2024-09-14T05:30:03.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797472+0000", 2024-09-14T05:30:03.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7478999999999999e-05 2024-09-14T05:30:03.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824161+0000", 2024-09-14T05:30:03.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026688714999999998 2024-09-14T05:30:03.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824200+0000", 2024-09-14T05:30:03.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8974000000000003e-05 2024-09-14T05:30:03.631 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.631 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.631 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.631 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4259 3.4 3:25f21dc9:::rbd_data.112d3d611e54.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 294912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.631 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:03.631 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.163737372999996, 2024-09-14T05:30:03.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010043553, 2024-09-14T05:30:03.631 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.631 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.632 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.632 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.632 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.632 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.632 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4259 2024-09-14T05:30:03.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.632 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:03.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:03.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391503+0000", 2024-09-14T05:30:03.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9099999999999999e-06 2024-09-14T05:30:03.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391821+0000", 2024-09-14T05:30:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031871800000000001 2024-09-14T05:30:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391844+0000", 2024-09-14T05:30:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2045e-05 2024-09-14T05:30:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391853+0000", 2024-09-14T05:30:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7249999999999992e-06 2024-09-14T05:30:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391917+0000", 2024-09-14T05:30:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3458000000000001e-05 2024-09-14T05:30:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392037+0000", 2024-09-14T05:30:03.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120449 2024-09-14T05:30:03.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:03.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392305+0000", 2024-09-14T05:30:03.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026739299999999999 2024-09-14T05:30:03.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:03.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.393609+0000", 2024-09-14T05:30:03.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013042450000000001 2024-09-14T05:30:03.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:03.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401430+0000", 2024-09-14T05:30:03.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078211670000000004 2024-09-14T05:30:03.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401499+0000", 2024-09-14T05:30:03.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9374000000000004e-05 2024-09-14T05:30:03.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401544+0000", 2024-09-14T05:30:03.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5068999999999998e-05 2024-09-14T05:30:03.640 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.640 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.640 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.641 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4452 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.641 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:30:03.641 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.672924765000001, 2024-09-14T05:30:03.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066061380000000001, 2024-09-14T05:30:03.641 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.641 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.641 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.641 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4452 2024-09-14T05:30:03.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.642 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:30:03.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:30:03.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882314+0000", 2024-09-14T05:30:03.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5899999999999998e-07 2024-09-14T05:30:03.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882315+0000", 2024-09-14T05:30:03.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6169999999999999e-06 2024-09-14T05:30:03.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882319+0000", 2024-09-14T05:30:03.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3629999999999998e-06 2024-09-14T05:30:03.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882324+0000", 2024-09-14T05:30:03.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1370000000000001e-06 2024-09-14T05:30:03.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883369+0000", 2024-09-14T05:30:03.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010448320000000001 2024-09-14T05:30:03.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883421+0000", 2024-09-14T05:30:03.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2401000000000002e-05 2024-09-14T05:30:03.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:03.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883638+0000", 2024-09-14T05:30:03.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021725600000000001 2024-09-14T05:30:03.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:03.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.884464+0000", 2024-09-14T05:30:03.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00082569800000000004 2024-09-14T05:30:03.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:03.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888860+0000", 2024-09-14T05:30:03.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0043963370000000002 2024-09-14T05:30:03.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888896+0000", 2024-09-14T05:30:03.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6022999999999997e-05 2024-09-14T05:30:03.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888920+0000", 2024-09-14T05:30:03.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3215000000000001e-05 2024-09-14T05:30:03.648 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.648 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.648 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.648 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4467 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.648 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:03.648 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.654556315000001, 2024-09-14T05:30:03.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0065700189999999999, 2024-09-14T05:30:03.648 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.648 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.649 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.649 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.649 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.649 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.649 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4467 2024-09-14T05:30:03.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.649 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:03.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:03.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:03.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-14T05:30:03.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900684+0000", 2024-09-14T05:30:03.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.629e-06 2024-09-14T05:30:03.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900688+0000", 2024-09-14T05:30:03.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8469999999999998e-06 2024-09-14T05:30:03.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900691+0000", 2024-09-14T05:30:03.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-14T05:30:03.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.901775+0000", 2024-09-14T05:30:03.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001084177 2024-09-14T05:30:03.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.901862+0000", 2024-09-14T05:30:03.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7167000000000001e-05 2024-09-14T05:30:03.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:03.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.902223+0000", 2024-09-14T05:30:03.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036068800000000001 2024-09-14T05:30:03.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:03.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.902843+0000", 2024-09-14T05:30:03.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000620489 2024-09-14T05:30:03.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:03.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907167+0000", 2024-09-14T05:30:03.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0043242649999999999 2024-09-14T05:30:03.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907217+0000", 2024-09-14T05:30:03.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9648000000000003e-05 2024-09-14T05:30:03.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907252+0000", 2024-09-14T05:30:03.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4780000000000002e-05 2024-09-14T05:30:03.654 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.654 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.654 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.654 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4554 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:03.654 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:03.654 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.570836562, 2024-09-14T05:30:03.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069998999999999999, 2024-09-14T05:30:03.655 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.655 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.655 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.655 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.655 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.655 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.655 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4554 2024-09-14T05:30:03.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.655 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:03.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:03.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:03.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9299999999999999e-07 2024-09-14T05:30:03.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984404+0000", 2024-09-14T05:30:03.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.855e-06 2024-09-14T05:30:03.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984407+0000", 2024-09-14T05:30:03.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-14T05:30:03.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984410+0000", 2024-09-14T05:30:03.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8159999999999998e-06 2024-09-14T05:30:03.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985511+0000", 2024-09-14T05:30:03.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00110053 2024-09-14T05:30:03.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985561+0000", 2024-09-14T05:30:03.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0389000000000002e-05 2024-09-14T05:30:03.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:03.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985772+0000", 2024-09-14T05:30:03.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000210571 2024-09-14T05:30:03.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:03.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.987075+0000", 2024-09-14T05:30:03.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001303128 2024-09-14T05:30:03.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:03.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991303+0000", 2024-09-14T05:30:03.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042283310000000001 2024-09-14T05:30:03.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991359+0000", 2024-09-14T05:30:03.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6335999999999997e-05 2024-09-14T05:30:03.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991402+0000", 2024-09-14T05:30:03.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2074999999999999e-05 2024-09-14T05:30:03.661 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.661 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.661 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.661 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4711 3.4 3:226df663:::rbd_data.112d3d611e54.000000000000002d:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:03.661 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:03.661 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.388165405999999, 2024-09-14T05:30:03.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011304193000000001, 2024-09-14T05:30:03.662 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.662 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.662 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.662 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.662 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.662 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.662 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4711 2024-09-14T05:30:03.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.662 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:03.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:03.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167074+0000", 2024-09-14T05:30:03.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0999999999999997e-07 2024-09-14T05:30:03.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167078+0000", 2024-09-14T05:30:03.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.878e-06 2024-09-14T05:30:03.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167087+0000", 2024-09-14T05:30:03.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6940000000000007e-06 2024-09-14T05:30:03.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167093+0000", 2024-09-14T05:30:03.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7740000000000001e-06 2024-09-14T05:30:03.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167128+0000", 2024-09-14T05:30:03.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5231999999999997e-05 2024-09-14T05:30:03.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167207+0000", 2024-09-14T05:30:03.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9036999999999996e-05 2024-09-14T05:30:03.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:03.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.169140+0000", 2024-09-14T05:30:03.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001932897 2024-09-14T05:30:03.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:03.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178258+0000", 2024-09-14T05:30:03.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091180819999999996 2024-09-14T05:30:03.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:03.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178314+0000", 2024-09-14T05:30:03.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5571e-05 2024-09-14T05:30:03.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178348+0000", 2024-09-14T05:30:03.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3747000000000001e-05 2024-09-14T05:30:03.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178377+0000", 2024-09-14T05:30:03.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9471e-05 2024-09-14T05:30:03.668 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.669 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.669 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.669 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4721 3.1 3:86302a6e:::rbd_data.112d3d611e54.0000000000000011:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:03.669 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:30:03.670 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.356179919999999, 2024-09-14T05:30:03.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066831750000000004, 2024-09-14T05:30:03.670 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.670 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.670 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.670 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.671 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.671 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.671 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4721 2024-09-14T05:30:03.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.671 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:30:03.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:30:03.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199059+0000", 2024-09-14T05:30:03.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.06e-07 2024-09-14T05:30:03.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199062+0000", 2024-09-14T05:30:03.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2279999999999998e-06 2024-09-14T05:30:03.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199072+0000", 2024-09-14T05:30:03.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5999999999999996e-06 2024-09-14T05:30:03.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199078+0000", 2024-09-14T05:30:03.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9109999999999998e-06 2024-09-14T05:30:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199123+0000", 2024-09-14T05:30:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5278000000000003e-05 2024-09-14T05:30:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199202+0000", 2024-09-14T05:30:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8521000000000003e-05 2024-09-14T05:30:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199271+0000", 2024-09-14T05:30:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9857999999999994e-05 2024-09-14T05:30:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199941+0000", 2024-09-14T05:30:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066984599999999998 2024-09-14T05:30:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205653+0000", 2024-09-14T05:30:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057117319999999997 2024-09-14T05:30:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205701+0000", 2024-09-14T05:30:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8254000000000003e-05 2024-09-14T05:30:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205741+0000", 2024-09-14T05:30:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0241000000000003e-05 2024-09-14T05:30:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4819 3.12 3:4e8efced:::rbd_data.112d3d611e54.000000000000000e:head [stat,truncate 1114112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:30:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.270865761, 2024-09-14T05:30:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062422770000000001, 2024-09-14T05:30:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4819 2024-09-14T05:30:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:30:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:30:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284373+0000", 2024-09-14T05:30:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3399999999999998e-07 2024-09-14T05:30:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284377+0000", 2024-09-14T05:30:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8539999999999999e-06 2024-09-14T05:30:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284388+0000", 2024-09-14T05:30:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1328999999999999e-05 2024-09-14T05:30:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284395+0000", 2024-09-14T05:30:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9759999999999998e-06 2024-09-14T05:30:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284723+0000", 2024-09-14T05:30:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032775799999999999 2024-09-14T05:30:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284801+0000", 2024-09-14T05:30:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7562e-05 2024-09-14T05:30:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:30:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284933+0000", 2024-09-14T05:30:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000131989 2024-09-14T05:30:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.287450+0000", 2024-09-14T05:30:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025177699999999999 2024-09-14T05:30:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290531+0000", 2024-09-14T05:30:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0030809959999999999 2024-09-14T05:30:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290588+0000", 2024-09-14T05:30:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6177000000000001e-05 2024-09-14T05:30:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290615+0000", 2024-09-14T05:30:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7132e-05 2024-09-14T05:30:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:7067 3.5 e35/24 3:a65d0cff:::rbd_data.112df7f2fe0a.0000000000000028:head v 35'89, mlcod=34'86)", 2024-09-14T05:30:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.412472489999999, 2024-09-14T05:30:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0064513080000000002, 2024-09-14T05:30:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142771+0000", 2024-09-14T05:30:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8949999999999997e-06 2024-09-14T05:30:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142779+0000", 2024-09-14T05:30:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8699999999999992e-06 2024-09-14T05:30:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142785+0000", 2024-09-14T05:30:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4899999999999997e-06 2024-09-14T05:30:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142843+0000", 2024-09-14T05:30:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7377000000000003e-05 2024-09-14T05:30:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142871+0000", 2024-09-14T05:30:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8305000000000001e-05 2024-09-14T05:30:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149177+0000", 2024-09-14T05:30:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063058639999999996 2024-09-14T05:30:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149217+0000", 2024-09-14T05:30:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0080999999999998e-05 2024-09-14T05:30:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:72582 3.b e47/24 3:d5072567:::rbd_data.112d4a90444c.0000000000000119:head v 47'386, mlcod=46'383)", 2024-09-14T05:30:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.030211552999999, 2024-09-14T05:30:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0067720940000000002, 2024-09-14T05:30:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1100000000000002e-07 2024-09-14T05:30:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525031+0000", 2024-09-14T05:30:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0740000000000003e-06 2024-09-14T05:30:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525040+0000", 2024-09-14T05:30:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5329999999999994e-06 2024-09-14T05:30:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525045+0000", 2024-09-14T05:30:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:30:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525090+0000", 2024-09-14T05:30:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4743000000000002e-05 2024-09-14T05:30:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525102+0000", 2024-09-14T05:30:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2067e-05 2024-09-14T05:30:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531757+0000", 2024-09-14T05:30:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066550380000000003 2024-09-14T05:30:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531799+0000", 2024-09-14T05:30:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1640000000000001e-05 2024-09-14T05:30:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:79671 3.15 e47/24 3:ab572765:::rbd_data.112d956b542f.0000000000000013:head v 47'120, mlcod=47'118)", 2024-09-14T05:30:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:30:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.404231335, 2024-09-14T05:30:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066700609999999997, 2024-09-14T05:30:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:30:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:30:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151008+0000", 2024-09-14T05:30:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:30:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151264+0000", 2024-09-14T05:30:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025656099999999999 2024-09-14T05:30:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151278+0000", 2024-09-14T05:30:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3288999999999999e-05 2024-09-14T05:30:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151287+0000", 2024-09-14T05:30:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7909999999999993e-06 2024-09-14T05:30:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151361+0000", 2024-09-14T05:30:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3979000000000004e-05 2024-09-14T05:30:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151383+0000", 2024-09-14T05:30:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2039999999999999e-05 2024-09-14T05:30:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.157542+0000", 2024-09-14T05:30:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006158915 2024-09-14T05:30:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.157677+0000", 2024-09-14T05:30:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013544300000000001 2024-09-14T05:30:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:03.699 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:03.701 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:30:03.703 INFO:teuthology.orchestra.run.smithi071.stdout:2970 read 0x6e66de1 thru 0x6e66ea7 (0xc7 bytes) 2024-09-14T05:30:03.703 INFO:teuthology.orchestra.run.smithi071.stdout:2971 write 0x3914658 thru 0x39222cb (0xdc74 bytes) 2024-09-14T05:30:03.704 INFO:teuthology.orchestra.run.smithi071.stdout:2974 read 0x698f126 thru 0x699bd92 (0xcc6d bytes) 2024-09-14T05:30:03.704 INFO:teuthology.orchestra.run.smithi071.stdout:2975 read 0x6b0f94c thru 0x6b1939c (0x9a51 bytes) 2024-09-14T05:30:03.704 INFO:teuthology.orchestra.run.smithi071.stdout:2976 writesame 0xd3f0ba9 thru 0xd3fac61 data_size 0x27(0xa0b9 bytes) 2024-09-14T05:30:03.704 INFO:teuthology.orchestra.run.smithi071.stdout:2978 punch from 0xc8079f9 to 0xc809b89, (0x2190 bytes) 2024-09-14T05:30:03.704 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:03.704 INFO:teuthology.orchestra.run.smithi071.stdout:2980 read 0x55fc922 thru 0x55feae7 (0x21c6 bytes) 2024-09-14T05:30:03.704 INFO:teuthology.orchestra.run.smithi071.stdout:2981 writesame 0x1389051 thru 0x1398c4e data_size 0x5(0xfbfe bytes) 2024-09-14T05:30:03.704 INFO:teuthology.orchestra.run.smithi071.stdout:2982 writesame 0x192d778 thru 0x193166e data_size 0x1b(0x3ef7 bytes) 2024-09-14T05:30:03.704 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:03.704 INFO:teuthology.orchestra.run.smithi071.stdout:2984 trunc from 0xd3fac62 to 0x63589e0 2024-09-14T05:30:03.704 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:03.704 INFO:teuthology.orchestra.run.smithi071.stdout:2986 writesame 0x57fcfe9 thru 0x580921d data_size 0x1(0xc235 bytes) 2024-09-14T05:30:03.704 INFO:teuthology.orchestra.run.smithi071.stdout:2987 writesame 0x446bd74 thru 0x446cae3 data_size 0xa(0xd70 bytes) 2024-09-14T05:30:03.705 INFO:teuthology.orchestra.run.smithi071.stdout:2989 read 0x34a26c0 thru 0x34a2a5a (0x39b bytes) 2024-09-14T05:30:03.705 INFO:teuthology.orchestra.run.smithi071.stdout:2990 write 0xa24dcdb thru 0xa256eaf (0x91d5 bytes) 2024-09-14T05:30:03.705 INFO:teuthology.orchestra.run.smithi071.stdout:2991 write 0xafb4708 thru 0xafba928 (0x6221 bytes) 2024-09-14T05:30:03.705 INFO:teuthology.orchestra.run.smithi071.stdout:2993 write 0xd23b0da thru 0xd23d3e9 (0x2310 bytes) 2024-09-14T05:30:03.705 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:03.705 INFO:teuthology.orchestra.run.smithi071.stdout:2997 writesame 0x51b49eb thru 0x51b97b7 data_size 0x3(0x4dcd bytes) 2024-09-14T05:30:03.705 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:03.705 INFO:teuthology.orchestra.run.smithi071.stdout:3000 trunc from 0xd23d3ea to 0xd624403 2024-09-14T05:30:03.705 INFO:teuthology.orchestra.run.smithi071.stdout:3001 write 0x5364929 thru 0x536db9f (0x9277 bytes) 2024-09-14T05:30:03.705 INFO:teuthology.orchestra.run.smithi071.stdout:3002 trunc from 0xd624403 to 0x4af1f4b 2024-09-14T05:30:03.705 INFO:teuthology.orchestra.run.smithi071.stdout:3003 read 0x348b364 thru 0x3493916 (0x85b3 bytes) 2024-09-14T05:30:03.705 INFO:teuthology.orchestra.run.smithi071.stdout:3004 trunc from 0x4af1f4b to 0x38fc40 2024-09-14T05:30:03.706 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:03.700+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:03.719 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:03.725 INFO:teuthology.orchestra.run.smithi071.stdout:3006 write 0x102d48d thru 0x103191b (0x448f bytes) 2024-09-14T05:30:03.749 INFO:teuthology.orchestra.run.smithi071.stdout:3007 write 0xd14ba70 thru 0xd14bbc0 (0x151 bytes) 2024-09-14T05:30:03.753 INFO:teuthology.orchestra.run.smithi071.stdout:3008 write 0x8df2689 thru 0x8df3493 (0xe0b bytes) 2024-09-14T05:30:03.756 INFO:teuthology.orchestra.run.smithi071.stdout:3010 writesame 0x54e36cd thru 0x54e6b95 data_size 0x1(0x34c9 bytes) 2024-09-14T05:30:03.770 INFO:teuthology.orchestra.run.smithi071.stdout:3011 trunc from 0xd14bbc1 to 0x8289764 2024-09-14T05:30:03.781 INFO:teuthology.orchestra.run.smithi071.stdout:3013 read 0x2be4ce1 thru 0x2beb118 (0x6438 bytes) 2024-09-14T05:30:03.781 INFO:teuthology.orchestra.run.smithi071.stdout:3014 write 0x6d21b6e thru 0x6d2d22b (0xb6be bytes) 2024-09-14T05:30:03.797 INFO:teuthology.orchestra.run.smithi071.stdout:3016 writesame 0xe455bc7 thru 0xe460a3c data_size 0x89(0xae76 bytes) 2024-09-14T05:30:03.797 INFO:teuthology.orchestra.run.smithi071.stdout:3017 write 0x2b4a1ca thru 0x2b557e7 (0xb61e bytes) 2024-09-14T05:30:03.801 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:03.802 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:03.801+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:03.802 INFO:teuthology.orchestra.run.smithi071.stdout:3021 write 0xfa37df thru 0xfac1af (0x89d1 bytes) 2024-09-14T05:30:03.809 INFO:teuthology.orchestra.run.smithi071.stdout:3022 trunc from 0xe460a3d to 0xa5d4b0a 2024-09-14T05:30:03.820 INFO:teuthology.orchestra.run.smithi071.stdout:3023 write 0xb2f3864 thru 0xb2f4f8f (0x172c bytes) 2024-09-14T05:30:03.826 INFO:teuthology.orchestra.run.smithi071.stdout:3024 trunc from 0xb2f4f90 to 0xd7e7e3f 2024-09-14T05:30:03.837 INFO:teuthology.orchestra.run.smithi071.stdout:3026 write 0xdb658a8 thru 0xdb65b79 (0x2d2 bytes) 2024-09-14T05:30:03.853 INFO:teuthology.orchestra.run.smithi071.stdout:3028 trunc from 0xdb65b7a to 0x1f1e0d9 2024-09-14T05:30:03.854 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:03.854 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:03.854 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:03.854 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:03.864 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:30:03.871 INFO:teuthology.orchestra.run.smithi071.stdout:3031 trunc from 0x1f1e0d9 to 0x63e78e3 2024-09-14T05:30:03.878 INFO:teuthology.orchestra.run.smithi071.stdout:3032 read 0x6380da5 thru 0x638168c (0x8e8 bytes) 2024-09-14T05:30:03.883 INFO:teuthology.orchestra.run.smithi071.stdout:3038 trunc from 0x63e78e3 to 0x1f67873 2024-09-14T05:30:03.889 INFO:teuthology.orchestra.run.smithi071.stdout:3039 read 0x1d27e70 thru 0x1d286e6 (0x877 bytes) 2024-09-14T05:30:03.902 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:03.902+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:03.908 INFO:teuthology.orchestra.run.smithi071.stdout:3040 write 0xbc7d7c7 thru 0xbc7ea30 (0x126a bytes) 2024-09-14T05:30:03.911 INFO:teuthology.orchestra.run.smithi071.stdout:3041 punch from 0x713852c to 0x71483ea, (0xfebe bytes) 2024-09-14T05:30:03.911 INFO:teuthology.orchestra.run.smithi071.stdout:3042 read 0xbb3f4ba thru 0xbb4906e (0x9bb5 bytes) 2024-09-14T05:30:03.911 INFO:teuthology.orchestra.run.smithi071.stdout:3043 read 0x1d9d087 thru 0x1da960e (0xc588 bytes) 2024-09-14T05:30:03.911 INFO:teuthology.orchestra.run.smithi071.stdout:3044 write 0x5170503 thru 0x517a436 (0x9f34 bytes) 2024-09-14T05:30:03.914 INFO:teuthology.orchestra.run.smithi071.stdout:3045 writesame 0x39cb784 thru 0x39dab7e data_size 0x1(0xf3fb bytes) 2024-09-14T05:30:03.946 INFO:teuthology.orchestra.run.smithi071.stdout:3046 read 0xb20cfa thru 0xb2c47b (0xb782 bytes) 2024-09-14T05:30:03.950 INFO:teuthology.orchestra.run.smithi071.stdout:3048 trunc from 0xbc7ea31 to 0xebbb1ce 2024-09-14T05:30:03.957 INFO:teuthology.orchestra.run.smithi071.stdout:3050 punch from 0x3dc03ea to 0x3dc7a64, (0x767a bytes) 2024-09-14T05:30:03.957 INFO:teuthology.orchestra.run.smithi071.stdout:3051 write 0x8e0c90 thru 0x8e2054 (0x13c5 bytes) 2024-09-14T05:30:03.959 INFO:teuthology.orchestra.run.smithi071.stdout:3052 punch from 0x4dbdebd to 0x4dc7f89, (0xa0cc bytes) 2024-09-14T05:30:03.959 INFO:teuthology.orchestra.run.smithi071.stdout:3055 read 0xe1080eb thru 0xe1101a2 (0x80b8 bytes) 2024-09-14T05:30:03.959 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:03.970 INFO:teuthology.orchestra.run.smithi071.stdout:3057 trunc from 0xebbb1ce to 0x5b329a6 2024-09-14T05:30:03.971 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:03.971 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:03.971 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:03.971 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:03.972 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:03.981 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:30:03.991 INFO:teuthology.orchestra.run.smithi071.stdout:3058 write 0xa0acb2a thru 0xa0b5aa2 (0x8f79 bytes) 2024-09-14T05:30:04.000 INFO:teuthology.orchestra.run.smithi071.stdout:3059 trunc from 0xa0b5aa3 to 0x52db765 2024-09-14T05:30:04.004 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:04.003+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:04.011 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.019 INFO:teuthology.orchestra.run.smithi071.stdout:3062 write 0x79a109c thru 0x79a2a0c (0x1971 bytes) 2024-09-14T05:30:04.022 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.022 INFO:teuthology.orchestra.run.smithi071.stdout:3064 read 0x349835d thru 0x34a82d0 (0xff74 bytes) 2024-09-14T05:30:04.022 INFO:teuthology.orchestra.run.smithi071.stdout:3065 punch from 0x142c3bf to 0x142dbb3, (0x17f4 bytes) 2024-09-14T05:30:04.022 INFO:teuthology.orchestra.run.smithi071.stdout:3066 write 0x625403 thru 0x62ec21 (0x981f bytes) 2024-09-14T05:30:04.025 INFO:teuthology.orchestra.run.smithi071.stdout:3067 read 0x217196b thru 0x2173146 (0x17dc bytes) 2024-09-14T05:30:04.033 INFO:teuthology.orchestra.run.smithi071.stdout:3069 trunc from 0x79a2a0d to 0x1536eb6 2024-09-14T05:30:04.042 INFO:teuthology.orchestra.run.smithi071.stdout:3071 punch from 0xd2141a to 0xd26ec6, (0x5aac bytes) 2024-09-14T05:30:04.058 INFO:teuthology.orchestra.run.smithi071.stdout:3072 writesame 0xb26b439 thru 0xb272554 data_size 0x26(0x711c bytes) 2024-09-14T05:30:04.070 INFO:teuthology.orchestra.run.smithi071.stdout:3073 trunc from 0xb272555 to 0x1354666 2024-09-14T05:30:04.076 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.076 INFO:teuthology.orchestra.run.smithi071.stdout:3078 read 0x132d34e thru 0x133a0ff (0xcdb2 bytes) 2024-09-14T05:30:04.076 INFO:teuthology.orchestra.run.smithi071.stdout:3080 read 0x10116a6 thru 0x1016bf0 (0x554b bytes) 2024-09-14T05:30:04.076 INFO:teuthology.orchestra.run.smithi071.stdout:3082 punch from 0x3a2371 to 0x3a6cd4, (0x4963 bytes) 2024-09-14T05:30:04.076 INFO:teuthology.orchestra.run.smithi071.stdout:3083 read 0x10cf8c3 thru 0x10d708d (0x77cb bytes) 2024-09-14T05:30:04.088 INFO:teuthology.orchestra.run.smithi071.stdout:3084 trunc from 0x1354666 to 0xbafa19f 2024-09-14T05:30:04.094 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2131 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.316734422000003, 2024-09-14T05:30:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049526593000000001, 2024-09-14T05:30:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2131 2024-09-14T05:30:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774867+0000", 2024-09-14T05:30:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4430000000000002e-06 2024-09-14T05:30:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790357+0000", 2024-09-14T05:30:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015489466 2024-09-14T05:30:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790383+0000", 2024-09-14T05:30:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6247000000000001e-05 2024-09-14T05:30:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790394+0000", 2024-09-14T05:30:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0728e-05 2024-09-14T05:30:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790471+0000", 2024-09-14T05:30:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7539000000000004e-05 2024-09-14T05:30:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790639+0000", 2024-09-14T05:30:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016787700000000001 2024-09-14T05:30:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-14T05:30:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790998+0000", 2024-09-14T05:30:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035931299999999999 2024-09-14T05:30:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.823529+0000", 2024-09-14T05:30:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032530798 2024-09-14T05:30:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824324+0000", 2024-09-14T05:30:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00079442399999999995 2024-09-14T05:30:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824362+0000", 2024-09-14T05:30:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8398000000000001e-05 2024-09-14T05:30:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824391+0000", 2024-09-14T05:30:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.936e-05 2024-09-14T05:30:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3892 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7602176~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.735156424000003, 2024-09-14T05:30:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047082951999999997, 2024-09-14T05:30:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3892 2024-09-14T05:30:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0199999999999998e-07 2024-09-14T05:30:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356454+0000", 2024-09-14T05:30:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0414999999999999e-05 2024-09-14T05:30:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356463+0000", 2024-09-14T05:30:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6739999999999992e-06 2024-09-14T05:30:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356468+0000", 2024-09-14T05:30:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5680000000000001e-06 2024-09-14T05:30:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403143+0000", 2024-09-14T05:30:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046675051000000002 2024-09-14T05:30:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403206+0000", 2024-09-14T05:30:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3127999999999996e-05 2024-09-14T05:30:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403526+0000", 2024-09-14T05:30:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031981400000000002 2024-09-14T05:30:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3893 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8093696~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.735111586999999, 2024-09-14T05:30:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047516305000000002, 2024-09-14T05:30:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3893 2024-09-14T05:30:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:04.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6899999999999997e-07 2024-09-14T05:30:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356491+0000", 2024-09-14T05:30:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8109999999999999e-06 2024-09-14T05:30:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356498+0000", 2024-09-14T05:30:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5320000000000003e-06 2024-09-14T05:30:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356502+0000", 2024-09-14T05:30:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000002e-06 2024-09-14T05:30:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403579+0000", 2024-09-14T05:30:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047076706000000003 2024-09-14T05:30:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403664+0000", 2024-09-14T05:30:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5240000000000004e-05 2024-09-14T05:30:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404004+0000", 2024-09-14T05:30:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033974500000000002 2024-09-14T05:30:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3993 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 262144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.735089715999997, 2024-09-14T05:30:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048088476999999998, 2024-09-14T05:30:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3993 2024-09-14T05:30:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:04.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.46e-07 2024-09-14T05:30:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356512+0000", 2024-09-14T05:30:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.827e-06 2024-09-14T05:30:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356518+0000", 2024-09-14T05:30:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7780000000000004e-06 2024-09-14T05:30:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356522+0000", 2024-09-14T05:30:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.865e-06 2024-09-14T05:30:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404036+0000", 2024-09-14T05:30:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047513725999999999 2024-09-14T05:30:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404135+0000", 2024-09-14T05:30:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9352999999999996e-05 2024-09-14T05:30:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404598+0000", 2024-09-14T05:30:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046258200000000001 2024-09-14T05:30:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3994 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.735047704000003, 2024-09-14T05:30:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048274625000000002, 2024-09-14T05:30:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3994 2024-09-14T05:30:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3799999999999997e-07 2024-09-14T05:30:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356554+0000", 2024-09-14T05:30:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2259999999999999e-06 2024-09-14T05:30:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356560+0000", 2024-09-14T05:30:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-14T05:30:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356566+0000", 2024-09-14T05:30:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3909999999999997e-06 2024-09-14T05:30:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404632+0000", 2024-09-14T05:30:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048066154 2024-09-14T05:30:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404719+0000", 2024-09-14T05:30:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7440000000000003e-05 2024-09-14T05:30:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404826+0000", 2024-09-14T05:30:04.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106813 2024-09-14T05:30:04.148 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.148 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.148 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.149 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3995 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.149 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:04.149 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.735022565000001, 2024-09-14T05:30:04.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048828209999999997, 2024-09-14T05:30:04.149 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.150 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.150 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.150 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.150 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.150 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.150 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3995 2024-09-14T05:30:04.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.151 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:04.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:04.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:04.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6599999999999996e-07 2024-09-14T05:30:04.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356579+0000", 2024-09-14T05:30:04.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0719999999999998e-06 2024-09-14T05:30:04.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356585+0000", 2024-09-14T05:30:04.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.959e-06 2024-09-14T05:30:04.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356590+0000", 2024-09-14T05:30:04.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.464e-06 2024-09-14T05:30:04.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404875+0000", 2024-09-14T05:30:04.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048285649999999999 2024-09-14T05:30:04.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404985+0000", 2024-09-14T05:30:04.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109417 2024-09-14T05:30:04.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405405+0000", 2024-09-14T05:30:04.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042008200000000001 2024-09-14T05:30:04.161 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.161 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.161 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.162 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3996 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.162 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:04.162 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.734713681999999, 2024-09-14T05:30:04.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049061595, 2024-09-14T05:30:04.162 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.162 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.162 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.162 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.162 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.162 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.162 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3996 2024-09-14T05:30:04.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.162 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:04.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:04.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:04.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.61e-07 2024-09-14T05:30:04.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356892+0000", 2024-09-14T05:30:04.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9050000000000002e-06 2024-09-14T05:30:04.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356901+0000", 2024-09-14T05:30:04.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4149999999999999e-06 2024-09-14T05:30:04.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356907+0000", 2024-09-14T05:30:04.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0140000000000004e-06 2024-09-14T05:30:04.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405448+0000", 2024-09-14T05:30:04.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048541647 2024-09-14T05:30:04.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405542+0000", 2024-09-14T05:30:04.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3639000000000003e-05 2024-09-14T05:30:04.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405947+0000", 2024-09-14T05:30:04.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040521399999999999 2024-09-14T05:30:04.166 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.166 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.166 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.166 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3997 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2228224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.166 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:04.166 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.734499700000001, 2024-09-14T05:30:04.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049308774999999999, 2024-09-14T05:30:04.166 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.166 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.166 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.167 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.167 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.167 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.167 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3997 2024-09-14T05:30:04.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.167 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:04.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:04.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:04.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6499999999999999e-07 2024-09-14T05:30:04.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357104+0000", 2024-09-14T05:30:04.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4989999999999999e-06 2024-09-14T05:30:04.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357112+0000", 2024-09-14T05:30:04.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4279999999999996e-06 2024-09-14T05:30:04.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357117+0000", 2024-09-14T05:30:04.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6190000000000002e-06 2024-09-14T05:30:04.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405989+0000", 2024-09-14T05:30:04.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048872080999999998 2024-09-14T05:30:04.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406074+0000", 2024-09-14T05:30:04.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4676e-05 2024-09-14T05:30:04.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406408+0000", 2024-09-14T05:30:04.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033480700000000001 2024-09-14T05:30:04.170 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.170 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.170 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.170 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3998 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.171 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:04.171 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.734178270999998, 2024-09-14T05:30:04.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049462314, 2024-09-14T05:30:04.171 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.171 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.171 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.171 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.171 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.171 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.171 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3998 2024-09-14T05:30:04.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.171 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:04.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:04.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357422+0000", 2024-09-14T05:30:04.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3099999999999996e-07 2024-09-14T05:30:04.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357429+0000", 2024-09-14T05:30:04.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2880000000000004e-06 2024-09-14T05:30:04.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357439+0000", 2024-09-14T05:30:04.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.302e-06 2024-09-14T05:30:04.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357445+0000", 2024-09-14T05:30:04.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4269999999999996e-06 2024-09-14T05:30:04.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406448+0000", 2024-09-14T05:30:04.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049003096000000003 2024-09-14T05:30:04.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406537+0000", 2024-09-14T05:30:04.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9135999999999995e-05 2024-09-14T05:30:04.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406883+0000", 2024-09-14T05:30:04.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000346134 2024-09-14T05:30:04.175 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.175 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.175 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.175 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3999 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.175 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:04.175 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.734107913999999, 2024-09-14T05:30:04.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049797556, 2024-09-14T05:30:04.175 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.175 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.176 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.176 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.176 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.176 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.176 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3999 2024-09-14T05:30:04.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.176 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:04.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:04.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357492+0000", 2024-09-14T05:30:04.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8299999999999996e-07 2024-09-14T05:30:04.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357495+0000", 2024-09-14T05:30:04.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0450000000000001e-06 2024-09-14T05:30:04.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357503+0000", 2024-09-14T05:30:04.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9689999999999994e-06 2024-09-14T05:30:04.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357510+0000", 2024-09-14T05:30:04.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-14T05:30:04.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406914+0000", 2024-09-14T05:30:04.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049404293000000002 2024-09-14T05:30:04.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406965+0000", 2024-09-14T05:30:04.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0470999999999998e-05 2024-09-14T05:30:04.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407289+0000", 2024-09-14T05:30:04.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032434899999999998 2024-09-14T05:30:04.179 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.179 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.179 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.180 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4000 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3702784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.180 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:04.180 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.733761569000002, 2024-09-14T05:30:04.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049936728999999999, 2024-09-14T05:30:04.180 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.180 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.180 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.180 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.180 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.180 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.180 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4000 2024-09-14T05:30:04.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.181 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:04.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:04.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357839+0000", 2024-09-14T05:30:04.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-14T05:30:04.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357842+0000", 2024-09-14T05:30:04.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4850000000000001e-06 2024-09-14T05:30:04.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357851+0000", 2024-09-14T05:30:04.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1819999999999995e-06 2024-09-14T05:30:04.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357859+0000", 2024-09-14T05:30:04.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8129999999999996e-06 2024-09-14T05:30:04.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407337+0000", 2024-09-14T05:30:04.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049478084999999998 2024-09-14T05:30:04.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407442+0000", 2024-09-14T05:30:04.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000105302 2024-09-14T05:30:04.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407774+0000", 2024-09-14T05:30:04.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033200100000000001 2024-09-14T05:30:04.184 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.184 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.184 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.184 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4001 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.184 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:04.184 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.733666214000003, 2024-09-14T05:30:04.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050304205999999997, 2024-09-14T05:30:04.184 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.185 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.185 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.185 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.185 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.185 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.185 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4001 2024-09-14T05:30:04.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.185 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:04.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:04.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357934+0000", 2024-09-14T05:30:04.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000004e-07 2024-09-14T05:30:04.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357937+0000", 2024-09-14T05:30:04.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0869999999999998e-06 2024-09-14T05:30:04.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357946+0000", 2024-09-14T05:30:04.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1439999999999992e-06 2024-09-14T05:30:04.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357952+0000", 2024-09-14T05:30:04.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.609e-06 2024-09-14T05:30:04.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407820+0000", 2024-09-14T05:30:04.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049868580000000003 2024-09-14T05:30:04.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407909+0000", 2024-09-14T05:30:04.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8887000000000005e-05 2024-09-14T05:30:04.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408237+0000", 2024-09-14T05:30:04.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032824599999999997 2024-09-14T05:30:04.188 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.188 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.188 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.189 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4002 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.189 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:04.189 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.733449618999998, 2024-09-14T05:30:04.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050539529999999999, 2024-09-14T05:30:04.189 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.189 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.189 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.189 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.189 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.189 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.189 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4002 2024-09-14T05:30:04.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.190 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:04.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:04.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:04.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5099999999999999e-07 2024-09-14T05:30:04.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358154+0000", 2024-09-14T05:30:04.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6799999999999999e-06 2024-09-14T05:30:04.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358165+0000", 2024-09-14T05:30:04.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0899e-05 2024-09-14T05:30:04.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358172+0000", 2024-09-14T05:30:04.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8689999999999998e-06 2024-09-14T05:30:04.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408281+0000", 2024-09-14T05:30:04.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050108699 2024-09-14T05:30:04.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408369+0000", 2024-09-14T05:30:04.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8542999999999996e-05 2024-09-14T05:30:04.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408689+0000", 2024-09-14T05:30:04.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032008900000000002 2024-09-14T05:30:04.193 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.193 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.193 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.193 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4003 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5177344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.193 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:04.193 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.733409784000003, 2024-09-14T05:30:04.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050868759, 2024-09-14T05:30:04.193 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.194 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.194 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.194 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.194 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.194 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.194 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4003 2024-09-14T05:30:04.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.194 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:04.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:04.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:04.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-07 2024-09-14T05:30:04.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358193+0000", 2024-09-14T05:30:04.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.588e-06 2024-09-14T05:30:04.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358204+0000", 2024-09-14T05:30:04.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1525999999999999e-05 2024-09-14T05:30:04.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358213+0000", 2024-09-14T05:30:04.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5010000000000004e-06 2024-09-14T05:30:04.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408719+0000", 2024-09-14T05:30:04.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050506708999999997 2024-09-14T05:30:04.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408766+0000", 2024-09-14T05:30:04.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6579999999999998e-05 2024-09-14T05:30:04.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409058+0000", 2024-09-14T05:30:04.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000292365 2024-09-14T05:30:04.197 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.198 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.198 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.198 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4004 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5668864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.198 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:04.198 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.732986941, 2024-09-14T05:30:04.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050839848, 2024-09-14T05:30:04.198 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.198 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.198 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.198 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.198 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.198 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.199 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4004 2024-09-14T05:30:04.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.199 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:04.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:04.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358613+0000", 2024-09-14T05:30:04.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:30:04.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358618+0000", 2024-09-14T05:30:04.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6009999999999997e-06 2024-09-14T05:30:04.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358630+0000", 2024-09-14T05:30:04.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2122e-05 2024-09-14T05:30:04.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358638+0000", 2024-09-14T05:30:04.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3969999999999994e-06 2024-09-14T05:30:04.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409088+0000", 2024-09-14T05:30:04.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050449565000000002 2024-09-14T05:30:04.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409153+0000", 2024-09-14T05:30:04.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4728999999999996e-05 2024-09-14T05:30:04.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409452+0000", 2024-09-14T05:30:04.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029958900000000001 2024-09-14T05:30:04.202 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.202 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.202 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.202 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4005 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6160384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.202 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:04.203 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.732770006999999, 2024-09-14T05:30:04.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051054752000000002, 2024-09-14T05:30:04.203 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.203 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.203 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.203 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4005 2024-09-14T05:30:04.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.203 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:04.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:04.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358830+0000", 2024-09-14T05:30:04.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-14T05:30:04.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358833+0000", 2024-09-14T05:30:04.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9529999999999999e-06 2024-09-14T05:30:04.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358843+0000", 2024-09-14T05:30:04.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8069999999999994e-06 2024-09-14T05:30:04.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358849+0000", 2024-09-14T05:30:04.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1040000000000003e-06 2024-09-14T05:30:04.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409487+0000", 2024-09-14T05:30:04.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050638626999999999 2024-09-14T05:30:04.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409535+0000", 2024-09-14T05:30:04.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7525000000000003e-05 2024-09-14T05:30:04.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409884+0000", 2024-09-14T05:30:04.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034914800000000003 2024-09-14T05:30:04.207 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.207 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.207 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.207 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4006 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.207 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:04.207 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.732732894999998, 2024-09-14T05:30:04.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051425323000000002, 2024-09-14T05:30:04.207 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.207 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.207 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.208 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4006 2024-09-14T05:30:04.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.208 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:04.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:04.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358867+0000", 2024-09-14T05:30:04.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2900000000000004e-07 2024-09-14T05:30:04.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358869+0000", 2024-09-14T05:30:04.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4210000000000002e-06 2024-09-14T05:30:04.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358876+0000", 2024-09-14T05:30:04.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4980000000000003e-06 2024-09-14T05:30:04.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358881+0000", 2024-09-14T05:30:04.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3090000000000003e-06 2024-09-14T05:30:04.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409915+0000", 2024-09-14T05:30:04.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051033348999999999 2024-09-14T05:30:04.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409964+0000", 2024-09-14T05:30:04.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9858000000000003e-05 2024-09-14T05:30:04.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410292+0000", 2024-09-14T05:30:04.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032735899999999999 2024-09-14T05:30:04.211 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.211 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.211 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.212 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4007 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.212 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:04.212 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.732710302000001, 2024-09-14T05:30:04.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05184076, 2024-09-14T05:30:04.212 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.212 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.212 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.212 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.212 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.212 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.212 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4007 2024-09-14T05:30:04.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.213 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:04.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:04.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:04.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:30:04.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358891+0000", 2024-09-14T05:30:04.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9819999999999998e-06 2024-09-14T05:30:04.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358897+0000", 2024-09-14T05:30:04.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9390000000000002e-06 2024-09-14T05:30:04.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358902+0000", 2024-09-14T05:30:04.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6770000000000003e-06 2024-09-14T05:30:04.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410335+0000", 2024-09-14T05:30:04.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051432991999999997 2024-09-14T05:30:04.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410416+0000", 2024-09-14T05:30:04.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1316999999999994e-05 2024-09-14T05:30:04.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410730+0000", 2024-09-14T05:30:04.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000313419 2024-09-14T05:30:04.216 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.216 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.216 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.216 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4008 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7634944~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.216 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:04.216 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.732685941, 2024-09-14T05:30:04.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052208082000000003, 2024-09-14T05:30:04.216 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.216 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.217 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.217 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.217 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.217 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.217 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4008 2024-09-14T05:30:04.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.217 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:04.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:04.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358914+0000", 2024-09-14T05:30:04.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6100000000000001e-07 2024-09-14T05:30:04.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358916+0000", 2024-09-14T05:30:04.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.902e-06 2024-09-14T05:30:04.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358923+0000", 2024-09-14T05:30:04.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4780000000000002e-06 2024-09-14T05:30:04.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358929+0000", 2024-09-14T05:30:04.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0970000000000002e-06 2024-09-14T05:30:04.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410761+0000", 2024-09-14T05:30:04.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051831436000000002 2024-09-14T05:30:04.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410829+0000", 2024-09-14T05:30:04.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7874999999999996e-05 2024-09-14T05:30:04.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411121+0000", 2024-09-14T05:30:04.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029283299999999999 2024-09-14T05:30:04.220 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.220 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.220 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.221 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4009 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8126464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:04.221 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:04.221 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.732538716000001, 2024-09-14T05:30:04.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052518169000000003, 2024-09-14T05:30:04.221 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:04.221 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:04.221 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:04.221 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:04.221 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:04.221 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.221 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4009 2024-09-14T05:30:04.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.222 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:04.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:04.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:04.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:04.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:04.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:04.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:04.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:04.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0200000000000002e-07 2024-09-14T05:30:04.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:04.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359066+0000", 2024-09-14T05:30:04.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7899999999999999e-06 2024-09-14T05:30:04.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:04.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359074+0000", 2024-09-14T05:30:04.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2579999999999995e-06 2024-09-14T05:30:04.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359082+0000", 2024-09-14T05:30:04.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.464e-06 2024-09-14T05:30:04.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:04.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411160+0000", 2024-09-14T05:30:04.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052078078 2024-09-14T05:30:04.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:04.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411244+0000", 2024-09-14T05:30:04.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4041000000000004e-05 2024-09-14T05:30:04.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:04.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:04.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:04.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411579+0000", 2024-09-14T05:30:04.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033493599999999998 2024-09-14T05:30:04.225 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.225 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.225 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.225 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:04.225 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:04.225 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:04.226 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T05:30:04.227 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:04.225+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:04.228 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:30:04.230 INFO:teuthology.orchestra.run.smithi071.stdout:3085 read 0x4d57d49 thru 0x4d5dd9a (0x6052 bytes) 2024-09-14T05:30:04.230 INFO:teuthology.orchestra.run.smithi071.stdout:3088 writesame 0x1a925ec thru 0x1a99165 data_size 0x1(0x6b7a bytes) 2024-09-14T05:30:04.231 INFO:teuthology.orchestra.run.smithi071.stdout:3089 trunc from 0xbafa19f to 0xbd316d4 2024-09-14T05:30:04.231 INFO:teuthology.orchestra.run.smithi071.stdout:3091 write 0xa915c64 thru 0xa92210c (0xc4a9 bytes) 2024-09-14T05:30:04.231 INFO:teuthology.orchestra.run.smithi071.stdout:3092 read 0x8b8ad0b thru 0x8b94030 (0x9326 bytes) 2024-09-14T05:30:04.231 INFO:teuthology.orchestra.run.smithi071.stdout:3094 writesame 0x30ddc12 thru 0x30ec9dc data_size 0x19(0xedcb bytes) 2024-09-14T05:30:04.231 INFO:teuthology.orchestra.run.smithi071.stdout:3095 trunc from 0xbd316d4 to 0x336eb62 2024-09-14T05:30:04.231 INFO:teuthology.orchestra.run.smithi071.stdout:3096 punch from 0x1b6b69f to 0x1b70b36, (0x5497 bytes) 2024-09-14T05:30:04.231 INFO:teuthology.orchestra.run.smithi071.stdout:3097 write 0x3c2b8b5 thru 0x3c381c2 (0xc90e bytes) 2024-09-14T05:30:04.231 INFO:teuthology.orchestra.run.smithi071.stdout:3099 writesame 0xb7b548e thru 0xb7b651b data_size 0xd(0x108e bytes) 2024-09-14T05:30:04.231 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.231 INFO:teuthology.orchestra.run.smithi071.stdout:3101 read 0x213e2fd thru 0x21485a6 (0xa2aa bytes) 2024-09-14T05:30:04.231 INFO:teuthology.orchestra.run.smithi071.stdout:3102 write 0x6c02c87 thru 0x6c03793 (0xb0d bytes) 2024-09-14T05:30:04.231 INFO:teuthology.orchestra.run.smithi071.stdout:3103 read 0x58de779 thru 0x58df64b (0xed3 bytes) 2024-09-14T05:30:04.231 INFO:teuthology.orchestra.run.smithi071.stdout:3104 punch from 0x5294c17 to 0x5297e8d, (0x3276 bytes) 2024-09-14T05:30:04.232 INFO:teuthology.orchestra.run.smithi071.stdout:3105 read 0x90d3b7e thru 0x90d8d5d (0x51e0 bytes) 2024-09-14T05:30:04.232 INFO:teuthology.orchestra.run.smithi071.stdout:3107 writesame 0x89af879 thru 0x89b9136 data_size 0x39(0x98be bytes) 2024-09-14T05:30:04.232 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.232 INFO:teuthology.orchestra.run.smithi071.stdout:3109 read 0x4fd1782 thru 0x4fdfe60 (0xe6df bytes) 2024-09-14T05:30:04.232 INFO:teuthology.orchestra.run.smithi071.stdout:3110 punch from 0x25e3365 to 0x25efde4, (0xca7f bytes) 2024-09-14T05:30:04.232 INFO:teuthology.orchestra.run.smithi071.stdout:3111 read 0x7e72529 thru 0x7e7c722 (0xa1fa bytes) 2024-09-14T05:30:04.232 INFO:teuthology.orchestra.run.smithi071.stdout:3112 write 0x6df0daa thru 0x6df5048 (0x429f bytes) 2024-09-14T05:30:04.232 INFO:teuthology.orchestra.run.smithi071.stdout:3113 read 0x482ba46 thru 0x4836cad (0xb268 bytes) 2024-09-14T05:30:04.232 INFO:teuthology.orchestra.run.smithi071.stdout:3115 read 0x2c3f5d9 thru 0x2c4232f (0x2d57 bytes) 2024-09-14T05:30:04.232 INFO:teuthology.orchestra.run.smithi071.stdout:3117 writesame 0xae721eb thru 0xae733a2 data_size 0x2a(0x11b8 bytes) 2024-09-14T05:30:04.232 INFO:teuthology.orchestra.run.smithi071.stdout:3118 writesame 0x25b9bd thru 0x26361f data_size 0x1(0x7c63 bytes) 2024-09-14T05:30:04.232 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.232 INFO:teuthology.orchestra.run.smithi071.stdout:3121 write 0x93055b1 thru 0x93135a7 (0xdff7 bytes) 2024-09-14T05:30:04.232 INFO:teuthology.orchestra.run.smithi071.stdout:3123 write 0xc1c5a95 thru 0xc1d0889 (0xadf5 bytes) 2024-09-14T05:30:04.233 INFO:teuthology.orchestra.run.smithi071.stdout:3124 write 0xb6d4c04 thru 0xb6dfaac (0xaea9 bytes) 2024-09-14T05:30:04.233 INFO:teuthology.orchestra.run.smithi071.stdout:3125 read 0x5f64dfd thru 0x5f714b7 (0xc6bb bytes) 2024-09-14T05:30:04.233 INFO:teuthology.orchestra.run.smithi071.stdout:3127 read 0x96bc11b thru 0x96bf302 (0x31e8 bytes) 2024-09-14T05:30:04.233 INFO:teuthology.orchestra.run.smithi071.stdout:3128 writesame 0x4a69f36 thru 0x4a78de3 data_size 0x2(0xeeae bytes) 2024-09-14T05:30:04.233 INFO:teuthology.orchestra.run.smithi071.stdout:3129 write 0x94df711 thru 0x94ec6ff (0xcfef bytes) 2024-09-14T05:30:04.235 INFO:teuthology.orchestra.run.smithi071.stdout:3131 writesame 0x47ac4b8 thru 0x47adc91 data_size 0x2(0x17da bytes) 2024-09-14T05:30:04.238 INFO:teuthology.orchestra.run.smithi071.stdout:3132 write 0x14cdf7 thru 0x1534aa (0x66b4 bytes) 2024-09-14T05:30:04.241 INFO:teuthology.orchestra.run.smithi071.stdout:3133 punch from 0x897cdc5 to 0x8981a8a, (0x4cc5 bytes) 2024-09-14T05:30:04.241 INFO:teuthology.orchestra.run.smithi071.stdout:3134 punch from 0x3163b5 to 0x31877e, (0x23c9 bytes) 2024-09-14T05:30:04.251 INFO:teuthology.orchestra.run.smithi071.stdout:3135 trunc from 0xc1d088a to 0x345a835 2024-09-14T05:30:04.266 INFO:teuthology.orchestra.run.smithi071.stdout:3137 read 0x1ced5ca thru 0x1cfca1e (0xf455 bytes) 2024-09-14T05:30:04.266 INFO:teuthology.orchestra.run.smithi071.stdout:3138 read 0x5a5310 thru 0x5ab992 (0x6683 bytes) 2024-09-14T05:30:04.271 INFO:teuthology.orchestra.run.smithi071.stdout:3139 write 0x35a0e2a thru 0x35aaf28 (0xa0ff bytes) 2024-09-14T05:30:04.275 INFO:teuthology.orchestra.run.smithi071.stdout:3140 punch from 0x564c1c to 0x56a3b8, (0x579c bytes) 2024-09-14T05:30:04.289 INFO:teuthology.orchestra.run.smithi071.stdout:3141 write 0xa4343bf thru 0xa436fd1 (0x2c13 bytes) 2024-09-14T05:30:04.298 INFO:teuthology.orchestra.run.smithi071.stdout:3142 writesame 0xa4fa943 thru 0xa4ffa76 data_size 0x2(0x5134 bytes) 2024-09-14T05:30:04.299 INFO:teuthology.orchestra.run.smithi071.stdout:3144 write 0x223fa58 thru 0x224ebe8 (0xf191 bytes) 2024-09-14T05:30:04.303 INFO:teuthology.orchestra.run.smithi071.stdout:3145 punch from 0x7761955 to 0x7763e96, (0x2541 bytes) 2024-09-14T05:30:04.303 INFO:teuthology.orchestra.run.smithi071.stdout:3146 write 0x6ea8fc thru 0x6f31ec (0x88f1 bytes) 2024-09-14T05:30:04.314 INFO:teuthology.orchestra.run.smithi071.stdout:3147 trunc from 0xa4ffa77 to 0x2789d9c 2024-09-14T05:30:04.324 INFO:teuthology.orchestra.run.smithi071.stdout:3150 read 0x1a6801c thru 0x1a6dcc6 (0x5cab bytes) 2024-09-14T05:30:04.327 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:04.326+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:04.336 INFO:teuthology.orchestra.run.smithi071.stdout:3152 writesame 0x84bb0a4 thru 0x84c9ffc data_size 0x1(0xef59 bytes) 2024-09-14T05:30:04.347 INFO:teuthology.orchestra.run.smithi071.stdout:3153 write 0xd5c90f0 thru 0xd5d8500 (0xf411 bytes) 2024-09-14T05:30:04.350 INFO:teuthology.orchestra.run.smithi071.stdout:3154 read 0x488e016 thru 0x4892579 (0x4564 bytes) 2024-09-14T05:30:04.352 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:04.353 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:04.357 INFO:teuthology.orchestra.run.smithi071.stdout:3155 trunc from 0xd5d8501 to 0x7f80597 2024-09-14T05:30:04.365 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:30:04.367 INFO:teuthology.orchestra.run.smithi071.stdout:3158 read 0x39a2fd9 thru 0x39a3eea (0xf12 bytes) 2024-09-14T05:30:04.367 INFO:teuthology.orchestra.run.smithi071.stdout:3159 read 0x1632cfb thru 0x163cc44 (0x9f4a bytes) 2024-09-14T05:30:04.367 INFO:teuthology.orchestra.run.smithi071.stdout:3160 write 0x2e4a25b thru 0x2e51f79 (0x7d1f bytes) 2024-09-14T05:30:04.370 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.370 INFO:teuthology.orchestra.run.smithi071.stdout:3162 punch from 0x7f20bf1 to 0x7f290fa, (0x8509 bytes) 2024-09-14T05:30:04.370 INFO:teuthology.orchestra.run.smithi071.stdout:3164 read 0x3019e1 thru 0x3065ed (0x4c0d bytes) 2024-09-14T05:30:04.370 INFO:teuthology.orchestra.run.smithi071.stdout:3165 writesame 0x46353fc thru 0x463b78d data_size 0x5(0x6392 bytes) 2024-09-14T05:30:04.375 INFO:teuthology.orchestra.run.smithi071.stdout:3166 punch from 0x4e9f000 to 0x4eaa3b9, (0xb3b9 bytes) 2024-09-14T05:30:04.376 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.376 INFO:teuthology.orchestra.run.smithi071.stdout:3168 read 0x4ebf648 thru 0x4ec1b4a (0x2503 bytes) 2024-09-14T05:30:04.376 INFO:teuthology.orchestra.run.smithi071.stdout:3169 write 0x41673b thru 0x41f49e (0x8d64 bytes) 2024-09-14T05:30:04.377 INFO:teuthology.orchestra.run.smithi071.stdout:3170 punch from 0x379e054 to 0x37a4c1d, (0x6bc9 bytes) 2024-09-14T05:30:04.381 INFO:teuthology.orchestra.run.smithi071.stdout:3171 trunc from 0x7f80597 to 0x43cf1ec 2024-09-14T05:30:04.391 INFO:teuthology.orchestra.run.smithi071.stdout:3174 read 0x34079d3 thru 0x3408057 (0x685 bytes) 2024-09-14T05:30:04.391 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.391 INFO:teuthology.orchestra.run.smithi071.stdout:3176 punch from 0x650307 to 0x65a61b, (0xa314 bytes) 2024-09-14T05:30:04.392 INFO:teuthology.orchestra.run.smithi071.stdout:3177 read 0x336979b thru 0x336b6c1 (0x1f27 bytes) 2024-09-14T05:30:04.392 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.392 INFO:teuthology.orchestra.run.smithi071.stdout:3179 read 0x3715d10 thru 0x371c15e (0x644f bytes) 2024-09-14T05:30:04.404 INFO:teuthology.orchestra.run.smithi071.stdout:3180 write 0xaabf88c thru 0xaac1b1c (0x2291 bytes) 2024-09-14T05:30:04.408 INFO:teuthology.orchestra.run.smithi071.stdout:3181 punch from 0x71a11d6 to 0x71aa0f1, (0x8f1b bytes) 2024-09-14T05:30:04.408 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.408 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.408 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.408 INFO:teuthology.orchestra.run.smithi071.stdout:3186 read 0x1a81830 thru 0x1a84aec (0x32bd bytes) 2024-09-14T05:30:04.409 INFO:teuthology.orchestra.run.smithi071.stdout:3189 punch from 0x6740aba to 0x67481d8, (0x771e bytes) 2024-09-14T05:30:04.409 INFO:teuthology.orchestra.run.smithi071.stdout:3190 punch from 0x4226808 to 0x422d274, (0x6a6c bytes) 2024-09-14T05:30:04.409 INFO:teuthology.orchestra.run.smithi071.stdout:3191 write 0x98bafed thru 0x98c7853 (0xc867 bytes) 2024-09-14T05:30:04.412 INFO:teuthology.orchestra.run.smithi071.stdout:3192 write 0x41737d4 thru 0x41815b6 (0xdde3 bytes) 2024-09-14T05:30:04.418 INFO:teuthology.orchestra.run.smithi071.stdout:3193 punch from 0x5f2f58 to 0x5f5883, (0x292b bytes) 2024-09-14T05:30:04.418 INFO:teuthology.orchestra.run.smithi071.stdout:3194 write 0x885d536 thru 0x8868805 (0xb2d0 bytes) 2024-09-14T05:30:04.421 INFO:teuthology.orchestra.run.smithi071.stdout:3196 writesame 0x4d78d91 thru 0x4d7e883 data_size 0xd(0x5af3 bytes) 2024-09-14T05:30:04.424 INFO:teuthology.orchestra.run.smithi071.stdout:3197 read 0x72bffa7 thru 0x72cfe5f (0xfeb9 bytes) 2024-09-14T05:30:04.428 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:04.427+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:04.429 INFO:teuthology.orchestra.run.smithi071.stdout:3198 trunc from 0xaac1b1d to 0x648c1e0 2024-09-14T05:30:04.453 INFO:teuthology.orchestra.run.smithi071.stdout:3201 write 0x997e0f0 thru 0x9984022 (0x5f33 bytes) 2024-09-14T05:30:04.465 INFO:teuthology.orchestra.run.smithi071.stdout:3202 trunc from 0x9984023 to 0x252165a 2024-09-14T05:30:04.483 INFO:teuthology.orchestra.run.smithi071.stdout:3203 trunc from 0x252165a to 0x908b8c3 2024-09-14T05:30:04.487 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:04.487 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:04.488 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:04.488 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:04.488 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:04.491 INFO:teuthology.orchestra.run.smithi071.stdout:3204 trunc from 0x908b8c3 to 0x6dec952 2024-09-14T05:30:04.497 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.497 INFO:teuthology.orchestra.run.smithi071.stdout:3206 write 0x2130595 thru 0x213614b (0x5bb7 bytes) 2024-09-14T05:30:04.500 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:30:04.502 INFO:teuthology.orchestra.run.smithi071.stdout:3207 read 0x6656d1b thru 0x666102a (0xa310 bytes) 2024-09-14T05:30:04.502 INFO:teuthology.orchestra.run.smithi071.stdout:3209 punch from 0x4c3215e to 0x4c33d0e, (0x1bb0 bytes) 2024-09-14T05:30:04.502 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.502 INFO:teuthology.orchestra.run.smithi071.stdout:3213 punch from 0x4ec9c44 to 0x4ed7ca6, (0xe062 bytes) 2024-09-14T05:30:04.502 INFO:teuthology.orchestra.run.smithi071.stdout:3214 trunc from 0x6dec952 to 0x73f4f29 2024-09-14T05:30:04.507 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.507 INFO:teuthology.orchestra.run.smithi071.stdout:3219 read 0x620b980 thru 0x6212b95 (0x7216 bytes) 2024-09-14T05:30:04.507 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.507 INFO:teuthology.orchestra.run.smithi071.stdout:3223 punch from 0x41cf47f to 0x41d5093, (0x5c14 bytes) 2024-09-14T05:30:04.508 INFO:teuthology.orchestra.run.smithi071.stdout:3224 read 0xd3b1cb thru 0xd3fc60 (0x4a96 bytes) 2024-09-14T05:30:04.521 INFO:teuthology.orchestra.run.smithi071.stdout:3226 write 0xe58bf27 thru 0xe590e5f (0x4f39 bytes) 2024-09-14T05:30:04.529 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:04.528+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:04.534 INFO:teuthology.orchestra.run.smithi071.stdout:3227 trunc from 0xe590e60 to 0x72b9b4b 2024-09-14T05:30:04.545 INFO:teuthology.orchestra.run.smithi071.stdout:3229 write 0x331d45e thru 0x332c442 (0xefe5 bytes) 2024-09-14T05:30:04.548 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.548 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.551 INFO:teuthology.orchestra.run.smithi071.stdout:3232 trunc from 0x72b9b4b to 0x4e115e9 2024-09-14T05:30:04.557 INFO:teuthology.orchestra.run.smithi071.stdout:3233 read 0x23490c1 thru 0x2355887 (0xc7c7 bytes) 2024-09-14T05:30:04.561 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:30:04.566 INFO:teuthology.orchestra.run.smithi071.stdout:3236 trunc from 0x4e115e9 to 0xb811a1d 2024-09-14T05:30:04.586 INFO:teuthology.orchestra.run.smithi071.stdout:3237 trunc from 0xb811a1d to 0x7cb608 2024-09-14T05:30:04.613 INFO:teuthology.orchestra.run.smithi071.stdout:3238 writesame 0x48d4122 thru 0x48e18c9 data_size 0x86(0xd7a8 bytes) 2024-09-14T05:30:04.613 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.624 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:04.624 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:04.624 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:04.624 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:04.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.624 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74187 3.f 3:f707b2d9:::rbd_data.112d46b1ca94.0000000000000061:11 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.624 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:04.624 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.054394313, 2024-09-14T05:30:04.625 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072367828999999995, 2024-09-14T05:30:04.625 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.625 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.625 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.625 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.625 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.625 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.625 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74187 2024-09-14T05:30:04.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.625 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.625 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.625 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:04.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:04.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:04.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0900000000000002e-07 2024-09-14T05:30:04.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568166+0000", 2024-09-14T05:30:04.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.683e-06 2024-09-14T05:30:04.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568172+0000", 2024-09-14T05:30:04.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0639999999999999e-06 2024-09-14T05:30:04.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.627 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568177+0000", 2024-09-14T05:30:04.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.967e-06 2024-09-14T05:30:04.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.628 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.628 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640193+0000", 2024-09-14T05:30:04.628 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072016284 2024-09-14T05:30:04.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.628 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.628 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640282+0000", 2024-09-14T05:30:04.628 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9011999999999994e-05 2024-09-14T05:30:04.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.628 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.628 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640532+0000", 2024-09-14T05:30:04.628 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024940999999999999 2024-09-14T05:30:04.629 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.629 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.629 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.629 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74188 3.f 3:f707b2d9:::rbd_data.112d46b1ca94.0000000000000061:11 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.629 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:04.629 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.054374940000001, 2024-09-14T05:30:04.629 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072600425999999996, 2024-09-14T05:30:04.629 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.629 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.629 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.629 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.629 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74188 2024-09-14T05:30:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568184+0000", 2024-09-14T05:30:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8700000000000001e-07 2024-09-14T05:30:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568200+0000", 2024-09-14T05:30:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6603e-05 2024-09-14T05:30:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568208+0000", 2024-09-14T05:30:04.632 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4800000000000004e-06 2024-09-14T05:30:04.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.632 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.632 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568213+0000", 2024-09-14T05:30:04.632 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6860000000000003e-06 2024-09-14T05:30:04.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.632 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.632 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640563+0000", 2024-09-14T05:30:04.632 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072349438000000002 2024-09-14T05:30:04.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640607+0000", 2024-09-14T05:30:04.633 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4014999999999998e-05 2024-09-14T05:30:04.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640784+0000", 2024-09-14T05:30:04.633 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017681700000000001 2024-09-14T05:30:04.633 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.633 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.633 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.633 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74194 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.634 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:04.634 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.054336808, 2024-09-14T05:30:04.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072795199000000005, 2024-09-14T05:30:04.634 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.634 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.634 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.634 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.634 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.634 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.634 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74194 2024-09-14T05:30:04.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.634 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.635 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.635 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:04.635 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.635 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.635 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:04.635 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.635 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.635 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568222+0000", 2024-09-14T05:30:04.635 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1300000000000001e-07 2024-09-14T05:30:04.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.636 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.636 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.636 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568224+0000", 2024-09-14T05:30:04.636 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0660000000000002e-06 2024-09-14T05:30:04.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.636 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.636 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.636 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568230+0000", 2024-09-14T05:30:04.636 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4470000000000003e-06 2024-09-14T05:30:04.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.636 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.636 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.636 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568235+0000", 2024-09-14T05:30:04.637 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1220000000000003e-06 2024-09-14T05:30:04.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.637 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.637 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640824+0000", 2024-09-14T05:30:04.637 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072588707000000002 2024-09-14T05:30:04.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.637 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.637 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640915+0000", 2024-09-14T05:30:04.637 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1008999999999995e-05 2024-09-14T05:30:04.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.638 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.638 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641017+0000", 2024-09-14T05:30:04.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010143499999999999 2024-09-14T05:30:04.638 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.638 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.638 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.638 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74208 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2693120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.638 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:04.638 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.054316232, 2024-09-14T05:30:04.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072905555999999996, 2024-09-14T05:30:04.638 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.639 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.639 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.639 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.639 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.639 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.639 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74208 2024-09-14T05:30:04.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.639 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:04.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.640 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:04.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.640 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:04.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9799999999999999e-07 2024-09-14T05:30:04.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.641 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.641 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568244+0000", 2024-09-14T05:30:04.641 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0339999999999999e-06 2024-09-14T05:30:04.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.641 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.641 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568251+0000", 2024-09-14T05:30:04.641 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4060000000000002e-06 2024-09-14T05:30:04.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.641 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.641 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568255+0000", 2024-09-14T05:30:04.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6589999999999998e-06 2024-09-14T05:30:04.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.642 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.642 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.642 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641048+0000", 2024-09-14T05:30:04.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072792647000000002 2024-09-14T05:30:04.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.642 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.642 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.642 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641086+0000", 2024-09-14T05:30:04.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8031999999999998e-05 2024-09-14T05:30:04.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.642 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.642 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.643 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641147+0000", 2024-09-14T05:30:04.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1379999999999998e-05 2024-09-14T05:30:04.643 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.643 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.643 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.643 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74236 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2662400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.643 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:04.643 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.053682568999999, 2024-09-14T05:30:04.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072517776000000006, 2024-09-14T05:30:04.643 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.643 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.643 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.644 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.644 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.644 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.644 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74236 2024-09-14T05:30:04.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.644 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:04.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:04.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568881+0000", 2024-09-14T05:30:04.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2340000000000002e-06 2024-09-14T05:30:04.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568886+0000", 2024-09-14T05:30:04.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9450000000000001e-06 2024-09-14T05:30:04.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.646 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568898+0000", 2024-09-14T05:30:04.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2150000000000001e-05 2024-09-14T05:30:04.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.646 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568907+0000", 2024-09-14T05:30:04.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9239999999999996e-06 2024-09-14T05:30:04.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.647 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641185+0000", 2024-09-14T05:30:04.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.07227807 2024-09-14T05:30:04.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.647 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641265+0000", 2024-09-14T05:30:04.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0575999999999996e-05 2024-09-14T05:30:04.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.647 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641393+0000", 2024-09-14T05:30:04.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012787700000000001 2024-09-14T05:30:04.648 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.648 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.648 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.648 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74239 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2695168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.648 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:04.648 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.053638176, 2024-09-14T05:30:04.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072593816000000005, 2024-09-14T05:30:04.648 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.648 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.648 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.648 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.649 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.649 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.649 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74239 2024-09-14T05:30:04.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.649 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.649 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.649 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:04.649 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.649 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.649 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:04.649 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:04.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4300000000000003e-07 2024-09-14T05:30:04.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.650 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568923+0000", 2024-09-14T05:30:04.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2019999999999998e-06 2024-09-14T05:30:04.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.650 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.651 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568933+0000", 2024-09-14T05:30:04.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0376e-05 2024-09-14T05:30:04.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.651 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.651 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568938+0000", 2024-09-14T05:30:04.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2850000000000002e-06 2024-09-14T05:30:04.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.651 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.651 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641424+0000", 2024-09-14T05:30:04.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072485230999999997 2024-09-14T05:30:04.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.652 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641464+0000", 2024-09-14T05:30:04.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0574000000000002e-05 2024-09-14T05:30:04.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.652 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641514+0000", 2024-09-14T05:30:04.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9604999999999997e-05 2024-09-14T05:30:04.652 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.652 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.652 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.652 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74252 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2664448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.653 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:04.653 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.053458261999999, 2024-09-14T05:30:04.653 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072631237000000001, 2024-09-14T05:30:04.653 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.653 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.653 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.653 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.653 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.653 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.653 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74252 2024-09-14T05:30:04.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.653 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.653 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.654 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:04.654 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.654 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.654 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.654 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:04.654 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.654 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.654 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.654 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569101+0000", 2024-09-14T05:30:04.654 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8700000000000005e-07 2024-09-14T05:30:04.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569104+0000", 2024-09-14T05:30:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.163e-06 2024-09-14T05:30:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569115+0000", 2024-09-14T05:30:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1335000000000001e-05 2024-09-14T05:30:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569121+0000", 2024-09-14T05:30:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0800000000000002e-06 2024-09-14T05:30:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641552+0000", 2024-09-14T05:30:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072430743000000006 2024-09-14T05:30:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641631+0000", 2024-09-14T05:30:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9388999999999995e-05 2024-09-14T05:30:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641731+0000", 2024-09-14T05:30:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9840000000000006e-05 2024-09-14T05:30:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74256 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2697216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.053427206, 2024-09-14T05:30:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072721090000000002, 2024-09-14T05:30:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74256 2024-09-14T05:30:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0299999999999999e-07 2024-09-14T05:30:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569133+0000", 2024-09-14T05:30:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.046e-06 2024-09-14T05:30:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569139+0000", 2024-09-14T05:30:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0279999999999997e-06 2024-09-14T05:30:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569144+0000", 2024-09-14T05:30:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6929999999999998e-06 2024-09-14T05:30:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641762+0000", 2024-09-14T05:30:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072617904999999996 2024-09-14T05:30:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641801+0000", 2024-09-14T05:30:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8902999999999997e-05 2024-09-14T05:30:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641852+0000", 2024-09-14T05:30:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1011999999999999e-05 2024-09-14T05:30:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74285 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2672640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.05258177, 2024-09-14T05:30:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072422069000000006, 2024-09-14T05:30:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74285 2024-09-14T05:30:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569977+0000", 2024-09-14T05:30:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3099999999999998e-07 2024-09-14T05:30:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569978+0000", 2024-09-14T05:30:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.598e-06 2024-09-14T05:30:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569984+0000", 2024-09-14T05:30:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4689999999999996e-06 2024-09-14T05:30:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569989+0000", 2024-09-14T05:30:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8409999999999999e-06 2024-09-14T05:30:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642229+0000", 2024-09-14T05:30:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072240046000000002 2024-09-14T05:30:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642306+0000", 2024-09-14T05:30:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7347000000000003e-05 2024-09-14T05:30:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642398+0000", 2024-09-14T05:30:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2336999999999994e-05 2024-09-14T05:30:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74292 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2705408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.052327062, 2024-09-14T05:30:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072361018999999999, 2024-09-14T05:30:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74292 2024-09-14T05:30:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570232+0000", 2024-09-14T05:30:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.06e-07 2024-09-14T05:30:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570235+0000", 2024-09-14T05:30:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3189999999999999e-06 2024-09-14T05:30:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570248+0000", 2024-09-14T05:30:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3006e-05 2024-09-14T05:30:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570254+0000", 2024-09-14T05:30:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.908e-06 2024-09-14T05:30:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642433+0000", 2024-09-14T05:30:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072179192000000003 2024-09-14T05:30:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642500+0000", 2024-09-14T05:30:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7080000000000001e-05 2024-09-14T05:30:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642592+0000", 2024-09-14T05:30:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1807999999999999e-05 2024-09-14T05:30:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74304 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2707456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.051970396, 2024-09-14T05:30:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072444960000000003, 2024-09-14T05:30:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74304 2024-09-14T05:30:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9800000000000004e-07 2024-09-14T05:30:04.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.673 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.673 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.673 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570590+0000", 2024-09-14T05:30:04.673 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0810000000000001e-06 2024-09-14T05:30:04.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.673 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.673 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.673 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570596+0000", 2024-09-14T05:30:04.673 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9789999999999998e-06 2024-09-14T05:30:04.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.673 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.673 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.673 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570601+0000", 2024-09-14T05:30:04.674 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6750000000000001e-06 2024-09-14T05:30:04.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.674 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.674 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642835+0000", 2024-09-14T05:30:04.674 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.07223359 2024-09-14T05:30:04.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.674 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.674 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642917+0000", 2024-09-14T05:30:04.674 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2013000000000002e-05 2024-09-14T05:30:04.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.675 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.675 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643033+0000", 2024-09-14T05:30:04.675 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116124 2024-09-14T05:30:04.675 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.675 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.675 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.675 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74321 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2709504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.675 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:04.675 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.051488406000001, 2024-09-14T05:30:04.676 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072350611999999995, 2024-09-14T05:30:04.676 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.676 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.676 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.676 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.676 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.676 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.676 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74321 2024-09-14T05:30:04.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.676 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.676 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.676 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.676 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:04.677 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.677 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.677 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:04.677 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.677 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.677 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:04.677 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3e-07 2024-09-14T05:30:04.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.677 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.678 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571073+0000", 2024-09-14T05:30:04.678 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6879999999999999e-06 2024-09-14T05:30:04.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.678 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.678 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.678 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571080+0000", 2024-09-14T05:30:04.678 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0319999999999998e-06 2024-09-14T05:30:04.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.678 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.678 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.678 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571085+0000", 2024-09-14T05:30:04.678 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2109999999999997e-06 2024-09-14T05:30:04.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.678 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.679 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.679 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643309+0000", 2024-09-14T05:30:04.679 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072223812999999998 2024-09-14T05:30:04.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.679 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.679 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643351+0000", 2024-09-14T05:30:04.679 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2239e-05 2024-09-14T05:30:04.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.679 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.679 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643420+0000", 2024-09-14T05:30:04.679 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8998999999999994e-05 2024-09-14T05:30:04.679 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.680 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.680 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.680 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.680 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74332 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2678784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.680 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:04.680 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.051165243, 2024-09-14T05:30:04.680 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072259062999999998, 2024-09-14T05:30:04.680 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.680 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.680 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.680 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.680 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.680 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.681 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74332 2024-09-14T05:30:04.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.681 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.681 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.681 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:04.681 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.681 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.681 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:04.681 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.682 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.682 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571394+0000", 2024-09-14T05:30:04.682 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:30:04.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.682 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.682 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.682 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571397+0000", 2024-09-14T05:30:04.682 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8469999999999998e-06 2024-09-14T05:30:04.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.682 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.682 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.682 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571411+0000", 2024-09-14T05:30:04.682 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.34e-05 2024-09-14T05:30:04.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.683 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.683 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.683 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571418+0000", 2024-09-14T05:30:04.683 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9149999999999999e-06 2024-09-14T05:30:04.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.683 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.683 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.683 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643458+0000", 2024-09-14T05:30:04.683 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072039881 2024-09-14T05:30:04.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.683 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.683 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.683 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643539+0000", 2024-09-14T05:30:04.684 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1650000000000006e-05 2024-09-14T05:30:04.684 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.684 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.684 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.684 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643652+0000", 2024-09-14T05:30:04.684 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000112593 2024-09-14T05:30:04.684 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.684 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.684 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.684 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.684 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.684 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74336 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2711552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.684 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:04.685 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.051128793, 2024-09-14T05:30:04.685 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072354038999999995, 2024-09-14T05:30:04.685 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.685 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.685 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.685 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.685 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.685 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.685 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74336 2024-09-14T05:30:04.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.685 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.685 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.685 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.685 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:04.686 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.686 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.686 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.686 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:04.686 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.686 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.686 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.686 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571430+0000", 2024-09-14T05:30:04.686 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8999999999999996e-07 2024-09-14T05:30:04.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.686 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.686 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.687 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571432+0000", 2024-09-14T05:30:04.687 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4159999999999998e-06 2024-09-14T05:30:04.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.687 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.687 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.687 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571439+0000", 2024-09-14T05:30:04.687 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.657e-06 2024-09-14T05:30:04.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.687 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.687 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.687 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571444+0000", 2024-09-14T05:30:04.687 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1329999999999998e-06 2024-09-14T05:30:04.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.688 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.688 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643690+0000", 2024-09-14T05:30:04.688 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072246088 2024-09-14T05:30:04.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.688 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.688 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643731+0000", 2024-09-14T05:30:04.688 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0311000000000001e-05 2024-09-14T05:30:04.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.688 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.689 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643783+0000", 2024-09-14T05:30:04.689 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2843999999999998e-05 2024-09-14T05:30:04.689 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.689 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.689 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.689 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74349 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.689 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:04.689 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.051107783999999, 2024-09-14T05:30:04.689 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072554297000000004, 2024-09-14T05:30:04.689 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.689 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.690 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.690 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.690 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.690 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.690 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74349 2024-09-14T05:30:04.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.690 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.690 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.690 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.690 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:04.690 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.690 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.691 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.691 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:04.691 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.691 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.691 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.691 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.691 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571451+0000", 2024-09-14T05:30:04.691 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.41e-07 2024-09-14T05:30:04.691 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.691 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.691 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.691 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571453+0000", 2024-09-14T05:30:04.691 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8360000000000001e-06 2024-09-14T05:30:04.691 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.692 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.692 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.692 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571475+0000", 2024-09-14T05:30:04.692 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2452000000000001e-05 2024-09-14T05:30:04.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.692 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.692 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.692 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571480+0000", 2024-09-14T05:30:04.692 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4430000000000002e-06 2024-09-14T05:30:04.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.692 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.692 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.692 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643821+0000", 2024-09-14T05:30:04.693 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072340347999999999 2024-09-14T05:30:04.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.693 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.693 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.693 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643904+0000", 2024-09-14T05:30:04.693 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3534000000000004e-05 2024-09-14T05:30:04.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.693 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.693 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.693 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644005+0000", 2024-09-14T05:30:04.693 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100343 2024-09-14T05:30:04.693 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.693 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.694 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.694 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.694 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74357 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2713600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.694 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:04.694 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.051063750999999, 2024-09-14T05:30:04.694 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072651144000000001, 2024-09-14T05:30:04.694 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.694 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.694 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.694 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.694 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.694 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.694 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74357 2024-09-14T05:30:04.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.695 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.695 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.695 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.695 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:04.695 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.695 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.695 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.695 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:04.695 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.695 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.696 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.696 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571495+0000", 2024-09-14T05:30:04.696 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8899999999999999e-07 2024-09-14T05:30:04.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.696 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.696 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.696 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571498+0000", 2024-09-14T05:30:04.696 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5840000000000002e-06 2024-09-14T05:30:04.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.696 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.696 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.696 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571504+0000", 2024-09-14T05:30:04.696 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6549999999999998e-06 2024-09-14T05:30:04.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.697 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.697 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.697 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571509+0000", 2024-09-14T05:30:04.697 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6500000000000004e-06 2024-09-14T05:30:04.697 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.697 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.697 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.697 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644043+0000", 2024-09-14T05:30:04.697 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072533832000000006 2024-09-14T05:30:04.697 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.697 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.697 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.697 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644090+0000", 2024-09-14T05:30:04.698 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7426000000000003e-05 2024-09-14T05:30:04.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.698 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.698 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.698 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644146+0000", 2024-09-14T05:30:04.698 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5408000000000002e-05 2024-09-14T05:30:04.698 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.698 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.698 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.698 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.698 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74364 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2682880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.698 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:04.698 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.050726979, 2024-09-14T05:30:04.699 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072531227000000004, 2024-09-14T05:30:04.699 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.699 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.699 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.699 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.699 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.699 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.699 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74364 2024-09-14T05:30:04.699 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.699 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.699 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.699 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.699 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:04.700 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.700 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.700 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:04.700 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.700 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.700 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571832+0000", 2024-09-14T05:30:04.700 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4e-07 2024-09-14T05:30:04.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.700 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.701 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571838+0000", 2024-09-14T05:30:04.701 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0519999999999998e-06 2024-09-14T05:30:04.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.701 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.701 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.701 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571865+0000", 2024-09-14T05:30:04.701 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7109000000000001e-05 2024-09-14T05:30:04.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.701 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.701 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.701 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571883+0000", 2024-09-14T05:30:04.701 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7428e-05 2024-09-14T05:30:04.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.702 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.702 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644182+0000", 2024-09-14T05:30:04.702 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072299551000000004 2024-09-14T05:30:04.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.702 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.702 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644260+0000", 2024-09-14T05:30:04.702 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7551000000000003e-05 2024-09-14T05:30:04.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.702 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.702 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644362+0000", 2024-09-14T05:30:04.702 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010259600000000001 2024-09-14T05:30:04.703 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.703 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.703 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.703 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.703 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.703 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74367 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2715648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.703 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:04.703 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.050363672, 2024-09-14T05:30:04.703 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072368259000000004, 2024-09-14T05:30:04.703 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.703 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.703 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.703 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.704 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.704 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.704 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74367 2024-09-14T05:30:04.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.704 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.704 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.704 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:04.704 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.704 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.704 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:04.704 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.705 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.705 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572195+0000", 2024-09-14T05:30:04.705 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.78e-07 2024-09-14T05:30:04.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.705 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.705 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572200+0000", 2024-09-14T05:30:04.705 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2520000000000001e-06 2024-09-14T05:30:04.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.706 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.706 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572213+0000", 2024-09-14T05:30:04.706 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3618000000000001e-05 2024-09-14T05:30:04.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.706 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.706 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572226+0000", 2024-09-14T05:30:04.706 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3040999999999999e-05 2024-09-14T05:30:04.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.706 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.707 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644409+0000", 2024-09-14T05:30:04.707 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072182982000000007 2024-09-14T05:30:04.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.707 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.707 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644489+0000", 2024-09-14T05:30:04.707 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9429000000000001e-05 2024-09-14T05:30:04.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.707 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.707 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644563+0000", 2024-09-14T05:30:04.707 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4059000000000004e-05 2024-09-14T05:30:04.707 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.708 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.708 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.708 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74380 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2684928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.708 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:04.708 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.050229893999999, 2024-09-14T05:30:04.708 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072442180999999994, 2024-09-14T05:30:04.708 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.708 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.708 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.708 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.708 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.708 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.709 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74380 2024-09-14T05:30:04.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.709 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:04.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:04.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.710 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.710 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572329+0000", 2024-09-14T05:30:04.710 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3300000000000001e-07 2024-09-14T05:30:04.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.710 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.710 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572333+0000", 2024-09-14T05:30:04.710 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8e-06 2024-09-14T05:30:04.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.710 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.710 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572346+0000", 2024-09-14T05:30:04.710 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.307e-05 2024-09-14T05:30:04.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.711 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.711 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572353+0000", 2024-09-14T05:30:04.711 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9e-06 2024-09-14T05:30:04.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.711 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.711 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644597+0000", 2024-09-14T05:30:04.711 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072244770999999999 2024-09-14T05:30:04.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.711 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.711 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644672+0000", 2024-09-14T05:30:04.712 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4945999999999997e-05 2024-09-14T05:30:04.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.712 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.712 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644770+0000", 2024-09-14T05:30:04.712 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7961000000000006e-05 2024-09-14T05:30:04.712 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.712 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.712 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.712 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74383 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2717696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:04.712 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:04.713 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.049973899999999, 2024-09-14T05:30:04.713 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072397258000000006, 2024-09-14T05:30:04.713 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:04.713 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:04.713 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:04.713 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:04.713 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:04.713 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:04.713 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74383 2024-09-14T05:30:04.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.713 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:04.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.713 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:04.713 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:04.714 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.714 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:04.714 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:04.714 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:04.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.714 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:04.714 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572585+0000", 2024-09-14T05:30:04.714 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:30:04.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.715 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:04.715 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572589+0000", 2024-09-14T05:30:04.715 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:30:04.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.715 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.715 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:04.715 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572602+0000", 2024-09-14T05:30:04.715 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2917e-05 2024-09-14T05:30:04.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.715 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.715 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:04.715 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572610+0000", 2024-09-14T05:30:04.715 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.943e-06 2024-09-14T05:30:04.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.716 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.716 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:04.716 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644791+0000", 2024-09-14T05:30:04.716 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072181113000000005 2024-09-14T05:30:04.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.716 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.716 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:04.716 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644861+0000", 2024-09-14T05:30:04.716 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0650999999999998e-05 2024-09-14T05:30:04.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:04.716 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:04.716 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:04.716 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644981+0000", 2024-09-14T05:30:04.716 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000120185 2024-09-14T05:30:04.717 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.717 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.717 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.717 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:04.717 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:04.717 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:04.718 INFO:teuthology.orchestra.run.smithi071.stdout:3241 writesame 0xd9c4b0f thru 0xd9c579c data_size 0x1(0xc8e bytes) 2024-09-14T05:30:04.718 INFO:teuthology.orchestra.run.smithi071.stdout:3242 read 0x31baf77 thru 0x31c22d4 (0x735e bytes) 2024-09-14T05:30:04.718 INFO:teuthology.orchestra.run.smithi071.stdout:3243 write 0xb044771 thru 0xb0518e6 (0xd176 bytes) 2024-09-14T05:30:04.718 INFO:teuthology.orchestra.run.smithi071.stdout:3244 write 0x72b23e7 thru 0x72c1e7f (0xfa99 bytes) 2024-09-14T05:30:04.719 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:30:04.721 INFO:teuthology.orchestra.run.smithi071.stdout:3245 punch from 0x5675175 to 0x5675eee, (0xd79 bytes) 2024-09-14T05:30:04.721 INFO:teuthology.orchestra.run.smithi071.stdout:3248 punch from 0xc0ab64b to 0xc0b3311, (0x7cc6 bytes) 2024-09-14T05:30:04.721 INFO:teuthology.orchestra.run.smithi071.stdout:3250 writesame 0xd271293 thru 0xd27656b data_size 0x1(0x52d9 bytes) 2024-09-14T05:30:04.721 INFO:teuthology.orchestra.run.smithi071.stdout:3251 read 0x95049d3 thru 0x950f92c (0xaf5a bytes) 2024-09-14T05:30:04.721 INFO:teuthology.orchestra.run.smithi071.stdout:3252 write 0x1e6a175 thru 0x1e70cf8 (0x6b84 bytes) 2024-09-14T05:30:04.721 INFO:teuthology.orchestra.run.smithi071.stdout:3255 read 0x43cf747 thru 0x43d7c3a (0x84f4 bytes) 2024-09-14T05:30:04.721 INFO:teuthology.orchestra.run.smithi071.stdout:3256 write 0x938843 thru 0x943629 (0xade7 bytes) 2024-09-14T05:30:04.721 INFO:teuthology.orchestra.run.smithi071.stdout:3258 read 0x749cbfb thru 0x74a7d27 (0xb12d bytes) 2024-09-14T05:30:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:3261 trunc from 0xd9c579d to 0x67c11b7 2024-09-14T05:30:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:3263 write 0xa1d689d thru 0xa1dc1c7 (0x592b bytes) 2024-09-14T05:30:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:3264 read 0x8301822 thru 0x830bef1 (0xa6d0 bytes) 2024-09-14T05:30:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:3268 writesame 0x91176b6 thru 0x9126105 data_size 0x5c(0xea50 bytes) 2024-09-14T05:30:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:3271 read 0x5d8761a thru 0x5d955cb (0xdfb2 bytes) 2024-09-14T05:30:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:04.723 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:04.723 INFO:teuthology.orchestra.run.smithi071.stdout:3273 write 0xe756529 thru 0xe7644a2 (0xdf7a bytes) 2024-09-14T05:30:04.723 INFO:teuthology.orchestra.run.smithi071.stdout:3274 read 0xb7571bd thru 0xb75e86b (0x76af bytes) 2024-09-14T05:30:04.724 INFO:teuthology.orchestra.run.smithi071.stdout:3275 punch from 0x9c359f6 to 0x9c39c46, (0x4250 bytes) 2024-09-14T05:30:04.724 INFO:teuthology.orchestra.run.smithi071.stdout:3276 writesame 0x967c464 thru 0x968ac7d data_size 0x9(0xe81a bytes) 2024-09-14T05:30:04.724 INFO:teuthology.orchestra.run.smithi071.stdout:3277 write 0xcca48c0 thru 0xccac709 (0x7e4a bytes) 2024-09-14T05:30:04.724 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:04.717+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:04.725 INFO:teuthology.orchestra.run.smithi071.stdout:3278 punch from 0x5d8a1ed to 0x5d8a4ee, (0x301 bytes) 2024-09-14T05:30:04.725 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.725 INFO:teuthology.orchestra.run.smithi071.stdout:3281 read 0xe03796e thru 0xe03f78d (0x7e20 bytes) 2024-09-14T05:30:04.725 INFO:teuthology.orchestra.run.smithi071.stdout:3282 write 0x9bfd3ee thru 0x9c08c6a (0xb87d bytes) 2024-09-14T05:30:04.725 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:04.725 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:04.726 INFO:teuthology.orchestra.run.smithi071.stdout:3283 read 0xaae99ee thru 0xaaf94d8 (0xfaeb bytes) 2024-09-14T05:30:04.726 INFO:teuthology.orchestra.run.smithi071.stdout:3284 write 0xe98be7 thru 0xe98d7d (0x197 bytes) 2024-09-14T05:30:04.726 INFO:teuthology.orchestra.run.smithi071.stdout:3285 writesame 0x6838f77 thru 0x6839bcb data_size 0xb(0xc55 bytes) 2024-09-14T05:30:04.729 INFO:teuthology.orchestra.run.smithi071.stdout:3286 write 0xe6436c thru 0xe6ff7d (0xbc12 bytes) 2024-09-14T05:30:04.732 INFO:teuthology.orchestra.run.smithi071.stdout:3287 write 0x15d1662 thru 0x15d331b (0x1cba bytes) 2024-09-14T05:30:04.735 INFO:teuthology.orchestra.run.smithi071.stdout:3288 read 0xcf2b153 thru 0xcf2e364 (0x3212 bytes) 2024-09-14T05:30:04.735 INFO:teuthology.orchestra.run.smithi071.stdout:3290 writesame 0x3e4203d thru 0x3e4de88 data_size 0x26(0xbe4c bytes) 2024-09-14T05:30:04.738 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.739 INFO:teuthology.orchestra.run.smithi071.stdout:3292 read 0xc6a2306 thru 0xc6a7569 (0x5264 bytes) 2024-09-14T05:30:04.739 INFO:teuthology.orchestra.run.smithi071.stdout:3293 punch from 0xc574f1b to 0xc57ae70, (0x5f55 bytes) 2024-09-14T05:30:04.739 INFO:teuthology.orchestra.run.smithi071.stdout:3294 punch from 0x28b366d to 0x28b3a42, (0x3d5 bytes) 2024-09-14T05:30:04.741 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:04.741 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:04.750 INFO:teuthology.orchestra.run.smithi071.stdout:3295 trunc from 0xe7644a3 to 0x5205255 2024-09-14T05:30:04.764 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:04.764 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:04.768 INFO:teuthology.orchestra.run.smithi071.stdout:3297 write 0x5a2b94f thru 0x5a30423 (0x4ad5 bytes) 2024-09-14T05:30:04.768 INFO:teuthology.orchestra.run.smithi071.stdout:3300 write 0x58e7c9a thru 0x58f5e2b (0xe192 bytes) 2024-09-14T05:30:04.771 INFO:teuthology.orchestra.run.smithi071.stdout:3301 write 0x37a0c65 thru 0x37a638c (0x5728 bytes) 2024-09-14T05:30:04.774 INFO:teuthology.orchestra.run.smithi071.stdout:3304 punch from 0x2e0af93 to 0x2e0d582, (0x25ef bytes) 2024-09-14T05:30:04.774 INFO:teuthology.orchestra.run.smithi071.stdout:3306 write 0x3c438d1 thru 0x3c48ebc (0x55ec bytes) 2024-09-14T05:30:04.777 INFO:teuthology.orchestra.run.smithi071.stdout:3307 read 0x3bcb730 thru 0x3bd88cb (0xd19c bytes) 2024-09-14T05:30:04.777 INFO:teuthology.orchestra.run.smithi071.stdout:3309 read 0x1eb49a2 thru 0x1ec00f1 (0xb750 bytes) 2024-09-14T05:30:04.777 INFO:teuthology.orchestra.run.smithi071.stdout:3311 punch from 0x46f1024 to 0x46f2040, (0x101c bytes) 2024-09-14T05:30:04.777 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:04.783 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:04.783 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:04.786 INFO:teuthology.orchestra.run.smithi071.stdout:3313 trunc from 0x5a30424 to 0xd498332 2024-09-14T05:30:04.792 INFO:teuthology.orchestra.run.smithi071.stdout:3314 write 0x3d67250 thru 0x3d75a05 (0xe7b6 bytes) 2024-09-14T05:30:04.795 INFO:teuthology.orchestra.run.smithi071.stdout:3315 read 0x944519e thru 0x94530f3 (0xdf56 bytes) 2024-09-14T05:30:04.795 INFO:teuthology.orchestra.run.smithi071.stdout:3317 clone 27 order 19 su 262144 sc 10 2024-09-14T05:30:04.801 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:04.801 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:04.817 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:04.818 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:04.819 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:04.818+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:04.834 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:04.834 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:04.841 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:04.841 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:04.850 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:30:04.919 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:04.918+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:04.994 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:04.994 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:05.003 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:30:05.047 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:05.046+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:05.105 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:05.105 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:05.114 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:30:05.155 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:05.154+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:05.222 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:05.222+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:05.224 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-14T05:30:05.266 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:05.266 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:05.276 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:30:05.279 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone26 from 0xd498332 (overlap 0x38fc40) to 0x0 2024-09-14T05:30:05.323 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:05.322+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:05.332 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-14T05:30:05.408 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:05.409 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:05.424 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:05.422+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:05.429 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:30:05.525 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:05.525+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:05.576 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:05.576 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:05.585 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:30:05.589 INFO:teuthology.orchestra.run.smithi071.stdout:3318 trunc from 0xd498332 to 0xe55e4f8 2024-09-14T05:30:05.608 INFO:teuthology.orchestra.run.smithi071.stdout:3319 trunc from 0xe55e4f8 to 0x4a7f70b 2024-09-14T05:30:05.625 INFO:teuthology.orchestra.run.smithi071.stdout:3320 writesame 0x4ea5afd thru 0x4ea9c8a data_size 0x1(0x418e bytes) 2024-09-14T05:30:05.625 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:05.626 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:05.625+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:05.630 INFO:teuthology.orchestra.run.smithi071.stdout:3322 writesame 0x534775d thru 0x5349dd7 data_size 0x1(0x267b bytes) 2024-09-14T05:30:05.630 INFO:teuthology.orchestra.run.smithi071.stdout:3324 writesame 0x4a44e2a thru 0x4a546b2 data_size 0x19(0xf889 bytes) 2024-09-14T05:30:05.642 INFO:teuthology.orchestra.run.smithi071.stdout:3325 trunc from 0x5349dd8 to 0xdf8aafc 2024-09-14T05:30:05.656 INFO:teuthology.orchestra.run.smithi071.stdout:3326 trunc from 0xdf8aafc to 0x6c67e39 2024-09-14T05:30:05.661 INFO:teuthology.orchestra.run.smithi071.stdout:3327 write 0x19a7054 thru 0x19aa402 (0x33af bytes) 2024-09-14T05:30:05.664 INFO:teuthology.orchestra.run.smithi071.stdout:3329 read 0x2d18249 thru 0x2d1f1a5 (0x6f5d bytes) 2024-09-14T05:30:05.664 INFO:teuthology.orchestra.run.smithi071.stdout:3330 punch from 0x60e24e9 to 0x60efde5, (0xd8fc bytes) 2024-09-14T05:30:05.664 INFO:teuthology.orchestra.run.smithi071.stdout:3332 read 0x6349011 thru 0x6353f84 (0xaf74 bytes) 2024-09-14T05:30:05.673 INFO:teuthology.orchestra.run.smithi071.stdout:3333 write 0xa0bc9b0 thru 0xa0c9595 (0xcbe6 bytes) 2024-09-14T05:30:05.684 INFO:teuthology.orchestra.run.smithi071.stdout:3335 trunc from 0xa0c9596 to 0x5df8d7b 2024-09-14T05:30:05.693 INFO:teuthology.orchestra.run.smithi071.stdout:3340 punch from 0x35927cf to 0x3594f6b, (0x279c bytes) 2024-09-14T05:30:05.693 INFO:teuthology.orchestra.run.smithi071.stdout:3341 punch from 0x3fa720e to 0x3fb16d1, (0xa4c3 bytes) 2024-09-14T05:30:05.696 INFO:teuthology.orchestra.run.smithi071.stdout:3342 trunc from 0x5df8d7b to 0x8a6c13e 2024-09-14T05:30:05.702 INFO:teuthology.orchestra.run.smithi071.stdout:3343 read 0x514ede5 thru 0x5157c44 (0x8e60 bytes) 2024-09-14T05:30:05.703 INFO:teuthology.orchestra.run.smithi071.stdout:3344 write 0x31b6ae7 thru 0x31be48f (0x79a9 bytes) 2024-09-14T05:30:05.710 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:05.710 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:05.710 INFO:teuthology.orchestra.run.smithi071.stdout:3345 trunc from 0x8a6c13e to 0x472de96 2024-09-14T05:30:05.721 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:30:05.726 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:05.725+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:05.734 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:05.734 INFO:teuthology.orchestra.run.smithi071.stdout:3349 punch from 0x3e8ab14 to 0x3e8cc48, (0x2134 bytes) 2024-09-14T05:30:05.734 INFO:teuthology.orchestra.run.smithi071.stdout:3350 write 0x41c71b8 thru 0x41ccf08 (0x5d51 bytes) 2024-09-14T05:30:05.754 INFO:teuthology.orchestra.run.smithi071.stdout:3351 writesame 0xde0fd29 thru 0xde1ab4a data_size 0x1f(0xae22 bytes) 2024-09-14T05:30:05.754 INFO:teuthology.orchestra.run.smithi071.stdout:3355 punch from 0x8b3ed61 to 0x8b457aa, (0x6a49 bytes) 2024-09-14T05:30:05.754 INFO:teuthology.orchestra.run.smithi071.stdout:3357 writesame 0x7248efe thru 0x7250df8 data_size 0x1(0x7efb bytes) 2024-09-14T05:30:05.772 INFO:teuthology.orchestra.run.smithi071.stdout:3358 read 0x3a344bb thru 0x3a38235 (0x3d7b bytes) 2024-09-14T05:30:05.772 INFO:teuthology.orchestra.run.smithi071.stdout:3361 write 0x7ab510b thru 0x7abb469 (0x635f bytes) 2024-09-14T05:30:05.775 INFO:teuthology.orchestra.run.smithi071.stdout:3362 write 0xc445ab6 thru 0xc44e7b6 (0x8d01 bytes) 2024-09-14T05:30:05.778 INFO:teuthology.orchestra.run.smithi071.stdout:3364 write 0x745166e thru 0x7451c72 (0x605 bytes) 2024-09-14T05:30:05.780 INFO:teuthology.orchestra.run.smithi071.stdout:3365 read 0x3e7a4d1 thru 0x3e84e6b (0xa99b bytes) 2024-09-14T05:30:05.781 INFO:teuthology.orchestra.run.smithi071.stdout:3366 writesame 0xbc608ce thru 0xbc68d5b data_size 0x2f(0x848e bytes) 2024-09-14T05:30:05.784 INFO:teuthology.orchestra.run.smithi071.stdout:3368 read 0x2f17964 thru 0x2f218cf (0x9f6c bytes) 2024-09-14T05:30:05.784 INFO:teuthology.orchestra.run.smithi071.stdout:3369 punch from 0x8956741 to 0x8964d01, (0xe5c0 bytes) 2024-09-14T05:30:05.801 INFO:teuthology.orchestra.run.smithi071.stdout:3370 trunc from 0xde1ab4b to 0x11d7843 2024-09-14T05:30:05.821 INFO:teuthology.orchestra.run.smithi071.stdout:3371 read 0x929993 thru 0x9346ca (0xad38 bytes) 2024-09-14T05:30:05.822 INFO:teuthology.orchestra.run.smithi071.stdout:3372 read 0xdf5a1f thru 0xdfe1ce (0x87b0 bytes) 2024-09-14T05:30:05.822 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:05.827 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:05.826+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:05.841 INFO:teuthology.orchestra.run.smithi071.stdout:3374 writesame 0xc600927 thru 0xc609e5f data_size 0x1(0x9539 bytes) 2024-09-14T05:30:05.841 INFO:teuthology.orchestra.run.smithi071.stdout:3375 read 0x7070305 thru 0x707e4aa (0xe1a6 bytes) 2024-09-14T05:30:05.842 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:05.842 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:05.842 INFO:teuthology.orchestra.run.smithi071.stdout:3379 writesame 0x13e016e thru 0x13e9254 data_size 0x5(0x90e7 bytes) 2024-09-14T05:30:05.845 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:05.845 INFO:teuthology.orchestra.run.smithi071.stdout:3385 writesame 0x510453d thru 0x51117b9 data_size 0xd(0xd27d bytes) 2024-09-14T05:30:05.849 INFO:teuthology.orchestra.run.smithi071.stdout:3386 read 0x82b3ad3 thru 0x82b604d (0x257b bytes) 2024-09-14T05:30:05.850 INFO:teuthology.orchestra.run.smithi071.stdout:3387 write 0x56b82f thru 0x57429e (0x8a70 bytes) 2024-09-14T05:30:05.853 INFO:teuthology.orchestra.run.smithi071.stdout:3389 write 0x18cf203 thru 0x18debe8 (0xf9e6 bytes) 2024-09-14T05:30:05.857 INFO:teuthology.orchestra.run.smithi071.stdout:3391 read 0xc948f4 thru 0xc9e8e2 (0x9fef bytes) 2024-09-14T05:30:05.857 INFO:teuthology.orchestra.run.smithi071.stdout:3392 read 0x8c6aed0 thru 0x8c707a8 (0x58d9 bytes) 2024-09-14T05:30:05.858 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:05.859 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:05.869 INFO:teuthology.orchestra.run.smithi071.stdout:3393 trunc from 0xc609e60 to 0x195a7ec 2024-09-14T05:30:05.872 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:30:05.878 INFO:teuthology.orchestra.run.smithi071.stdout:3394 punch from 0x78a898 to 0x7988a4, (0xe00c bytes) 2024-09-14T05:30:05.878 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:05.891 INFO:teuthology.orchestra.run.smithi071.stdout:3396 trunc from 0x195a7ec to 0xc37aaca 2024-09-14T05:30:05.904 INFO:teuthology.orchestra.run.smithi071.stdout:3398 writesame 0xede2044 thru 0xede8a19 data_size 0x2e(0x69d6 bytes) 2024-09-14T05:30:05.904 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:05.913 INFO:teuthology.orchestra.run.smithi071.stdout:3400 trunc from 0xede8a1a to 0x8000f82 2024-09-14T05:30:05.921 INFO:teuthology.orchestra.run.smithi071.stdout:3401 trunc from 0x8000f82 to 0x4df3bdf 2024-09-14T05:30:05.927 INFO:teuthology.orchestra.run.smithi071.stdout:3402 read 0x250f9a9 thru 0x251a06e (0xa6c6 bytes) 2024-09-14T05:30:05.927 INFO:teuthology.orchestra.run.smithi071.stdout:3403 punch from 0x3dc3cb9 to 0x3dce3e2, (0xa729 bytes) 2024-09-14T05:30:05.928 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:05.927+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:05.932 INFO:teuthology.orchestra.run.smithi071.stdout:3405 writesame 0x4f4eec5 thru 0x4f52f06 data_size 0x46(0x4042 bytes) 2024-09-14T05:30:05.932 INFO:teuthology.orchestra.run.smithi071.stdout:3406 punch from 0x1cd7ee1 to 0x1cdc996, (0x4ab5 bytes) 2024-09-14T05:30:05.932 INFO:teuthology.orchestra.run.smithi071.stdout:3407 read 0x8a570 thru 0x98aec (0xe57d bytes) 2024-09-14T05:30:05.948 INFO:teuthology.orchestra.run.smithi071.stdout:3408 write 0xe2ccfda thru 0xe2d422b (0x7252 bytes) 2024-09-14T05:30:05.951 INFO:teuthology.orchestra.run.smithi071.stdout:3409 write 0x207bbfb thru 0x2088959 (0xcd5f bytes) 2024-09-14T05:30:05.955 INFO:teuthology.orchestra.run.smithi071.stdout:3410 write 0xdbf59b8 thru 0xdbf9fba (0x4603 bytes) 2024-09-14T05:30:05.958 INFO:teuthology.orchestra.run.smithi071.stdout:3411 punch from 0xb58e769 to 0xb592e9a, (0x4731 bytes) 2024-09-14T05:30:05.958 INFO:teuthology.orchestra.run.smithi071.stdout:3412 read 0x8327a01 thru 0x8329702 (0x1d02 bytes) 2024-09-14T05:30:05.958 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:05.958 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:05.958 INFO:teuthology.orchestra.run.smithi071.stdout:3416 clone 28 order 24 su 131072 sc 10 2024-09-14T05:30:05.996 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:05.996 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:06.005 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:30:06.045 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:06.045+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:06.078 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T05:30:06.078 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T05:30:06.108 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:06.108 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:06.117 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:30:06.129 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:06.129+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:06.220 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:06.221 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:06.230 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:06.228+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:06.231 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:30:06.283 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone27 from 0xe2d422c (overlap 0x11d7843) to 0x0 2024-09-14T05:30:06.330 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:06.330+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:06.373 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-14T05:30:06.374 INFO:teuthology.orchestra.run.smithi071.stdout:3418 write 0xbe3b98 thru 0xbeeb20 (0xaf89 bytes) 2024-09-14T05:30:06.386 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:06.386 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:06.395 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:30:06.395 INFO:teuthology.orchestra.run.smithi071.stdout:3419 trunc from 0xe2d422c to 0x51857db 2024-09-14T05:30:06.426 INFO:teuthology.orchestra.run.smithi071.stdout:3420 write 0x38a2916 thru 0x38a4b8f (0x227a bytes) 2024-09-14T05:30:06.428 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:06.428 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:06.428 INFO:teuthology.orchestra.run.smithi071.stdout:3423 clone 29 order 18 su 16384 sc 15 2024-09-14T05:30:06.438 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:06.437+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:06.502 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:06.502 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:06.511 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:30:06.531 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:06.531+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:06.633 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:06.632+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:06.654 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:06.654 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:06.662 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:30:06.734 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:06.732+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:06.766 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:06.766 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:06.774 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:30:06.846 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:30:06.858 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:06.857+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:06.917 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:06.918 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:06.926 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:30:06.936 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:06.935+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:06.965 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:06.965 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:06.982 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:06.982 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:06.998 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:06.998 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:07.013 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:07.014 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:07.030 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:07.030 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:07.036 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:07.035+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:07.046 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:07.047 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:07.064 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:07.064 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:07.072 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:07.072 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:07.072 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:07.072 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:07.072 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:07.080 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:30:07.081 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:07.081 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:07.137 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:07.136+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:07.186 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:07.186 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:07.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:1134 3.7 e30/24 3:e1ef7f87:::rbd_data.112dd25fb4c.000000000000005f:head v 30'40, mlcod=30'36)", 2024-09-14T05:30:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.166405+0000", 2024-09-14T05:30:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.018487686, 2024-09-14T05:30:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061353650000000003, 2024-09-14T05:30:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166405+0000", 2024-09-14T05:30:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166405+0000", 2024-09-14T05:30:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166406+0000", 2024-09-14T05:30:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.85e-07 2024-09-14T05:30:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166413+0000", 2024-09-14T05:30:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1029999999999996e-06 2024-09-14T05:30:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166425+0000", 2024-09-14T05:30:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.258e-05 2024-09-14T05:30:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166433+0000", 2024-09-14T05:30:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2910000000000002e-06 2024-09-14T05:30:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166492+0000", 2024-09-14T05:30:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9107999999999997e-05 2024-09-14T05:30:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.166513+0000", 2024-09-14T05:30:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.128e-05 2024-09-14T05:30:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.172500+0000", 2024-09-14T05:30:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059870519999999997 2024-09-14T05:30:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.172540+0000", 2024-09-14T05:30:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0166000000000001e-05 2024-09-14T05:30:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1205 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.987002818000001, 2024-09-14T05:30:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0086364279999999998, 2024-09-14T05:30:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1205 2024-09-14T05:30:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197891+0000", 2024-09-14T05:30:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.364e-06 2024-09-14T05:30:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197896+0000", 2024-09-14T05:30:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3000000000000001e-06 2024-09-14T05:30:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197910+0000", 2024-09-14T05:30:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3339000000000001e-05 2024-09-14T05:30:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197919+0000", 2024-09-14T05:30:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8869999999999998e-06 2024-09-14T05:30:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197974+0000", 2024-09-14T05:30:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5720000000000002e-05 2024-09-14T05:30:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.198094+0000", 2024-09-14T05:30:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119739 2024-09-14T05:30:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206526+0000", 2024-09-14T05:30:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084320790000000003 2024-09-14T05:30:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1220 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.978246454000001, 2024-09-14T05:30:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012822337, 2024-09-14T05:30:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1220 2024-09-14T05:30:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206647+0000", 2024-09-14T05:30:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4600000000000003e-07 2024-09-14T05:30:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206652+0000", 2024-09-14T05:30:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6619999999999996e-06 2024-09-14T05:30:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206662+0000", 2024-09-14T05:30:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.995e-06 2024-09-14T05:30:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206667+0000", 2024-09-14T05:30:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-14T05:30:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206706+0000", 2024-09-14T05:30:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9523000000000001e-05 2024-09-14T05:30:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206755+0000", 2024-09-14T05:30:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000002e-05 2024-09-14T05:30:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206877+0000", 2024-09-14T05:30:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121938 2024-09-14T05:30:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.207310+0000", 2024-09-14T05:30:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000432452 2024-09-14T05:30:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219364+0000", 2024-09-14T05:30:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012054481000000001 2024-09-14T05:30:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219425+0000", 2024-09-14T05:30:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0618000000000001e-05 2024-09-14T05:30:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219468+0000", 2024-09-14T05:30:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3739000000000003e-05 2024-09-14T05:30:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1243 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.965281226999998, 2024-09-14T05:30:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078660580000000004, 2024-09-14T05:30:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1243 2024-09-14T05:30:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219612+0000", 2024-09-14T05:30:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.09e-07 2024-09-14T05:30:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219616+0000", 2024-09-14T05:30:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.216e-06 2024-09-14T05:30:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219630+0000", 2024-09-14T05:30:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3064999999999999e-05 2024-09-14T05:30:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219637+0000", 2024-09-14T05:30:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7409999999999992e-06 2024-09-14T05:30:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219697+0000", 2024-09-14T05:30:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.02e-05 2024-09-14T05:30:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219833+0000", 2024-09-14T05:30:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013595600000000001 2024-09-14T05:30:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227477+0000", 2024-09-14T05:30:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076439710000000003 2024-09-14T05:30:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1264 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.957270434000002, 2024-09-14T05:30:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080867990000000004, 2024-09-14T05:30:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1264 2024-09-14T05:30:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227623+0000", 2024-09-14T05:30:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:30:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227627+0000", 2024-09-14T05:30:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.194e-06 2024-09-14T05:30:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227639+0000", 2024-09-14T05:30:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1296e-05 2024-09-14T05:30:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227648+0000", 2024-09-14T05:30:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2280000000000004e-06 2024-09-14T05:30:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227689+0000", 2024-09-14T05:30:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1329999999999999e-05 2024-09-14T05:30:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227741+0000", 2024-09-14T05:30:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1641000000000002e-05 2024-09-14T05:30:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:30:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227914+0000", 2024-09-14T05:30:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000173215 2024-09-14T05:30:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228474+0000", 2024-09-14T05:30:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055980199999999998 2024-09-14T05:30:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235608+0000", 2024-09-14T05:30:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0071343830000000002 2024-09-14T05:30:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235669+0000", 2024-09-14T05:30:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1068999999999994e-05 2024-09-14T05:30:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235709+0000", 2024-09-14T05:30:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9671e-05 2024-09-14T05:30:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1520 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:30:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.862262815000001, 2024-09-14T05:30:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061963629999999999, 2024-09-14T05:30:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1520 2024-09-14T05:30:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:30:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:30:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322631+0000", 2024-09-14T05:30:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.54e-07 2024-09-14T05:30:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322635+0000", 2024-09-14T05:30:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0450000000000002e-06 2024-09-14T05:30:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322646+0000", 2024-09-14T05:30:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1919999999999999e-05 2024-09-14T05:30:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322654+0000", 2024-09-14T05:30:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3139999999999998e-06 2024-09-14T05:30:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322704+0000", 2024-09-14T05:30:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0420000000000002e-05 2024-09-14T05:30:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322794+0000", 2024-09-14T05:30:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0037000000000005e-05 2024-09-14T05:30:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328826+0000", 2024-09-14T05:30:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060318730000000001 2024-09-14T05:30:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1526 3.4 3:2695cbe4:::rbd_data.112dd25fb4c.0000000000000119:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.858729871000001, 2024-09-14T05:30:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0079537359999999994, 2024-09-14T05:30:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1526 2024-09-14T05:30:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3600000000000003e-07 2024-09-14T05:30:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326168+0000", 2024-09-14T05:30:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3460000000000004e-06 2024-09-14T05:30:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326178+0000", 2024-09-14T05:30:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0291e-05 2024-09-14T05:30:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326183+0000", 2024-09-14T05:30:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-14T05:30:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326230+0000", 2024-09-14T05:30:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6650999999999998e-05 2024-09-14T05:30:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326295+0000", 2024-09-14T05:30:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5158000000000002e-05 2024-09-14T05:30:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326408+0000", 2024-09-14T05:30:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113039 2024-09-14T05:30:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.331648+0000", 2024-09-14T05:30:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052404929999999997 2024-09-14T05:30:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334011+0000", 2024-09-14T05:30:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002362669 2024-09-14T05:30:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334064+0000", 2024-09-14T05:30:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3198999999999997e-05 2024-09-14T05:30:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334116+0000", 2024-09-14T05:30:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2133999999999999e-05 2024-09-14T05:30:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1529 3.1 3:809746f7:::rbd_data.112dd25fb4c.0000000000000118:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:30:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.856965320999997, 2024-09-14T05:30:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061981060000000001, 2024-09-14T05:30:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1529 2024-09-14T05:30:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:30:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:30:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327928+0000", 2024-09-14T05:30:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0100000000000004e-07 2024-09-14T05:30:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327933+0000", 2024-09-14T05:30:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4689999999999999e-06 2024-09-14T05:30:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327944+0000", 2024-09-14T05:30:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1623999999999999e-05 2024-09-14T05:30:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327951+0000", 2024-09-14T05:30:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9759999999999998e-06 2024-09-14T05:30:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328003+0000", 2024-09-14T05:30:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1521999999999999e-05 2024-09-14T05:30:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328087+0000", 2024-09-14T05:30:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3988999999999998e-05 2024-09-14T05:30:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328304+0000", 2024-09-14T05:30:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000217254 2024-09-14T05:30:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.333363+0000", 2024-09-14T05:30:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050586499999999996 2024-09-14T05:30:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334026+0000", 2024-09-14T05:30:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066303499999999997 2024-09-14T05:30:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334080+0000", 2024-09-14T05:30:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4261000000000001e-05 2024-09-14T05:30:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334125+0000", 2024-09-14T05:30:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5525000000000002e-05 2024-09-14T05:30:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1531 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.855911704999997, 2024-09-14T05:30:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082114309999999999, 2024-09-14T05:30:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1531 2024-09-14T05:30:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328982+0000", 2024-09-14T05:30:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:30:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328985+0000", 2024-09-14T05:30:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7900000000000001e-06 2024-09-14T05:30:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328996+0000", 2024-09-14T05:30:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.028e-05 2024-09-14T05:30:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329001+0000", 2024-09-14T05:30:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0610000000000003e-06 2024-09-14T05:30:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329035+0000", 2024-09-14T05:30:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4366999999999998e-05 2024-09-14T05:30:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329082+0000", 2024-09-14T05:30:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6646e-05 2024-09-14T05:30:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329195+0000", 2024-09-14T05:30:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113279 2024-09-14T05:30:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334293+0000", 2024-09-14T05:30:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050980399999999999 2024-09-14T05:30:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337101+0000", 2024-09-14T05:30:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0028076170000000001 2024-09-14T05:30:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337154+0000", 2024-09-14T05:30:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3641999999999999e-05 2024-09-14T05:30:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337192+0000", 2024-09-14T05:30:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7932000000000002e-05 2024-09-14T05:30:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:2131 3.b e30/24 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head v 30'36, mlcod=30'34)", 2024-09-14T05:30:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.392564350000001, 2024-09-14T05:30:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031871709999999998, 2024-09-14T05:30:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792331+0000", 2024-09-14T05:30:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6900000000000001e-06 2024-09-14T05:30:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797319+0000", 2024-09-14T05:30:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049876349999999998 2024-09-14T05:30:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797349+0000", 2024-09-14T05:30:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0046e-05 2024-09-14T05:30:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797359+0000", 2024-09-14T05:30:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0037999999999999e-05 2024-09-14T05:30:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797445+0000", 2024-09-14T05:30:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6132999999999997e-05 2024-09-14T05:30:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797472+0000", 2024-09-14T05:30:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7478999999999999e-05 2024-09-14T05:30:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824161+0000", 2024-09-14T05:30:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026688714999999998 2024-09-14T05:30:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824200+0000", 2024-09-14T05:30:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8974000000000003e-05 2024-09-14T05:30:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4259 3.4 3:25f21dc9:::rbd_data.112d3d611e54.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 294912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 35.793391735999997, 2024-09-14T05:30:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010043553, 2024-09-14T05:30:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4259 2024-09-14T05:30:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391503+0000", 2024-09-14T05:30:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9099999999999999e-06 2024-09-14T05:30:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391821+0000", 2024-09-14T05:30:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031871800000000001 2024-09-14T05:30:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391844+0000", 2024-09-14T05:30:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2045e-05 2024-09-14T05:30:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391853+0000", 2024-09-14T05:30:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7249999999999992e-06 2024-09-14T05:30:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391917+0000", 2024-09-14T05:30:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3458000000000001e-05 2024-09-14T05:30:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392037+0000", 2024-09-14T05:30:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120449 2024-09-14T05:30:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392305+0000", 2024-09-14T05:30:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026739299999999999 2024-09-14T05:30:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.393609+0000", 2024-09-14T05:30:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013042450000000001 2024-09-14T05:30:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401430+0000", 2024-09-14T05:30:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078211670000000004 2024-09-14T05:30:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401499+0000", 2024-09-14T05:30:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9374000000000004e-05 2024-09-14T05:30:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401544+0000", 2024-09-14T05:30:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5068999999999998e-05 2024-09-14T05:30:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4452 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:30:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 35.302579127999998, 2024-09-14T05:30:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066061380000000001, 2024-09-14T05:30:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4452 2024-09-14T05:30:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:30:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:30:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882314+0000", 2024-09-14T05:30:07.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5899999999999998e-07 2024-09-14T05:30:07.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882315+0000", 2024-09-14T05:30:07.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6169999999999999e-06 2024-09-14T05:30:07.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882319+0000", 2024-09-14T05:30:07.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3629999999999998e-06 2024-09-14T05:30:07.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882324+0000", 2024-09-14T05:30:07.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1370000000000001e-06 2024-09-14T05:30:07.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883369+0000", 2024-09-14T05:30:07.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010448320000000001 2024-09-14T05:30:07.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883421+0000", 2024-09-14T05:30:07.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2401000000000002e-05 2024-09-14T05:30:07.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:07.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883638+0000", 2024-09-14T05:30:07.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021725600000000001 2024-09-14T05:30:07.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:07.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.884464+0000", 2024-09-14T05:30:07.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00082569800000000004 2024-09-14T05:30:07.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:07.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888860+0000", 2024-09-14T05:30:07.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0043963370000000002 2024-09-14T05:30:07.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:07.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888896+0000", 2024-09-14T05:30:07.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6022999999999997e-05 2024-09-14T05:30:07.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888920+0000", 2024-09-14T05:30:07.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3215000000000001e-05 2024-09-14T05:30:07.256 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.256 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.256 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.256 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4467 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.256 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:07.256 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 35.284210678000001, 2024-09-14T05:30:07.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0065700189999999999, 2024-09-14T05:30:07.256 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.256 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.256 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:07.256 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.257 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.257 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.257 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4467 2024-09-14T05:30:07.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.257 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:07.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:07.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.258 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.258 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:07.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-14T05:30:07.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.258 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.258 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900684+0000", 2024-09-14T05:30:07.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.629e-06 2024-09-14T05:30:07.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.258 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900688+0000", 2024-09-14T05:30:07.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8469999999999998e-06 2024-09-14T05:30:07.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900691+0000", 2024-09-14T05:30:07.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-14T05:30:07.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.901775+0000", 2024-09-14T05:30:07.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001084177 2024-09-14T05:30:07.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.901862+0000", 2024-09-14T05:30:07.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7167000000000001e-05 2024-09-14T05:30:07.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:07.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.902223+0000", 2024-09-14T05:30:07.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036068800000000001 2024-09-14T05:30:07.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:07.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.902843+0000", 2024-09-14T05:30:07.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000620489 2024-09-14T05:30:07.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:07.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907167+0000", 2024-09-14T05:30:07.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0043242649999999999 2024-09-14T05:30:07.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:07.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907217+0000", 2024-09-14T05:30:07.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9648000000000003e-05 2024-09-14T05:30:07.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907252+0000", 2024-09-14T05:30:07.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4780000000000002e-05 2024-09-14T05:30:07.262 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.262 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.262 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.262 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4554 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.262 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:07.262 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 35.200490924999997, 2024-09-14T05:30:07.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069998999999999999, 2024-09-14T05:30:07.262 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.262 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.262 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:07.263 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.263 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.263 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.263 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4554 2024-09-14T05:30:07.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.263 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:07.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:07.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:07.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9299999999999999e-07 2024-09-14T05:30:07.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984404+0000", 2024-09-14T05:30:07.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.855e-06 2024-09-14T05:30:07.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984407+0000", 2024-09-14T05:30:07.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-14T05:30:07.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984410+0000", 2024-09-14T05:30:07.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8159999999999998e-06 2024-09-14T05:30:07.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985511+0000", 2024-09-14T05:30:07.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00110053 2024-09-14T05:30:07.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985561+0000", 2024-09-14T05:30:07.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0389000000000002e-05 2024-09-14T05:30:07.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:07.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985772+0000", 2024-09-14T05:30:07.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000210571 2024-09-14T05:30:07.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:07.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.987075+0000", 2024-09-14T05:30:07.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001303128 2024-09-14T05:30:07.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:07.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991303+0000", 2024-09-14T05:30:07.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042283310000000001 2024-09-14T05:30:07.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:07.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991359+0000", 2024-09-14T05:30:07.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6335999999999997e-05 2024-09-14T05:30:07.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991402+0000", 2024-09-14T05:30:07.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2074999999999999e-05 2024-09-14T05:30:07.268 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.268 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.268 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.268 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4711 3.4 3:226df663:::rbd_data.112d3d611e54.000000000000002d:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:07.268 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:07.268 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 35.017819768999999, 2024-09-14T05:30:07.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011304193000000001, 2024-09-14T05:30:07.268 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.268 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.269 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:07.269 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.269 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.269 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.269 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4711 2024-09-14T05:30:07.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.269 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:07.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:07.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167074+0000", 2024-09-14T05:30:07.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0999999999999997e-07 2024-09-14T05:30:07.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167078+0000", 2024-09-14T05:30:07.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.878e-06 2024-09-14T05:30:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167087+0000", 2024-09-14T05:30:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6940000000000007e-06 2024-09-14T05:30:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167093+0000", 2024-09-14T05:30:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7740000000000001e-06 2024-09-14T05:30:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167128+0000", 2024-09-14T05:30:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5231999999999997e-05 2024-09-14T05:30:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167207+0000", 2024-09-14T05:30:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9036999999999996e-05 2024-09-14T05:30:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.169140+0000", 2024-09-14T05:30:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001932897 2024-09-14T05:30:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178258+0000", 2024-09-14T05:30:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091180819999999996 2024-09-14T05:30:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178314+0000", 2024-09-14T05:30:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5571e-05 2024-09-14T05:30:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178348+0000", 2024-09-14T05:30:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3747000000000001e-05 2024-09-14T05:30:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178377+0000", 2024-09-14T05:30:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9471e-05 2024-09-14T05:30:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4721 3.1 3:86302a6e:::rbd_data.112d3d611e54.0000000000000011:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:30:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 34.985834283000003, 2024-09-14T05:30:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066831750000000004, 2024-09-14T05:30:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4721 2024-09-14T05:30:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:30:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:30:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199059+0000", 2024-09-14T05:30:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.06e-07 2024-09-14T05:30:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199062+0000", 2024-09-14T05:30:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2279999999999998e-06 2024-09-14T05:30:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199072+0000", 2024-09-14T05:30:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5999999999999996e-06 2024-09-14T05:30:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199078+0000", 2024-09-14T05:30:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9109999999999998e-06 2024-09-14T05:30:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199123+0000", 2024-09-14T05:30:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5278000000000003e-05 2024-09-14T05:30:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199202+0000", 2024-09-14T05:30:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8521000000000003e-05 2024-09-14T05:30:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199271+0000", 2024-09-14T05:30:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9857999999999994e-05 2024-09-14T05:30:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199941+0000", 2024-09-14T05:30:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066984599999999998 2024-09-14T05:30:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205653+0000", 2024-09-14T05:30:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057117319999999997 2024-09-14T05:30:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205701+0000", 2024-09-14T05:30:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8254000000000003e-05 2024-09-14T05:30:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205741+0000", 2024-09-14T05:30:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0241000000000003e-05 2024-09-14T05:30:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4819 3.12 3:4e8efced:::rbd_data.112d3d611e54.000000000000000e:head [stat,truncate 1114112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:30:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 34.900520124000003, 2024-09-14T05:30:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062422770000000001, 2024-09-14T05:30:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4819 2024-09-14T05:30:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:30:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:30:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284373+0000", 2024-09-14T05:30:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3399999999999998e-07 2024-09-14T05:30:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284377+0000", 2024-09-14T05:30:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8539999999999999e-06 2024-09-14T05:30:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284388+0000", 2024-09-14T05:30:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1328999999999999e-05 2024-09-14T05:30:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284395+0000", 2024-09-14T05:30:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9759999999999998e-06 2024-09-14T05:30:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284723+0000", 2024-09-14T05:30:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032775799999999999 2024-09-14T05:30:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284801+0000", 2024-09-14T05:30:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7562e-05 2024-09-14T05:30:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:30:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284933+0000", 2024-09-14T05:30:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000131989 2024-09-14T05:30:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.287450+0000", 2024-09-14T05:30:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025177699999999999 2024-09-14T05:30:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290531+0000", 2024-09-14T05:30:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0030809959999999999 2024-09-14T05:30:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290588+0000", 2024-09-14T05:30:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6177000000000001e-05 2024-09-14T05:30:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290615+0000", 2024-09-14T05:30:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7132e-05 2024-09-14T05:30:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:7067 3.5 e35/24 3:a65d0cff:::rbd_data.112df7f2fe0a.0000000000000028:head v 35'89, mlcod=34'86)", 2024-09-14T05:30:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.042126852999999, 2024-09-14T05:30:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0064513080000000002, 2024-09-14T05:30:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142771+0000", 2024-09-14T05:30:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8949999999999997e-06 2024-09-14T05:30:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142779+0000", 2024-09-14T05:30:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8699999999999992e-06 2024-09-14T05:30:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142785+0000", 2024-09-14T05:30:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4899999999999997e-06 2024-09-14T05:30:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142843+0000", 2024-09-14T05:30:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7377000000000003e-05 2024-09-14T05:30:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142871+0000", 2024-09-14T05:30:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8305000000000001e-05 2024-09-14T05:30:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149177+0000", 2024-09-14T05:30:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063058639999999996 2024-09-14T05:30:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149217+0000", 2024-09-14T05:30:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0080999999999998e-05 2024-09-14T05:30:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:72582 3.b e47/24 3:d5072567:::rbd_data.112d4a90444c.0000000000000119:head v 47'386, mlcod=46'383)", 2024-09-14T05:30:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 17.659865916000001, 2024-09-14T05:30:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0067720940000000002, 2024-09-14T05:30:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1100000000000002e-07 2024-09-14T05:30:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525031+0000", 2024-09-14T05:30:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0740000000000003e-06 2024-09-14T05:30:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525040+0000", 2024-09-14T05:30:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5329999999999994e-06 2024-09-14T05:30:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525045+0000", 2024-09-14T05:30:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:30:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525090+0000", 2024-09-14T05:30:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4743000000000002e-05 2024-09-14T05:30:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525102+0000", 2024-09-14T05:30:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2067e-05 2024-09-14T05:30:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531757+0000", 2024-09-14T05:30:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066550380000000003 2024-09-14T05:30:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531799+0000", 2024-09-14T05:30:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1640000000000001e-05 2024-09-14T05:30:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:79671 3.15 e47/24 3:ab572765:::rbd_data.112d956b542f.0000000000000013:head v 47'120, mlcod=47'118)", 2024-09-14T05:30:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:30:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 17.033885697999999, 2024-09-14T05:30:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066700609999999997, 2024-09-14T05:30:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:30:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:30:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151008+0000", 2024-09-14T05:30:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:30:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151264+0000", 2024-09-14T05:30:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025656099999999999 2024-09-14T05:30:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151278+0000", 2024-09-14T05:30:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3288999999999999e-05 2024-09-14T05:30:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151287+0000", 2024-09-14T05:30:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7909999999999993e-06 2024-09-14T05:30:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151361+0000", 2024-09-14T05:30:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3979000000000004e-05 2024-09-14T05:30:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151383+0000", 2024-09-14T05:30:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2039999999999999e-05 2024-09-14T05:30:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.157542+0000", 2024-09-14T05:30:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006158915 2024-09-14T05:30:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.157677+0000", 2024-09-14T05:30:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013544300000000001 2024-09-14T05:30:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.300 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:07.301 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T05:30:07.302 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T05:30:07.303 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:30:07.303 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:07.299+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:07.304 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T05:30:07.304 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone28 from 0x51857db (overlap 0x51857db) to 0x0 2024-09-14T05:30:07.355 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-14T05:30:07.362 INFO:teuthology.orchestra.run.smithi071.stdout:3424 trunc from 0x51857db to 0xaf190d 2024-09-14T05:30:07.402 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:07.401+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:07.415 INFO:teuthology.orchestra.run.smithi071.stdout:3425 write 0x30e4a40 thru 0x30e7e65 (0x3426 bytes) 2024-09-14T05:30:07.442 INFO:teuthology.orchestra.run.smithi071.stdout:3426 writesame 0xeb9a2e8 thru 0xeba4030 data_size 0x1(0x9d49 bytes) 2024-09-14T05:30:07.442 INFO:teuthology.orchestra.run.smithi071.stdout:3427 writesame 0x32db83b thru 0x32e3944 data_size 0x53(0x810a bytes) 2024-09-14T05:30:07.451 INFO:teuthology.orchestra.run.smithi071.stdout:3428 writesame 0x4129bb6 thru 0x412cd65 data_size 0x8(0x31b0 bytes) 2024-09-14T05:30:07.451 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:07.451 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:07.451 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:07.451 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:07.456 INFO:teuthology.orchestra.run.smithi071.stdout:3429 write 0x1610a1d thru 0x161f6b7 (0xec9b bytes) 2024-09-14T05:30:07.460 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:30:07.465 INFO:teuthology.orchestra.run.smithi071.stdout:3430 punch from 0x3125f98 to 0x312d6b1, (0x7719 bytes) 2024-09-14T05:30:07.466 INFO:teuthology.orchestra.run.smithi071.stdout:3431 read 0x9cd0c87 thru 0x9cd8859 (0x7bd3 bytes) 2024-09-14T05:30:07.466 INFO:teuthology.orchestra.run.smithi071.stdout:3432 punch from 0x7cd4fcf to 0x7cd9fa5, (0x4fd6 bytes) 2024-09-14T05:30:07.466 INFO:teuthology.orchestra.run.smithi071.stdout:3434 write 0x5e3b441 thru 0x5e3dc28 (0x27e8 bytes) 2024-09-14T05:30:07.471 INFO:teuthology.orchestra.run.smithi071.stdout:3435 read 0x4cd20f3 thru 0x4cd7ea8 (0x5db6 bytes) 2024-09-14T05:30:07.471 INFO:teuthology.orchestra.run.smithi071.stdout:3436 writesame 0x363acd thru 0x36c585 data_size 0x1(0x8ab9 bytes) 2024-09-14T05:30:07.492 INFO:teuthology.orchestra.run.smithi071.stdout:3439 write 0xa81c091 thru 0xa828013 (0xbf83 bytes) 2024-09-14T05:30:07.500 INFO:teuthology.orchestra.run.smithi071.stdout:3443 read 0xd18d41d thru 0xd1967d1 (0x93b5 bytes) 2024-09-14T05:30:07.500 INFO:teuthology.orchestra.run.smithi071.stdout:3446 write 0x747f7e thru 0x749115 (0x1198 bytes) 2024-09-14T05:30:07.503 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:07.502+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:07.507 INFO:teuthology.orchestra.run.smithi071.stdout:3448 read 0x789fee4 thru 0x78a87b6 (0x88d3 bytes) 2024-09-14T05:30:07.514 INFO:teuthology.orchestra.run.smithi071.stdout:3449 trunc from 0xeba4031 to 0x9d6f8b9 2024-09-14T05:30:07.524 INFO:teuthology.orchestra.run.smithi071.stdout:3450 write 0x34ae57c thru 0x34bd763 (0xf1e8 bytes) 2024-09-14T05:30:07.532 INFO:teuthology.orchestra.run.smithi071.stdout:truncating to largest ever: 0xee3bfbe 2024-09-14T05:30:07.538 INFO:teuthology.orchestra.run.smithi071.stdout:3451 trunc from 0x9d6f8b9 to 0xee3bfbe 2024-09-14T05:30:07.543 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:07.543 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:07.544 INFO:teuthology.orchestra.run.smithi071.stdout:3454 read 0x383272f thru 0x3839918 (0x71ea bytes) 2024-09-14T05:30:07.544 INFO:teuthology.orchestra.run.smithi071.stdout:3455 writesame 0x656a3a8 thru 0x657816c data_size 0x1(0xddc5 bytes) 2024-09-14T05:30:07.561 INFO:teuthology.orchestra.run.smithi071.stdout:3457 writesame 0x92829d thru 0x92b8fb data_size 0x1(0x365f bytes) 2024-09-14T05:30:07.568 INFO:teuthology.orchestra.run.smithi071.stdout:3461 read 0xb39643b thru 0xb39c641 (0x6207 bytes) 2024-09-14T05:30:07.568 INFO:teuthology.orchestra.run.smithi071.stdout:3465 read 0xc83aa10 thru 0xc83f80d (0x4dfe bytes) 2024-09-14T05:30:07.571 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:07.571 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:07.571 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:07.571 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:07.571 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:07.574 INFO:teuthology.orchestra.run.smithi071.stdout:3466 trunc from 0xee3bfbe to 0x91a4163 2024-09-14T05:30:07.580 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:30:07.580 INFO:teuthology.orchestra.run.smithi071.stdout:3467 punch from 0x4a1ec50 to 0x4a26a06, (0x7db6 bytes) 2024-09-14T05:30:07.590 INFO:teuthology.orchestra.run.smithi071.stdout:3468 writesame 0xbf3307b thru 0xbf3ac37 data_size 0x1(0x7bbd bytes) 2024-09-14T05:30:07.591 INFO:teuthology.orchestra.run.smithi071.stdout:3469 read 0x5afc4d6 thru 0x5b02804 (0x632f bytes) 2024-09-14T05:30:07.591 INFO:teuthology.orchestra.run.smithi071.stdout:3470 write 0x946859a thru 0x9473dba (0xb821 bytes) 2024-09-14T05:30:07.599 INFO:teuthology.orchestra.run.smithi071.stdout:3472 read 0xb0ac46a thru 0xb0b44c9 (0x8060 bytes) 2024-09-14T05:30:07.605 INFO:teuthology.orchestra.run.smithi071.stdout:3473 trunc from 0xbf3ac38 to 0x6e45cc6 2024-09-14T05:30:07.624 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:07.622+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:07.630 INFO:teuthology.orchestra.run.smithi071.stdout:3474 write 0xe0e5af3 thru 0xe0e6c6d (0x117b bytes) 2024-09-14T05:30:07.645 INFO:teuthology.orchestra.run.smithi071.stdout:3475 trunc from 0xe0e6c6e to 0x47355dc 2024-09-14T05:30:07.657 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:07.664 INFO:teuthology.orchestra.run.smithi071.stdout:3477 trunc from 0x47355dc to 0xa041282 2024-09-14T05:30:07.669 INFO:teuthology.orchestra.run.smithi071.stdout:3479 read 0x8cc1f5d thru 0x8cc5fee (0x4092 bytes) 2024-09-14T05:30:07.671 INFO:teuthology.orchestra.run.smithi071.stdout:3480 trunc from 0xa041282 to 0x82decf2 2024-09-14T05:30:07.687 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2131 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.910820454000003, 2024-09-14T05:30:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049526593000000001, 2024-09-14T05:30:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2131 2024-09-14T05:30:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774867+0000", 2024-09-14T05:30:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4430000000000002e-06 2024-09-14T05:30:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790357+0000", 2024-09-14T05:30:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015489466 2024-09-14T05:30:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790383+0000", 2024-09-14T05:30:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6247000000000001e-05 2024-09-14T05:30:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790394+0000", 2024-09-14T05:30:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0728e-05 2024-09-14T05:30:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790471+0000", 2024-09-14T05:30:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7539000000000004e-05 2024-09-14T05:30:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790639+0000", 2024-09-14T05:30:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016787700000000001 2024-09-14T05:30:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-14T05:30:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790998+0000", 2024-09-14T05:30:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035931299999999999 2024-09-14T05:30:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.823529+0000", 2024-09-14T05:30:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032530798 2024-09-14T05:30:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824324+0000", 2024-09-14T05:30:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00079442399999999995 2024-09-14T05:30:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824362+0000", 2024-09-14T05:30:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8398000000000001e-05 2024-09-14T05:30:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824391+0000", 2024-09-14T05:30:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.936e-05 2024-09-14T05:30:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3892 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7602176~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.329242456000003, 2024-09-14T05:30:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047082951999999997, 2024-09-14T05:30:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3892 2024-09-14T05:30:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0199999999999998e-07 2024-09-14T05:30:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356454+0000", 2024-09-14T05:30:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0414999999999999e-05 2024-09-14T05:30:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356463+0000", 2024-09-14T05:30:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6739999999999992e-06 2024-09-14T05:30:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356468+0000", 2024-09-14T05:30:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5680000000000001e-06 2024-09-14T05:30:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403143+0000", 2024-09-14T05:30:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046675051000000002 2024-09-14T05:30:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403206+0000", 2024-09-14T05:30:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3127999999999996e-05 2024-09-14T05:30:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403526+0000", 2024-09-14T05:30:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031981400000000002 2024-09-14T05:30:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3893 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8093696~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.329197618999999, 2024-09-14T05:30:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047516305000000002, 2024-09-14T05:30:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3893 2024-09-14T05:30:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6899999999999997e-07 2024-09-14T05:30:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356491+0000", 2024-09-14T05:30:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8109999999999999e-06 2024-09-14T05:30:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356498+0000", 2024-09-14T05:30:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5320000000000003e-06 2024-09-14T05:30:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356502+0000", 2024-09-14T05:30:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000002e-06 2024-09-14T05:30:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403579+0000", 2024-09-14T05:30:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047076706000000003 2024-09-14T05:30:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403664+0000", 2024-09-14T05:30:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5240000000000004e-05 2024-09-14T05:30:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404004+0000", 2024-09-14T05:30:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033974500000000002 2024-09-14T05:30:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3993 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 262144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.329175747999997, 2024-09-14T05:30:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048088476999999998, 2024-09-14T05:30:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3993 2024-09-14T05:30:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.46e-07 2024-09-14T05:30:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356512+0000", 2024-09-14T05:30:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.827e-06 2024-09-14T05:30:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356518+0000", 2024-09-14T05:30:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7780000000000004e-06 2024-09-14T05:30:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356522+0000", 2024-09-14T05:30:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.865e-06 2024-09-14T05:30:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404036+0000", 2024-09-14T05:30:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047513725999999999 2024-09-14T05:30:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404135+0000", 2024-09-14T05:30:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9352999999999996e-05 2024-09-14T05:30:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404598+0000", 2024-09-14T05:30:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046258200000000001 2024-09-14T05:30:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3994 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.329133736000003, 2024-09-14T05:30:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048274625000000002, 2024-09-14T05:30:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3994 2024-09-14T05:30:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3799999999999997e-07 2024-09-14T05:30:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356554+0000", 2024-09-14T05:30:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2259999999999999e-06 2024-09-14T05:30:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356560+0000", 2024-09-14T05:30:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-14T05:30:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356566+0000", 2024-09-14T05:30:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3909999999999997e-06 2024-09-14T05:30:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404632+0000", 2024-09-14T05:30:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048066154 2024-09-14T05:30:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404719+0000", 2024-09-14T05:30:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7440000000000003e-05 2024-09-14T05:30:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404826+0000", 2024-09-14T05:30:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106813 2024-09-14T05:30:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3995 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.329108597000001, 2024-09-14T05:30:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048828209999999997, 2024-09-14T05:30:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3995 2024-09-14T05:30:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6599999999999996e-07 2024-09-14T05:30:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356579+0000", 2024-09-14T05:30:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0719999999999998e-06 2024-09-14T05:30:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356585+0000", 2024-09-14T05:30:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.959e-06 2024-09-14T05:30:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356590+0000", 2024-09-14T05:30:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.464e-06 2024-09-14T05:30:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404875+0000", 2024-09-14T05:30:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048285649999999999 2024-09-14T05:30:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404985+0000", 2024-09-14T05:30:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109417 2024-09-14T05:30:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405405+0000", 2024-09-14T05:30:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042008200000000001 2024-09-14T05:30:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3996 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.328799713999999, 2024-09-14T05:30:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049061595, 2024-09-14T05:30:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3996 2024-09-14T05:30:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.61e-07 2024-09-14T05:30:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356892+0000", 2024-09-14T05:30:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9050000000000002e-06 2024-09-14T05:30:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356901+0000", 2024-09-14T05:30:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4149999999999999e-06 2024-09-14T05:30:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356907+0000", 2024-09-14T05:30:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0140000000000004e-06 2024-09-14T05:30:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405448+0000", 2024-09-14T05:30:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048541647 2024-09-14T05:30:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405542+0000", 2024-09-14T05:30:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3639000000000003e-05 2024-09-14T05:30:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405947+0000", 2024-09-14T05:30:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040521399999999999 2024-09-14T05:30:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3997 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2228224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.328585732000001, 2024-09-14T05:30:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049308774999999999, 2024-09-14T05:30:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3997 2024-09-14T05:30:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6499999999999999e-07 2024-09-14T05:30:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357104+0000", 2024-09-14T05:30:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4989999999999999e-06 2024-09-14T05:30:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357112+0000", 2024-09-14T05:30:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4279999999999996e-06 2024-09-14T05:30:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357117+0000", 2024-09-14T05:30:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6190000000000002e-06 2024-09-14T05:30:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405989+0000", 2024-09-14T05:30:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048872080999999998 2024-09-14T05:30:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406074+0000", 2024-09-14T05:30:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4676e-05 2024-09-14T05:30:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406408+0000", 2024-09-14T05:30:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033480700000000001 2024-09-14T05:30:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3998 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.328264302999997, 2024-09-14T05:30:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049462314, 2024-09-14T05:30:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3998 2024-09-14T05:30:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357422+0000", 2024-09-14T05:30:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3099999999999996e-07 2024-09-14T05:30:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357429+0000", 2024-09-14T05:30:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2880000000000004e-06 2024-09-14T05:30:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357439+0000", 2024-09-14T05:30:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.302e-06 2024-09-14T05:30:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357445+0000", 2024-09-14T05:30:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4269999999999996e-06 2024-09-14T05:30:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406448+0000", 2024-09-14T05:30:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049003096000000003 2024-09-14T05:30:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406537+0000", 2024-09-14T05:30:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9135999999999995e-05 2024-09-14T05:30:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406883+0000", 2024-09-14T05:30:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000346134 2024-09-14T05:30:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3999 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.328193945999999, 2024-09-14T05:30:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049797556, 2024-09-14T05:30:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3999 2024-09-14T05:30:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357492+0000", 2024-09-14T05:30:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8299999999999996e-07 2024-09-14T05:30:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357495+0000", 2024-09-14T05:30:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0450000000000001e-06 2024-09-14T05:30:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357503+0000", 2024-09-14T05:30:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9689999999999994e-06 2024-09-14T05:30:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357510+0000", 2024-09-14T05:30:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-14T05:30:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406914+0000", 2024-09-14T05:30:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049404293000000002 2024-09-14T05:30:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406965+0000", 2024-09-14T05:30:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0470999999999998e-05 2024-09-14T05:30:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407289+0000", 2024-09-14T05:30:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032434899999999998 2024-09-14T05:30:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4000 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3702784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.327847601000002, 2024-09-14T05:30:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049936728999999999, 2024-09-14T05:30:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4000 2024-09-14T05:30:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357839+0000", 2024-09-14T05:30:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-14T05:30:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357842+0000", 2024-09-14T05:30:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4850000000000001e-06 2024-09-14T05:30:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357851+0000", 2024-09-14T05:30:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1819999999999995e-06 2024-09-14T05:30:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357859+0000", 2024-09-14T05:30:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8129999999999996e-06 2024-09-14T05:30:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407337+0000", 2024-09-14T05:30:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049478084999999998 2024-09-14T05:30:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407442+0000", 2024-09-14T05:30:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000105302 2024-09-14T05:30:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407774+0000", 2024-09-14T05:30:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033200100000000001 2024-09-14T05:30:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4001 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.327752246000003, 2024-09-14T05:30:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050304205999999997, 2024-09-14T05:30:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4001 2024-09-14T05:30:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357934+0000", 2024-09-14T05:30:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000004e-07 2024-09-14T05:30:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357937+0000", 2024-09-14T05:30:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0869999999999998e-06 2024-09-14T05:30:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357946+0000", 2024-09-14T05:30:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1439999999999992e-06 2024-09-14T05:30:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357952+0000", 2024-09-14T05:30:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.609e-06 2024-09-14T05:30:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407820+0000", 2024-09-14T05:30:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049868580000000003 2024-09-14T05:30:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407909+0000", 2024-09-14T05:30:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8887000000000005e-05 2024-09-14T05:30:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408237+0000", 2024-09-14T05:30:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032824599999999997 2024-09-14T05:30:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4002 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.327535650999998, 2024-09-14T05:30:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050539529999999999, 2024-09-14T05:30:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4002 2024-09-14T05:30:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5099999999999999e-07 2024-09-14T05:30:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358154+0000", 2024-09-14T05:30:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6799999999999999e-06 2024-09-14T05:30:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358165+0000", 2024-09-14T05:30:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0899e-05 2024-09-14T05:30:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358172+0000", 2024-09-14T05:30:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8689999999999998e-06 2024-09-14T05:30:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408281+0000", 2024-09-14T05:30:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050108699 2024-09-14T05:30:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408369+0000", 2024-09-14T05:30:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8542999999999996e-05 2024-09-14T05:30:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408689+0000", 2024-09-14T05:30:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032008900000000002 2024-09-14T05:30:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4003 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5177344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.327495816000003, 2024-09-14T05:30:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050868759, 2024-09-14T05:30:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4003 2024-09-14T05:30:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-07 2024-09-14T05:30:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358193+0000", 2024-09-14T05:30:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.588e-06 2024-09-14T05:30:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358204+0000", 2024-09-14T05:30:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1525999999999999e-05 2024-09-14T05:30:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358213+0000", 2024-09-14T05:30:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5010000000000004e-06 2024-09-14T05:30:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408719+0000", 2024-09-14T05:30:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050506708999999997 2024-09-14T05:30:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408766+0000", 2024-09-14T05:30:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6579999999999998e-05 2024-09-14T05:30:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409058+0000", 2024-09-14T05:30:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000292365 2024-09-14T05:30:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4004 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5668864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.327072973, 2024-09-14T05:30:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050839848, 2024-09-14T05:30:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4004 2024-09-14T05:30:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358613+0000", 2024-09-14T05:30:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:30:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358618+0000", 2024-09-14T05:30:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6009999999999997e-06 2024-09-14T05:30:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358630+0000", 2024-09-14T05:30:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2122e-05 2024-09-14T05:30:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358638+0000", 2024-09-14T05:30:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3969999999999994e-06 2024-09-14T05:30:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409088+0000", 2024-09-14T05:30:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050449565000000002 2024-09-14T05:30:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409153+0000", 2024-09-14T05:30:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4728999999999996e-05 2024-09-14T05:30:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409452+0000", 2024-09-14T05:30:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029958900000000001 2024-09-14T05:30:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4005 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6160384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.326856038999999, 2024-09-14T05:30:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051054752000000002, 2024-09-14T05:30:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4005 2024-09-14T05:30:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358830+0000", 2024-09-14T05:30:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-14T05:30:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358833+0000", 2024-09-14T05:30:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9529999999999999e-06 2024-09-14T05:30:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358843+0000", 2024-09-14T05:30:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8069999999999994e-06 2024-09-14T05:30:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358849+0000", 2024-09-14T05:30:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1040000000000003e-06 2024-09-14T05:30:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409487+0000", 2024-09-14T05:30:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050638626999999999 2024-09-14T05:30:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409535+0000", 2024-09-14T05:30:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7525000000000003e-05 2024-09-14T05:30:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409884+0000", 2024-09-14T05:30:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034914800000000003 2024-09-14T05:30:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4006 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.326818926999998, 2024-09-14T05:30:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051425323000000002, 2024-09-14T05:30:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4006 2024-09-14T05:30:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358867+0000", 2024-09-14T05:30:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2900000000000004e-07 2024-09-14T05:30:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358869+0000", 2024-09-14T05:30:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4210000000000002e-06 2024-09-14T05:30:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358876+0000", 2024-09-14T05:30:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4980000000000003e-06 2024-09-14T05:30:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358881+0000", 2024-09-14T05:30:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3090000000000003e-06 2024-09-14T05:30:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409915+0000", 2024-09-14T05:30:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051033348999999999 2024-09-14T05:30:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409964+0000", 2024-09-14T05:30:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9858000000000003e-05 2024-09-14T05:30:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410292+0000", 2024-09-14T05:30:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032735899999999999 2024-09-14T05:30:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4007 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.326796334000001, 2024-09-14T05:30:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05184076, 2024-09-14T05:30:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4007 2024-09-14T05:30:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:30:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358891+0000", 2024-09-14T05:30:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9819999999999998e-06 2024-09-14T05:30:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358897+0000", 2024-09-14T05:30:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9390000000000002e-06 2024-09-14T05:30:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358902+0000", 2024-09-14T05:30:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6770000000000003e-06 2024-09-14T05:30:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410335+0000", 2024-09-14T05:30:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051432991999999997 2024-09-14T05:30:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410416+0000", 2024-09-14T05:30:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1316999999999994e-05 2024-09-14T05:30:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410730+0000", 2024-09-14T05:30:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000313419 2024-09-14T05:30:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4008 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7634944~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.326771973, 2024-09-14T05:30:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052208082000000003, 2024-09-14T05:30:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4008 2024-09-14T05:30:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358914+0000", 2024-09-14T05:30:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6100000000000001e-07 2024-09-14T05:30:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358916+0000", 2024-09-14T05:30:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.902e-06 2024-09-14T05:30:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358923+0000", 2024-09-14T05:30:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4780000000000002e-06 2024-09-14T05:30:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358929+0000", 2024-09-14T05:30:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0970000000000002e-06 2024-09-14T05:30:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410761+0000", 2024-09-14T05:30:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051831436000000002 2024-09-14T05:30:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410829+0000", 2024-09-14T05:30:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7874999999999996e-05 2024-09-14T05:30:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411121+0000", 2024-09-14T05:30:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029283299999999999 2024-09-14T05:30:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4009 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8126464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.326624748, 2024-09-14T05:30:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052518169000000003, 2024-09-14T05:30:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4009 2024-09-14T05:30:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0200000000000002e-07 2024-09-14T05:30:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359066+0000", 2024-09-14T05:30:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7899999999999999e-06 2024-09-14T05:30:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359074+0000", 2024-09-14T05:30:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2579999999999995e-06 2024-09-14T05:30:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359082+0000", 2024-09-14T05:30:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.464e-06 2024-09-14T05:30:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411160+0000", 2024-09-14T05:30:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052078078 2024-09-14T05:30:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411244+0000", 2024-09-14T05:30:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4041000000000004e-05 2024-09-14T05:30:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411579+0000", 2024-09-14T05:30:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033493599999999998 2024-09-14T05:30:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:07.786 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:07.789 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:30:07.790 INFO:teuthology.orchestra.run.smithi071.stdout:3481 trunc from 0x82decf2 to 0x9db291 2024-09-14T05:30:07.791 INFO:teuthology.orchestra.run.smithi071.stdout:3482 trunc from 0x9db291 to 0xc0d1b5a 2024-09-14T05:30:07.791 INFO:teuthology.orchestra.run.smithi071.stdout:3483 punch from 0xb4eac32 to 0xb4f4186, (0x9554 bytes) 2024-09-14T05:30:07.791 INFO:teuthology.orchestra.run.smithi071.stdout:3485 write 0xe985a1e thru 0xe986f9c (0x157f bytes) 2024-09-14T05:30:07.791 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:07.791 INFO:teuthology.orchestra.run.smithi071.stdout:3487 punch from 0x660b2f4 to 0x661350a, (0x8216 bytes) 2024-09-14T05:30:07.791 INFO:teuthology.orchestra.run.smithi071.stdout:3488 write 0xb8c0e80 thru 0xb8c5032 (0x41b3 bytes) 2024-09-14T05:30:07.791 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:07.791 INFO:teuthology.orchestra.run.smithi071.stdout:3491 read 0xab38a0e thru 0xab3ee5e (0x6451 bytes) 2024-09-14T05:30:07.791 INFO:teuthology.orchestra.run.smithi071.stdout:3492 read 0x97cdacb thru 0x97d88a0 (0xadd6 bytes) 2024-09-14T05:30:07.792 INFO:teuthology.orchestra.run.smithi071.stdout:3493 trunc from 0xe986f9d to 0xc053c7c 2024-09-14T05:30:07.792 INFO:teuthology.orchestra.run.smithi071.stdout:3494 read 0x42d123e thru 0x42db43c (0xa1ff bytes) 2024-09-14T05:30:07.792 INFO:teuthology.orchestra.run.smithi071.stdout:3495 punch from 0x5799c95 to 0x57a7977, (0xdce2 bytes) 2024-09-14T05:30:07.792 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:07.792 INFO:teuthology.orchestra.run.smithi071.stdout:3497 write 0xd61bf89 thru 0xd628360 (0xc3d8 bytes) 2024-09-14T05:30:07.792 INFO:teuthology.orchestra.run.smithi071.stdout:3498 read 0x559bde3 thru 0x559f351 (0x356f bytes) 2024-09-14T05:30:07.792 INFO:teuthology.orchestra.run.smithi071.stdout:3499 read 0x32cc75f thru 0x32ccbcd (0x46f bytes) 2024-09-14T05:30:07.792 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:07.792 INFO:teuthology.orchestra.run.smithi071.stdout:3501 write 0x3eb9b79 thru 0x3ec85bb (0xea43 bytes) 2024-09-14T05:30:07.792 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:07.792 INFO:teuthology.orchestra.run.smithi071.stdout:3503 write 0x1fcba30 thru 0x1fccf56 (0x1527 bytes) 2024-09-14T05:30:07.793 INFO:teuthology.orchestra.run.smithi071.stdout:3504 read 0x109e212 thru 0x10a95f0 (0xb3df bytes) 2024-09-14T05:30:07.793 INFO:teuthology.orchestra.run.smithi071.stdout:3505 read 0x60061ae thru 0x6013431 (0xd284 bytes) 2024-09-14T05:30:07.793 INFO:teuthology.orchestra.run.smithi071.stdout:3506 write 0x914b6ff thru 0x914ebb1 (0x34b3 bytes) 2024-09-14T05:30:07.793 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:07.793 INFO:teuthology.orchestra.run.smithi071.stdout:3515 writesame 0xb3107ad thru 0xb311e3d data_size 0x1(0x1691 bytes) 2024-09-14T05:30:07.793 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:07.787+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:07.793 INFO:teuthology.orchestra.run.smithi071.stdout:3516 trunc from 0xd628361 to 0xb11dc07 2024-09-14T05:30:07.800 INFO:teuthology.orchestra.run.smithi071.stdout:3517 writesame 0x329e6cb thru 0x32a0550 data_size 0x1(0x1e86 bytes) 2024-09-14T05:30:07.808 INFO:teuthology.orchestra.run.smithi071.stdout:3518 write 0x9810868 thru 0x981259c (0x1d35 bytes) 2024-09-14T05:30:07.812 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:07.819 INFO:teuthology.orchestra.run.smithi071.stdout:3521 trunc from 0xb11dc07 to 0x5262cee 2024-09-14T05:30:07.840 INFO:teuthology.orchestra.run.smithi071.stdout:3522 write 0xa92c8c7 thru 0xa92d684 (0xdbe bytes) 2024-09-14T05:30:07.850 INFO:teuthology.orchestra.run.smithi071.stdout:3523 write 0xb89afb0 thru 0xb8aabb3 (0xfc04 bytes) 2024-09-14T05:30:07.857 INFO:teuthology.orchestra.run.smithi071.stdout:3524 write 0x7c5a00b thru 0x7c670c9 (0xd0bf bytes) 2024-09-14T05:30:07.862 INFO:teuthology.orchestra.run.smithi071.stdout:3526 read 0x9ad2467 thru 0x9ad29de (0x578 bytes) 2024-09-14T05:30:07.862 INFO:teuthology.orchestra.run.smithi071.stdout:3528 write 0x971193a thru 0x971b7f0 (0x9eb7 bytes) 2024-09-14T05:30:07.868 INFO:teuthology.orchestra.run.smithi071.stdout:3529 punch from 0xc15b12 to 0xc20949, (0xae37 bytes) 2024-09-14T05:30:07.873 INFO:teuthology.orchestra.run.smithi071.stdout:3534 trunc from 0xb8aabb4 to 0x6b9d13b 2024-09-14T05:30:07.888 INFO:teuthology.orchestra.run.smithi071.stdout:3535 punch from 0x11ad64d to 0x11bbd65, (0xe718 bytes) 2024-09-14T05:30:07.888 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:07.887+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:07.897 INFO:teuthology.orchestra.run.smithi071.stdout:3536 write 0xabf862d thru 0xabfd7c7 (0x519b bytes) 2024-09-14T05:30:07.906 INFO:teuthology.orchestra.run.smithi071.stdout:3537 writesame 0xaff9427 thru 0xaffde32 data_size 0xe(0x4a0c bytes) 2024-09-14T05:30:07.907 INFO:teuthology.orchestra.run.smithi071.stdout:3539 write 0xa51dd4b thru 0xa51e491 (0x747 bytes) 2024-09-14T05:30:07.909 INFO:teuthology.orchestra.run.smithi071.stdout:3540 read 0x692d17b thru 0x6938f9b (0xbe21 bytes) 2024-09-14T05:30:07.912 INFO:teuthology.orchestra.run.smithi071.stdout:3541 trunc from 0xaffde33 to 0x907b5e4 2024-09-14T05:30:07.917 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:07.917 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:07.917 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:07.917 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:07.922 INFO:teuthology.orchestra.run.smithi071.stdout:3542 read 0x4aa78a3 thru 0x4ab5668 (0xddc6 bytes) 2024-09-14T05:30:07.927 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:30:07.929 INFO:teuthology.orchestra.run.smithi071.stdout:3543 write 0x9d81738 thru 0x9d903f8 (0xecc1 bytes) 2024-09-14T05:30:07.937 INFO:teuthology.orchestra.run.smithi071.stdout:3545 read 0x8314e50 thru 0x8315de0 (0xf91 bytes) 2024-09-14T05:30:07.941 INFO:teuthology.orchestra.run.smithi071.stdout:3547 trunc from 0x9d903f9 to 0xcdddd39 2024-09-14T05:30:07.946 INFO:teuthology.orchestra.run.smithi071.stdout:3549 read 0x496624f thru 0x4970d85 (0xab37 bytes) 2024-09-14T05:30:07.946 INFO:teuthology.orchestra.run.smithi071.stdout:3551 read 0x870a476 thru 0x871228a (0x7e15 bytes) 2024-09-14T05:30:07.946 INFO:teuthology.orchestra.run.smithi071.stdout:3554 write 0xc8532b0 thru 0xc85c023 (0x8d74 bytes) 2024-09-14T05:30:07.957 INFO:teuthology.orchestra.run.smithi071.stdout:3555 write 0x8212dcc thru 0x8220c11 (0xde46 bytes) 2024-09-14T05:30:07.965 INFO:teuthology.orchestra.run.smithi071.stdout:3556 write 0x484e03d thru 0x485bf0d (0xded1 bytes) 2024-09-14T05:30:07.970 INFO:teuthology.orchestra.run.smithi071.stdout:3558 read 0x15228a5 thru 0x152b970 (0x90cc bytes) 2024-09-14T05:30:07.971 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:07.971 INFO:teuthology.orchestra.run.smithi071.stdout:3560 write 0xcdf969 thru 0xcedad4 (0xe16c bytes) 2024-09-14T05:30:07.985 INFO:teuthology.orchestra.run.smithi071.stdout:3562 trunc from 0xcdddd39 to 0x91a8300 2024-09-14T05:30:07.989 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:07.987+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:08.001 INFO:teuthology.orchestra.run.smithi071.stdout:3563 trunc from 0x91a8300 to 0xd660fed 2024-09-14T05:30:08.007 INFO:teuthology.orchestra.run.smithi071.stdout:3564 writesame 0x671b64d thru 0x671c9f0 data_size 0x6(0x13a4 bytes) 2024-09-14T05:30:08.011 INFO:teuthology.orchestra.run.smithi071.stdout:3566 write 0xa688f3b thru 0xa68c101 (0x31c7 bytes) 2024-09-14T05:30:08.024 INFO:teuthology.orchestra.run.smithi071.stdout:3567 trunc from 0xd660fed to 0x6a64be9 2024-09-14T05:30:08.042 INFO:teuthology.orchestra.run.smithi071.stdout:3568 write 0x81d42b2 thru 0x81d8b63 (0x48b2 bytes) 2024-09-14T05:30:08.048 INFO:teuthology.orchestra.run.smithi071.stdout:3573 read 0x7fe5d5 thru 0x80777b (0x91a7 bytes) 2024-09-14T05:30:08.049 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.051 INFO:teuthology.orchestra.run.smithi071.stdout:3575 trunc from 0x81d8b64 to 0x3e99a15 2024-09-14T05:30:08.053 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:08.053 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:08.053 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:08.053 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:08.053 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:08.063 INFO:teuthology.orchestra.run.smithi071.stdout:3576 read 0x33e8852 thru 0x33f29a8 (0xa157 bytes) 2024-09-14T05:30:08.064 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:30:08.069 INFO:teuthology.orchestra.run.smithi071.stdout:3577 trunc from 0x3e99a15 to 0x8f91c40 2024-09-14T05:30:08.074 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.085 INFO:teuthology.orchestra.run.smithi071.stdout:3580 write 0xcd70f24 thru 0xcd71cb2 (0xd8f bytes) 2024-09-14T05:30:08.087 INFO:teuthology.orchestra.run.smithi071.stdout:3581 read 0x7d0227d thru 0x7d0c21b (0x9f9f bytes) 2024-09-14T05:30:08.089 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:08.089+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:08.094 INFO:teuthology.orchestra.run.smithi071.stdout:3582 trunc from 0xcd71cb3 to 0x7b0475a 2024-09-14T05:30:08.103 INFO:teuthology.orchestra.run.smithi071.stdout:3583 read 0x3480ff thru 0x351169 (0x906b bytes) 2024-09-14T05:30:08.104 INFO:teuthology.orchestra.run.smithi071.stdout:3584 read 0x7ae5574 thru 0x7af174b (0xc1d8 bytes) 2024-09-14T05:30:08.113 INFO:teuthology.orchestra.run.smithi071.stdout:3586 writesame 0xad0f108 thru 0xad16d39 data_size 0x15(0x7c32 bytes) 2024-09-14T05:30:08.113 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.121 INFO:teuthology.orchestra.run.smithi071.stdout:3588 trunc from 0xad16d3a to 0x42af559 2024-09-14T05:30:08.127 INFO:teuthology.orchestra.run.smithi071.stdout:3589 read 0x22b184c thru 0x22b4e24 (0x35d9 bytes) 2024-09-14T05:30:08.127 INFO:teuthology.orchestra.run.smithi071.stdout:3590 punch from 0x1546bf2 to 0x154e8ac, (0x7cba bytes) 2024-09-14T05:30:08.141 INFO:teuthology.orchestra.run.smithi071.stdout:3591 write 0xb0cf3a6 thru 0xb0debc2 (0xf81d bytes) 2024-09-14T05:30:08.162 INFO:teuthology.orchestra.run.smithi071.stdout:3593 write 0xc7a896f thru 0xc7aa0c6 (0x1758 bytes) 2024-09-14T05:30:08.166 INFO:teuthology.orchestra.run.smithi071.stdout:3594 read 0x10cee45 thru 0x10d1937 (0x2af3 bytes) 2024-09-14T05:30:08.167 INFO:teuthology.orchestra.run.smithi071.stdout:3597 write 0x3f897fc thru 0x3f96567 (0xcd6c bytes) 2024-09-14T05:30:08.172 INFO:teuthology.orchestra.run.smithi071.stdout:3598 write 0x494b165 thru 0x49574d6 (0xc372 bytes) 2024-09-14T05:30:08.183 INFO:teuthology.orchestra.run.smithi071.stdout:3601 trunc from 0xc7aa0c7 to 0x852ac86 2024-09-14T05:30:08.190 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:08.189+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:08.193 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.194 INFO:teuthology.orchestra.run.smithi071.stdout:3604 read 0x3772bdf thru 0x377ad93 (0x81b5 bytes) 2024-09-14T05:30:08.196 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:08.196 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:08.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:08.196 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:08.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.196 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74187 3.f 3:f707b2d9:::rbd_data.112d46b1ca94.0000000000000061:11 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.196 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:08.196 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.625883849000001, 2024-09-14T05:30:08.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072367828999999995, 2024-09-14T05:30:08.197 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.197 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.197 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.197 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.197 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.197 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.197 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74187 2024-09-14T05:30:08.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.197 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:08.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:08.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:08.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0900000000000002e-07 2024-09-14T05:30:08.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568166+0000", 2024-09-14T05:30:08.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.683e-06 2024-09-14T05:30:08.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568172+0000", 2024-09-14T05:30:08.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0639999999999999e-06 2024-09-14T05:30:08.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568177+0000", 2024-09-14T05:30:08.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.967e-06 2024-09-14T05:30:08.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640193+0000", 2024-09-14T05:30:08.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072016284 2024-09-14T05:30:08.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640282+0000", 2024-09-14T05:30:08.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9011999999999994e-05 2024-09-14T05:30:08.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640532+0000", 2024-09-14T05:30:08.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024940999999999999 2024-09-14T05:30:08.200 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.200 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.200 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.201 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74188 3.f 3:f707b2d9:::rbd_data.112d46b1ca94.0000000000000061:11 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.201 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:08.201 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.625864476, 2024-09-14T05:30:08.201 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072600425999999996, 2024-09-14T05:30:08.201 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.201 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.201 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.201 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.201 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.201 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.201 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74188 2024-09-14T05:30:08.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.202 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:08.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:08.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568184+0000", 2024-09-14T05:30:08.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8700000000000001e-07 2024-09-14T05:30:08.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568200+0000", 2024-09-14T05:30:08.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6603e-05 2024-09-14T05:30:08.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568208+0000", 2024-09-14T05:30:08.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4800000000000004e-06 2024-09-14T05:30:08.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568213+0000", 2024-09-14T05:30:08.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6860000000000003e-06 2024-09-14T05:30:08.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640563+0000", 2024-09-14T05:30:08.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072349438000000002 2024-09-14T05:30:08.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640607+0000", 2024-09-14T05:30:08.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4014999999999998e-05 2024-09-14T05:30:08.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640784+0000", 2024-09-14T05:30:08.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017681700000000001 2024-09-14T05:30:08.205 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.205 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.205 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.205 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74194 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.205 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:08.205 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.625826344, 2024-09-14T05:30:08.206 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072795199000000005, 2024-09-14T05:30:08.206 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.206 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.206 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.206 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.206 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.206 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.206 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74194 2024-09-14T05:30:08.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.206 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.206 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.206 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:08.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:08.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568222+0000", 2024-09-14T05:30:08.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1300000000000001e-07 2024-09-14T05:30:08.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568224+0000", 2024-09-14T05:30:08.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0660000000000002e-06 2024-09-14T05:30:08.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568230+0000", 2024-09-14T05:30:08.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4470000000000003e-06 2024-09-14T05:30:08.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568235+0000", 2024-09-14T05:30:08.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1220000000000003e-06 2024-09-14T05:30:08.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640824+0000", 2024-09-14T05:30:08.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072588707000000002 2024-09-14T05:30:08.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640915+0000", 2024-09-14T05:30:08.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1008999999999995e-05 2024-09-14T05:30:08.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641017+0000", 2024-09-14T05:30:08.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010143499999999999 2024-09-14T05:30:08.209 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.210 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.210 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.210 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74208 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2693120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.210 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:08.210 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.625805767999999, 2024-09-14T05:30:08.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072905555999999996, 2024-09-14T05:30:08.210 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.210 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.210 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.210 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.210 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.211 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.211 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74208 2024-09-14T05:30:08.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.211 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:08.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:08.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:08.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9799999999999999e-07 2024-09-14T05:30:08.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568244+0000", 2024-09-14T05:30:08.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0339999999999999e-06 2024-09-14T05:30:08.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568251+0000", 2024-09-14T05:30:08.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4060000000000002e-06 2024-09-14T05:30:08.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568255+0000", 2024-09-14T05:30:08.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6589999999999998e-06 2024-09-14T05:30:08.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641048+0000", 2024-09-14T05:30:08.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072792647000000002 2024-09-14T05:30:08.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641086+0000", 2024-09-14T05:30:08.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8031999999999998e-05 2024-09-14T05:30:08.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641147+0000", 2024-09-14T05:30:08.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1379999999999998e-05 2024-09-14T05:30:08.214 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.214 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.214 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.214 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74236 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2662400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.215 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:08.215 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.625172105000001, 2024-09-14T05:30:08.215 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072517776000000006, 2024-09-14T05:30:08.215 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.215 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.215 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.215 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.215 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.215 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.215 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74236 2024-09-14T05:30:08.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.215 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.215 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:08.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:08.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568881+0000", 2024-09-14T05:30:08.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2340000000000002e-06 2024-09-14T05:30:08.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568886+0000", 2024-09-14T05:30:08.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9450000000000001e-06 2024-09-14T05:30:08.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568898+0000", 2024-09-14T05:30:08.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2150000000000001e-05 2024-09-14T05:30:08.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568907+0000", 2024-09-14T05:30:08.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9239999999999996e-06 2024-09-14T05:30:08.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641185+0000", 2024-09-14T05:30:08.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.07227807 2024-09-14T05:30:08.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641265+0000", 2024-09-14T05:30:08.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0575999999999996e-05 2024-09-14T05:30:08.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641393+0000", 2024-09-14T05:30:08.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012787700000000001 2024-09-14T05:30:08.219 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.219 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.219 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.219 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74239 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2695168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.219 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:08.219 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.625127712000001, 2024-09-14T05:30:08.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072593816000000005, 2024-09-14T05:30:08.219 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.219 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.219 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.219 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.219 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.220 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.220 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74239 2024-09-14T05:30:08.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.220 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:08.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:08.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:08.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4300000000000003e-07 2024-09-14T05:30:08.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568923+0000", 2024-09-14T05:30:08.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2019999999999998e-06 2024-09-14T05:30:08.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568933+0000", 2024-09-14T05:30:08.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0376e-05 2024-09-14T05:30:08.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568938+0000", 2024-09-14T05:30:08.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2850000000000002e-06 2024-09-14T05:30:08.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641424+0000", 2024-09-14T05:30:08.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072485230999999997 2024-09-14T05:30:08.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641464+0000", 2024-09-14T05:30:08.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0574000000000002e-05 2024-09-14T05:30:08.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641514+0000", 2024-09-14T05:30:08.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9604999999999997e-05 2024-09-14T05:30:08.223 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.223 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.223 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.223 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74252 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2664448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.223 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:08.224 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.624947798000001, 2024-09-14T05:30:08.224 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072631237000000001, 2024-09-14T05:30:08.224 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.224 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.224 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.224 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.224 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.224 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.224 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74252 2024-09-14T05:30:08.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.224 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.224 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:08.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:08.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569101+0000", 2024-09-14T05:30:08.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8700000000000005e-07 2024-09-14T05:30:08.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569104+0000", 2024-09-14T05:30:08.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.163e-06 2024-09-14T05:30:08.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569115+0000", 2024-09-14T05:30:08.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1335000000000001e-05 2024-09-14T05:30:08.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569121+0000", 2024-09-14T05:30:08.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0800000000000002e-06 2024-09-14T05:30:08.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641552+0000", 2024-09-14T05:30:08.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072430743000000006 2024-09-14T05:30:08.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641631+0000", 2024-09-14T05:30:08.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9388999999999995e-05 2024-09-14T05:30:08.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641731+0000", 2024-09-14T05:30:08.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9840000000000006e-05 2024-09-14T05:30:08.228 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.228 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.228 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.228 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.228 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74256 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2697216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.228 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:08.228 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.624916742, 2024-09-14T05:30:08.228 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072721090000000002, 2024-09-14T05:30:08.228 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.228 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.228 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.229 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.229 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.229 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.229 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74256 2024-09-14T05:30:08.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.229 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.229 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.229 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:08.229 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.229 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.229 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:08.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:08.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0299999999999999e-07 2024-09-14T05:30:08.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569133+0000", 2024-09-14T05:30:08.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.046e-06 2024-09-14T05:30:08.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569139+0000", 2024-09-14T05:30:08.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0279999999999997e-06 2024-09-14T05:30:08.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569144+0000", 2024-09-14T05:30:08.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6929999999999998e-06 2024-09-14T05:30:08.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641762+0000", 2024-09-14T05:30:08.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072617904999999996 2024-09-14T05:30:08.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.232 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641801+0000", 2024-09-14T05:30:08.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8902999999999997e-05 2024-09-14T05:30:08.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.232 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641852+0000", 2024-09-14T05:30:08.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1011999999999999e-05 2024-09-14T05:30:08.232 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.232 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.232 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.232 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74285 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2672640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.233 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:08.233 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.624071306000001, 2024-09-14T05:30:08.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072422069000000006, 2024-09-14T05:30:08.233 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.233 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.233 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.233 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.233 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.233 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.233 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74285 2024-09-14T05:30:08.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.233 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.233 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:08.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:08.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569977+0000", 2024-09-14T05:30:08.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3099999999999998e-07 2024-09-14T05:30:08.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569978+0000", 2024-09-14T05:30:08.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.598e-06 2024-09-14T05:30:08.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569984+0000", 2024-09-14T05:30:08.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4689999999999996e-06 2024-09-14T05:30:08.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569989+0000", 2024-09-14T05:30:08.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8409999999999999e-06 2024-09-14T05:30:08.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642229+0000", 2024-09-14T05:30:08.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072240046000000002 2024-09-14T05:30:08.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642306+0000", 2024-09-14T05:30:08.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7347000000000003e-05 2024-09-14T05:30:08.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642398+0000", 2024-09-14T05:30:08.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2336999999999994e-05 2024-09-14T05:30:08.237 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.237 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.237 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.237 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74292 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2705408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.237 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:08.237 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.623816598000001, 2024-09-14T05:30:08.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072361018999999999, 2024-09-14T05:30:08.237 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.237 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.237 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.237 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.237 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74292 2024-09-14T05:30:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570232+0000", 2024-09-14T05:30:08.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.06e-07 2024-09-14T05:30:08.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570235+0000", 2024-09-14T05:30:08.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3189999999999999e-06 2024-09-14T05:30:08.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570248+0000", 2024-09-14T05:30:08.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3006e-05 2024-09-14T05:30:08.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570254+0000", 2024-09-14T05:30:08.240 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.908e-06 2024-09-14T05:30:08.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642433+0000", 2024-09-14T05:30:08.240 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072179192000000003 2024-09-14T05:30:08.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642500+0000", 2024-09-14T05:30:08.241 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7080000000000001e-05 2024-09-14T05:30:08.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.241 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.241 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642592+0000", 2024-09-14T05:30:08.241 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1807999999999999e-05 2024-09-14T05:30:08.241 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.241 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.241 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.241 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74304 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2707456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.241 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:08.241 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.623459931999999, 2024-09-14T05:30:08.242 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072444960000000003, 2024-09-14T05:30:08.242 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.242 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.242 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.242 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.242 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.242 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.242 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74304 2024-09-14T05:30:08.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.242 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.242 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.242 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:08.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.243 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.243 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:08.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.243 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.243 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:08.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9800000000000004e-07 2024-09-14T05:30:08.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570590+0000", 2024-09-14T05:30:08.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0810000000000001e-06 2024-09-14T05:30:08.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570596+0000", 2024-09-14T05:30:08.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9789999999999998e-06 2024-09-14T05:30:08.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570601+0000", 2024-09-14T05:30:08.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6750000000000001e-06 2024-09-14T05:30:08.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.245 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642835+0000", 2024-09-14T05:30:08.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.07223359 2024-09-14T05:30:08.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.245 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642917+0000", 2024-09-14T05:30:08.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2013000000000002e-05 2024-09-14T05:30:08.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.245 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643033+0000", 2024-09-14T05:30:08.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116124 2024-09-14T05:30:08.246 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.246 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.246 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.246 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74321 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2709504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.246 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:08.246 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.622977941999999, 2024-09-14T05:30:08.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072350611999999995, 2024-09-14T05:30:08.246 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.246 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.246 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.246 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.246 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.247 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.247 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74321 2024-09-14T05:30:08.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.247 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.247 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.247 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:08.247 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.247 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.247 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:08.247 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.248 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.248 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:08.248 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3e-07 2024-09-14T05:30:08.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.248 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.248 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571073+0000", 2024-09-14T05:30:08.248 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6879999999999999e-06 2024-09-14T05:30:08.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.248 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.248 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571080+0000", 2024-09-14T05:30:08.249 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0319999999999998e-06 2024-09-14T05:30:08.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.249 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.249 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571085+0000", 2024-09-14T05:30:08.249 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2109999999999997e-06 2024-09-14T05:30:08.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.249 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.249 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643309+0000", 2024-09-14T05:30:08.249 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072223812999999998 2024-09-14T05:30:08.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.249 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.250 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643351+0000", 2024-09-14T05:30:08.250 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2239e-05 2024-09-14T05:30:08.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.250 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.250 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643420+0000", 2024-09-14T05:30:08.250 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8998999999999994e-05 2024-09-14T05:30:08.250 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.250 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.250 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.250 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74332 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2678784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.250 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:08.250 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.622654779000001, 2024-09-14T05:30:08.251 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072259062999999998, 2024-09-14T05:30:08.251 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.251 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.251 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.251 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.251 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.251 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.251 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74332 2024-09-14T05:30:08.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.251 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.251 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.251 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:08.251 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.252 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:08.252 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.252 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571394+0000", 2024-09-14T05:30:08.252 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:30:08.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.253 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571397+0000", 2024-09-14T05:30:08.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8469999999999998e-06 2024-09-14T05:30:08.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.253 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.253 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571411+0000", 2024-09-14T05:30:08.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.34e-05 2024-09-14T05:30:08.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.253 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.253 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571418+0000", 2024-09-14T05:30:08.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9149999999999999e-06 2024-09-14T05:30:08.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.254 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643458+0000", 2024-09-14T05:30:08.254 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072039881 2024-09-14T05:30:08.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.254 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643539+0000", 2024-09-14T05:30:08.254 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1650000000000006e-05 2024-09-14T05:30:08.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.254 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643652+0000", 2024-09-14T05:30:08.254 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000112593 2024-09-14T05:30:08.254 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.255 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.255 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.255 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74336 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2711552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.255 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:08.255 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.622618329000002, 2024-09-14T05:30:08.255 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072354038999999995, 2024-09-14T05:30:08.255 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.255 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.255 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.255 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.255 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.255 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.256 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74336 2024-09-14T05:30:08.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.256 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.256 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.256 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:08.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.256 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.256 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:08.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.257 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571430+0000", 2024-09-14T05:30:08.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8999999999999996e-07 2024-09-14T05:30:08.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.257 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571432+0000", 2024-09-14T05:30:08.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4159999999999998e-06 2024-09-14T05:30:08.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.257 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571439+0000", 2024-09-14T05:30:08.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.657e-06 2024-09-14T05:30:08.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571444+0000", 2024-09-14T05:30:08.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1329999999999998e-06 2024-09-14T05:30:08.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643690+0000", 2024-09-14T05:30:08.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072246088 2024-09-14T05:30:08.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643731+0000", 2024-09-14T05:30:08.260 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0311000000000001e-05 2024-09-14T05:30:08.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.260 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643783+0000", 2024-09-14T05:30:08.260 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2843999999999998e-05 2024-09-14T05:30:08.261 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.261 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.261 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.261 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74349 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.261 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:08.261 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.622597320000001, 2024-09-14T05:30:08.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072554297000000004, 2024-09-14T05:30:08.261 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.261 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.261 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.261 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.261 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.262 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.262 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74349 2024-09-14T05:30:08.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.262 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.262 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.262 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:08.262 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.262 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.262 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:08.262 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571451+0000", 2024-09-14T05:30:08.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.41e-07 2024-09-14T05:30:08.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571453+0000", 2024-09-14T05:30:08.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8360000000000001e-06 2024-09-14T05:30:08.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571475+0000", 2024-09-14T05:30:08.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2452000000000001e-05 2024-09-14T05:30:08.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571480+0000", 2024-09-14T05:30:08.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4430000000000002e-06 2024-09-14T05:30:08.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643821+0000", 2024-09-14T05:30:08.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072340347999999999 2024-09-14T05:30:08.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643904+0000", 2024-09-14T05:30:08.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3534000000000004e-05 2024-09-14T05:30:08.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.265 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.265 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644005+0000", 2024-09-14T05:30:08.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100343 2024-09-14T05:30:08.265 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.265 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.265 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.265 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74357 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2713600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.265 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:08.265 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.622553286999999, 2024-09-14T05:30:08.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072651144000000001, 2024-09-14T05:30:08.266 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.266 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.266 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.266 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.266 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.266 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.266 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74357 2024-09-14T05:30:08.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.266 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.266 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:08.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:08.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571495+0000", 2024-09-14T05:30:08.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8899999999999999e-07 2024-09-14T05:30:08.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571498+0000", 2024-09-14T05:30:08.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5840000000000002e-06 2024-09-14T05:30:08.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571504+0000", 2024-09-14T05:30:08.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6549999999999998e-06 2024-09-14T05:30:08.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571509+0000", 2024-09-14T05:30:08.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6500000000000004e-06 2024-09-14T05:30:08.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644043+0000", 2024-09-14T05:30:08.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072533832000000006 2024-09-14T05:30:08.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.269 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.269 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644090+0000", 2024-09-14T05:30:08.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7426000000000003e-05 2024-09-14T05:30:08.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.269 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.269 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644146+0000", 2024-09-14T05:30:08.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5408000000000002e-05 2024-09-14T05:30:08.269 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.269 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.269 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.270 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74364 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2682880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.270 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:08.270 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.622216515000002, 2024-09-14T05:30:08.270 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072531227000000004, 2024-09-14T05:30:08.270 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.270 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.270 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.270 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.270 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.270 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.270 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74364 2024-09-14T05:30:08.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.271 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.271 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.271 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:08.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.271 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.271 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:08.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.272 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571832+0000", 2024-09-14T05:30:08.272 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4e-07 2024-09-14T05:30:08.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.272 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571838+0000", 2024-09-14T05:30:08.272 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0519999999999998e-06 2024-09-14T05:30:08.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.272 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571865+0000", 2024-09-14T05:30:08.272 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7109000000000001e-05 2024-09-14T05:30:08.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571883+0000", 2024-09-14T05:30:08.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7428e-05 2024-09-14T05:30:08.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644182+0000", 2024-09-14T05:30:08.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072299551000000004 2024-09-14T05:30:08.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644260+0000", 2024-09-14T05:30:08.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7551000000000003e-05 2024-09-14T05:30:08.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.274 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.274 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644362+0000", 2024-09-14T05:30:08.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010259600000000001 2024-09-14T05:30:08.274 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.274 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.274 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.274 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74367 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2715648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.274 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:08.275 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.621853208000001, 2024-09-14T05:30:08.275 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072368259000000004, 2024-09-14T05:30:08.275 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.275 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.275 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.275 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.275 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.275 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.275 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74367 2024-09-14T05:30:08.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.275 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.275 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.275 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:08.276 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.276 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.276 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:08.276 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.276 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.276 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572195+0000", 2024-09-14T05:30:08.276 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.78e-07 2024-09-14T05:30:08.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.276 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572200+0000", 2024-09-14T05:30:08.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2520000000000001e-06 2024-09-14T05:30:08.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572213+0000", 2024-09-14T05:30:08.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3618000000000001e-05 2024-09-14T05:30:08.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572226+0000", 2024-09-14T05:30:08.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3040999999999999e-05 2024-09-14T05:30:08.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644409+0000", 2024-09-14T05:30:08.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072182982000000007 2024-09-14T05:30:08.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644489+0000", 2024-09-14T05:30:08.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9429000000000001e-05 2024-09-14T05:30:08.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644563+0000", 2024-09-14T05:30:08.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4059000000000004e-05 2024-09-14T05:30:08.278 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.279 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.279 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.279 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74380 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2684928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.279 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:08.279 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.621719429999999, 2024-09-14T05:30:08.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072442180999999994, 2024-09-14T05:30:08.279 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.279 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.279 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.279 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.279 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.279 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.280 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74380 2024-09-14T05:30:08.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.280 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.280 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.280 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:08.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.280 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.280 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:08.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.281 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.281 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572329+0000", 2024-09-14T05:30:08.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3300000000000001e-07 2024-09-14T05:30:08.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.281 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.281 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572333+0000", 2024-09-14T05:30:08.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8e-06 2024-09-14T05:30:08.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.281 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.281 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572346+0000", 2024-09-14T05:30:08.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.307e-05 2024-09-14T05:30:08.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572353+0000", 2024-09-14T05:30:08.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9e-06 2024-09-14T05:30:08.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644597+0000", 2024-09-14T05:30:08.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072244770999999999 2024-09-14T05:30:08.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644672+0000", 2024-09-14T05:30:08.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4945999999999997e-05 2024-09-14T05:30:08.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644770+0000", 2024-09-14T05:30:08.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7961000000000006e-05 2024-09-14T05:30:08.283 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.283 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.283 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.283 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74383 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2717696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.283 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:08.283 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 18.621463435999999, 2024-09-14T05:30:08.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072397258000000006, 2024-09-14T05:30:08.284 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.284 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.284 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.284 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.284 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.284 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.284 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74383 2024-09-14T05:30:08.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.284 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:08.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:08.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572585+0000", 2024-09-14T05:30:08.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:30:08.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572589+0000", 2024-09-14T05:30:08.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:30:08.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572602+0000", 2024-09-14T05:30:08.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2917e-05 2024-09-14T05:30:08.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572610+0000", 2024-09-14T05:30:08.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.943e-06 2024-09-14T05:30:08.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644791+0000", 2024-09-14T05:30:08.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072181113000000005 2024-09-14T05:30:08.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644861+0000", 2024-09-14T05:30:08.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0650999999999998e-05 2024-09-14T05:30:08.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644981+0000", 2024-09-14T05:30:08.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000120185 2024-09-14T05:30:08.287 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.288 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.288 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.288 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.288 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.288 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:08.288 INFO:teuthology.orchestra.run.smithi071.stdout:3606 trunc from 0x852ac86 to 0xed9db56 2024-09-14T05:30:08.289 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:30:08.290 INFO:teuthology.orchestra.run.smithi071.stdout:3607 write 0x30bad23 thru 0x30c6be9 (0xbec7 bytes) 2024-09-14T05:30:08.290 INFO:teuthology.orchestra.run.smithi071.stdout:3609 trunc from 0xed9db56 to 0x441e0a5 2024-09-14T05:30:08.290 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-14T05:30:08.290 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T05:30:08.292 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T05:30:08.293 INFO:teuthology.orchestra.run.smithi071.stdout:3610 trunc from 0x441e0a5 to 0x92d6b57 2024-09-14T05:30:08.293 INFO:teuthology.orchestra.run.smithi071.stdout:3611 trunc from 0x92d6b57 to 0x893a906 2024-09-14T05:30:08.293 INFO:teuthology.orchestra.run.smithi071.stdout:3612 punch from 0x6c882cb to 0x6c96bc5, (0xe8fa bytes) 2024-09-14T05:30:08.293 INFO:teuthology.orchestra.run.smithi071.stdout:3616 punch from 0x9f4d40 to 0x9f7846, (0x2b06 bytes) 2024-09-14T05:30:08.293 INFO:teuthology.orchestra.run.smithi071.stdout:3618 write 0x83b5d2c thru 0x83c1fae (0xc283 bytes) 2024-09-14T05:30:08.293 INFO:teuthology.orchestra.run.smithi071.stdout:3621 write 0x1816cbf thru 0x1822120 (0xb462 bytes) 2024-09-14T05:30:08.293 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.294 INFO:teuthology.orchestra.run.smithi071.stdout:3624 trunc from 0x893a906 to 0xcf32c2a 2024-09-14T05:30:08.294 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:08.293+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:08.299 INFO:teuthology.orchestra.run.smithi071.stdout:3625 read 0x55d07e3 thru 0x55db593 (0xadb1 bytes) 2024-09-14T05:30:08.299 INFO:teuthology.orchestra.run.smithi071.stdout:3628 punch from 0x46a5d73 to 0x46aa5df, (0x486c bytes) 2024-09-14T05:30:08.299 INFO:teuthology.orchestra.run.smithi071.stdout:3630 write 0xa083a78 thru 0xa08d894 (0x9e1d bytes) 2024-09-14T05:30:08.310 INFO:teuthology.orchestra.run.smithi071.stdout:3631 write 0x653cc5e thru 0x6547a35 (0xadd8 bytes) 2024-09-14T05:30:08.319 INFO:teuthology.orchestra.run.smithi071.stdout:3632 read 0xa5c7b64 thru 0xa5cc2ef (0x478c bytes) 2024-09-14T05:30:08.319 INFO:teuthology.orchestra.run.smithi071.stdout:3633 punch from 0x5e279e6 to 0x5e2dac4, (0x60de bytes) 2024-09-14T05:30:08.319 INFO:teuthology.orchestra.run.smithi071.stdout:3634 read 0x3bbf4cd thru 0x3bc9181 (0x9cb5 bytes) 2024-09-14T05:30:08.319 INFO:teuthology.orchestra.run.smithi071.stdout:3635 punch from 0x389d67e to 0x38ad24a, (0xfbcc bytes) 2024-09-14T05:30:08.319 INFO:teuthology.orchestra.run.smithi071.stdout:3636 writesame 0x774c3da thru 0x775aabb data_size 0x3(0xe6e2 bytes) 2024-09-14T05:30:08.333 INFO:teuthology.orchestra.run.smithi071.stdout:3637 trunc from 0xcf32c2a to 0xdd867c5 2024-09-14T05:30:08.338 INFO:teuthology.orchestra.run.smithi071.stdout:3638 read 0x787b820 thru 0x7889356 (0xdb37 bytes) 2024-09-14T05:30:08.338 INFO:teuthology.orchestra.run.smithi071.stdout:3640 write 0xbf7d9bc thru 0xbf7ecb9 (0x12fe bytes) 2024-09-14T05:30:08.348 INFO:teuthology.orchestra.run.smithi071.stdout:3643 write 0xe10600f thru 0xe10cb7b (0x6b6d bytes) 2024-09-14T05:30:08.353 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.353 INFO:teuthology.orchestra.run.smithi071.stdout:3646 write 0xa15e949 thru 0xa16e8a4 (0xff5c bytes) 2024-09-14T05:30:08.362 INFO:teuthology.orchestra.run.smithi071.stdout:3647 writesame 0xa34fc81 thru 0xa35a0f0 data_size 0x18(0xa470 bytes) 2024-09-14T05:30:08.368 INFO:teuthology.orchestra.run.smithi071.stdout:3648 write 0x2d77a65 thru 0x2d78d5c (0x12f8 bytes) 2024-09-14T05:30:08.370 INFO:teuthology.orchestra.run.smithi071.stdout:3649 read 0xab87678 thru 0xab8eeac (0x7835 bytes) 2024-09-14T05:30:08.370 INFO:teuthology.orchestra.run.smithi071.stdout:3650 write 0xc239cd1 thru 0xc245657 (0xb987 bytes) 2024-09-14T05:30:08.376 INFO:teuthology.orchestra.run.smithi071.stdout:3652 read 0x9a0076d thru 0x9a06c60 (0x64f4 bytes) 2024-09-14T05:30:08.376 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.376 INFO:teuthology.orchestra.run.smithi071.stdout:3655 read 0xdc47526 thru 0xdc480c0 (0xb9b bytes) 2024-09-14T05:30:08.386 INFO:teuthology.orchestra.run.smithi071.stdout:3656 trunc from 0xe10cb7c to 0x5e46ea1 2024-09-14T05:30:08.395 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:08.394+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:08.404 INFO:teuthology.orchestra.run.smithi071.stdout:3657 read 0x3aec7f3 thru 0x3aef0de (0x28ec bytes) 2024-09-14T05:30:08.405 INFO:teuthology.orchestra.run.smithi071.stdout:3658 read 0x42fec69 thru 0x43065c8 (0x7960 bytes) 2024-09-14T05:30:08.414 INFO:teuthology.orchestra.run.smithi071.stdout:3659 write 0x8aa8f6c thru 0x8aa97f3 (0x888 bytes) 2024-09-14T05:30:08.417 INFO:teuthology.orchestra.run.smithi071.stdout:3662 read 0x55bef2 thru 0x55f827 (0x3936 bytes) 2024-09-14T05:30:08.417 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.424 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:08.424 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:08.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.425 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:112034 3.1a 3:5af8b9d6:::rbd_header.112d4952ad3:head [call rbd.set_size in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-14T05:30:08.425 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:08.422268+0000", 2024-09-14T05:30:08.425 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.002211859, 2024-09-14T05:30:08.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0022292700000000002, 2024-09-14T05:30:08.425 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.425 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.425 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "waiting for sub ops", 2024-09-14T05:30:08.425 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.425 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.425 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.425 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 112034 2024-09-14T05:30:08.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.426 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.426 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:08.422268+0000", 2024-09-14T05:30:08.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.426 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:08.422268+0000", 2024-09-14T05:30:08.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:08.422269+0000", 2024-09-14T05:30:08.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.796e-06 2024-09-14T05:30:08.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:08.422275+0000", 2024-09-14T05:30:08.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1210000000000003e-06 2024-09-14T05:30:08.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:08.422288+0000", 2024-09-14T05:30:08.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3029e-05 2024-09-14T05:30:08.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:08.422296+0000", 2024-09-14T05:30:08.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2450000000000001e-06 2024-09-14T05:30:08.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:08.422350+0000", 2024-09-14T05:30:08.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3872000000000002e-05 2024-09-14T05:30:08.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:08.422400+0000", 2024-09-14T05:30:08.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0708999999999998e-05 2024-09-14T05:30:08.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:08.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:08.422515+0000", 2024-09-14T05:30:08.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011512 2024-09-14T05:30:08.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:08.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:08.423236+0000", 2024-09-14T05:30:08.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072088999999999996 2024-09-14T05:30:08.429 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.429 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.430 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.430 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.430 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:30:08.430 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 1 2024-09-14T05:30:08.430 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:08.430 INFO:teuthology.orchestra.run.smithi071.stdout:3664 write 0xa544d28 thru 0xa549073 (0x434c bytes) 2024-09-14T05:30:08.436 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:30:08.445 INFO:teuthology.orchestra.run.smithi071.stdout:3665 trunc from 0xa549074 to 0x8f88e8 2024-09-14T05:30:08.486 INFO:teuthology.orchestra.run.smithi071.stdout:3666 writesame 0xce6099c thru 0xce6f165 data_size 0x1(0xe7ca bytes) 2024-09-14T05:30:08.486 INFO:teuthology.orchestra.run.smithi071.stdout:3668 read 0x950607e thru 0x9509865 (0x37e8 bytes) 2024-09-14T05:30:08.486 INFO:teuthology.orchestra.run.smithi071.stdout:3669 read 0x529f4e7 thru 0x52a60a1 (0x6bbb bytes) 2024-09-14T05:30:08.486 INFO:teuthology.orchestra.run.smithi071.stdout:3670 read 0x830f66 thru 0x83adcb (0x9e66 bytes) 2024-09-14T05:30:08.487 INFO:teuthology.orchestra.run.smithi071.stdout:3671 write 0x31ce1e9 thru 0x31d6414 (0x822c bytes) 2024-09-14T05:30:08.491 INFO:teuthology.orchestra.run.smithi071.stdout:3672 read 0xf7d40d thru 0xf86bde (0x97d2 bytes) 2024-09-14T05:30:08.491 INFO:teuthology.orchestra.run.smithi071.stdout:3673 writesame 0x8102bd4 thru 0x8107fd6 data_size 0x3(0x5403 bytes) 2024-09-14T05:30:08.496 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:08.496+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:08.497 INFO:teuthology.orchestra.run.smithi071.stdout:3674 punch from 0xb3d4820 to 0xb3dd805, (0x8fe5 bytes) 2024-09-14T05:30:08.497 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.497 INFO:teuthology.orchestra.run.smithi071.stdout:3676 write 0xa6634c6 thru 0xa6647ec (0x1327 bytes) 2024-09-14T05:30:08.502 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.502 INFO:teuthology.orchestra.run.smithi071.stdout:3678 punch from 0x949e453 to 0x94a15d9, (0x3186 bytes) 2024-09-14T05:30:08.502 INFO:teuthology.orchestra.run.smithi071.stdout:3679 writesame 0xb611f39 thru 0xb6200c8 data_size 0x48(0xe190 bytes) 2024-09-14T05:30:08.511 INFO:teuthology.orchestra.run.smithi071.stdout:3681 read 0xa5fcfac thru 0xa606d40 (0x9d95 bytes) 2024-09-14T05:30:08.511 INFO:teuthology.orchestra.run.smithi071.stdout:3682 writesame 0xe1617b thru 0xe1f393 data_size 0x5b(0x9219 bytes) 2024-09-14T05:30:08.516 INFO:teuthology.orchestra.run.smithi071.stdout:3683 trunc from 0xce6f166 to 0xcec3de8 2024-09-14T05:30:08.521 INFO:teuthology.orchestra.run.smithi071.stdout:3684 write 0x59bd2bf thru 0x59c13c6 (0x4108 bytes) 2024-09-14T05:30:08.525 INFO:teuthology.orchestra.run.smithi071.stdout:3685 writesame 0xccb408a thru 0xccb8275 data_size 0x2(0x41ec bytes) 2024-09-14T05:30:08.531 INFO:teuthology.orchestra.run.smithi071.stdout:3686 read 0x9b544 thru 0x9b815 (0x2d2 bytes) 2024-09-14T05:30:08.536 INFO:teuthology.orchestra.run.smithi071.stdout:3687 trunc from 0xcec3de8 to 0x9880e97 2024-09-14T05:30:08.558 INFO:teuthology.orchestra.run.smithi071.stdout:3688 writesame 0xc3744ea thru 0xc384051 data_size 0x14(0xfb68 bytes) 2024-09-14T05:30:08.558 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.558 INFO:teuthology.orchestra.run.smithi071.stdout:3692 punch from 0x20586c to 0x205ea9, (0x63d bytes) 2024-09-14T05:30:08.573 INFO:teuthology.orchestra.run.smithi071.stdout:3693 trunc from 0xc384052 to 0xba5574 2024-09-14T05:30:08.590 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:08.590 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:08.590 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:08.590 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:08.590 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:08.596 INFO:teuthology.orchestra.run.smithi071.stdout:3694 write 0x3f36899 thru 0x3f3f2b7 (0x8a1f bytes) 2024-09-14T05:30:08.596 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:08.596+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:08.600 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:30:08.604 INFO:teuthology.orchestra.run.smithi071.stdout:3696 read 0x2ff1936 thru 0x2ffaf8f (0x965a bytes) 2024-09-14T05:30:08.608 INFO:teuthology.orchestra.run.smithi071.stdout:3697 trunc from 0x3f3f2b8 to 0xcef2a1 2024-09-14T05:30:08.621 INFO:teuthology.orchestra.run.smithi071.stdout:3700 read 0x2fec28 thru 0x30716c (0x8545 bytes) 2024-09-14T05:30:08.622 INFO:teuthology.orchestra.run.smithi071.stdout:3701 read 0x78926e thru 0x78a503 (0x1296 bytes) 2024-09-14T05:30:08.631 INFO:teuthology.orchestra.run.smithi071.stdout:3702 write 0x190a43d thru 0x19166b9 (0xc27d bytes) 2024-09-14T05:30:08.644 INFO:teuthology.orchestra.run.smithi071.stdout:3703 punch from 0x552e4e to 0x555763, (0x2915 bytes) 2024-09-14T05:30:08.644 INFO:teuthology.orchestra.run.smithi071.stdout:3704 punch from 0x403e56 to 0x404389, (0x533 bytes) 2024-09-14T05:30:08.664 INFO:teuthology.orchestra.run.smithi071.stdout:3705 writesame 0xcd7b1ea thru 0xcd89975 data_size 0x74(0xe78c bytes) 2024-09-14T05:30:08.664 INFO:teuthology.orchestra.run.smithi071.stdout:3707 write 0x187181b thru 0x187dfd7 (0xc7bd bytes) 2024-09-14T05:30:08.675 INFO:teuthology.orchestra.run.smithi071.stdout:3708 trunc from 0xcd89976 to 0xc22c4a9 2024-09-14T05:30:08.684 INFO:teuthology.orchestra.run.smithi071.stdout:3709 trunc from 0xc22c4a9 to 0x95ad54f 2024-09-14T05:30:08.690 INFO:teuthology.orchestra.run.smithi071.stdout:3711 read 0x26e88f7 thru 0x26f37c8 (0xaed2 bytes) 2024-09-14T05:30:08.690 INFO:teuthology.orchestra.run.smithi071.stdout:3712 punch from 0x37dabfe to 0x37e76f6, (0xcaf8 bytes) 2024-09-14T05:30:08.690 INFO:teuthology.orchestra.run.smithi071.stdout:3715 read 0x3bf2f59 thru 0x3c001f7 (0xd29f bytes) 2024-09-14T05:30:08.690 INFO:teuthology.orchestra.run.smithi071.stdout:3717 read 0x69d8fb4 thru 0x69decaf (0x5cfc bytes) 2024-09-14T05:30:08.690 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.690 INFO:teuthology.orchestra.run.smithi071.stdout:3720 punch from 0x7501090 to 0x7502753, (0x16c3 bytes) 2024-09-14T05:30:08.690 INFO:teuthology.orchestra.run.smithi071.stdout:3721 punch from 0x2d400fe to 0x2d4d890, (0xd792 bytes) 2024-09-14T05:30:08.690 INFO:teuthology.orchestra.run.smithi071.stdout:3722 write 0x2726639 thru 0x2732dc5 (0xc78d bytes) 2024-09-14T05:30:08.695 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.695 INFO:teuthology.orchestra.run.smithi071.stdout:3724 writesame 0x5789707 thru 0x579282d data_size 0x1(0x9127 bytes) 2024-09-14T05:30:08.697 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:08.696+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:08.716 INFO:teuthology.orchestra.run.smithi071.stdout:3726 writesame 0xddd8cdc thru 0xdde604f data_size 0x4e(0xd374 bytes) 2024-09-14T05:30:08.726 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:08.726 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:08.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:08.726 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:08.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.726 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:30:08.726 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:30:08.727 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.947966657999999, 2024-09-14T05:30:08.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.24562984099999999, 2024-09-14T05:30:08.727 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.727 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.727 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:08.727 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.727 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:30:08.727 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:30:08.727 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1 2024-09-14T05:30:08.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.727 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:30:08.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:30:08.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776565+0000", 2024-09-14T05:30:08.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4780000000000001e-06 2024-09-14T05:30:08.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.729 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.729 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776571+0000", 2024-09-14T05:30:08.729 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7669999999999998e-06 2024-09-14T05:30:08.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.729 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.729 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776580+0000", 2024-09-14T05:30:08.729 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9830000000000002e-06 2024-09-14T05:30:08.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776589+0000", 2024-09-14T05:30:08.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7579999999999998e-06 2024-09-14T05:30:08.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776662+0000", 2024-09-14T05:30:08.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3332999999999997e-05 2024-09-14T05:30:08.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:30:08.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776670+0000", 2024-09-14T05:30:08.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9850000000000006e-06 2024-09-14T05:30:08.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.020867+0000", 2024-09-14T05:30:08.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.244196265 2024-09-14T05:30:08.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021022+0000", 2024-09-14T05:30:08.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000155699 2024-09-14T05:30:08.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.732 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:08.732 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021169+0000", 2024-09-14T05:30:08.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014661700000000001 2024-09-14T05:30:08.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.732 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:30:08.732 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021891+0000", 2024-09-14T05:30:08.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072146700000000005 2024-09-14T05:30:08.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.732 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:08.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022124+0000", 2024-09-14T05:30:08.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023354199999999999 2024-09-14T05:30:08.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:30:08.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022164+0000", 2024-09-14T05:30:08.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9897000000000001e-05 2024-09-14T05:30:08.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022193+0000", 2024-09-14T05:30:08.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9050000000000001e-05 2024-09-14T05:30:08.733 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.734 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.734 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.734 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:1989 3.1f e30/24 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head v 30'29, mlcod=30'27)", 2024-09-14T05:30:08.734 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:30:08.734 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 38.131195671, 2024-09-14T05:30:08.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.064242492999999998, 2024-09-14T05:30:08.734 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.734 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.734 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:08.734 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:30:08.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:30:08.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593336+0000", 2024-09-14T05:30:08.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3909999999999999e-06 2024-09-14T05:30:08.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616632+0000", 2024-09-14T05:30:08.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.023296598000000002 2024-09-14T05:30:08.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616657+0000", 2024-09-14T05:30:08.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5228e-05 2024-09-14T05:30:08.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616684+0000", 2024-09-14T05:30:08.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6441999999999999e-05 2024-09-14T05:30:08.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616755+0000", 2024-09-14T05:30:08.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1452000000000006e-05 2024-09-14T05:30:08.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616780+0000", 2024-09-14T05:30:08.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4436000000000001e-05 2024-09-14T05:30:08.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:30:08.738 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657527+0000", 2024-09-14T05:30:08.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.040747323000000002 2024-09-14T05:30:08.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.738 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.738 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657577+0000", 2024-09-14T05:30:08.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9623000000000002e-05 2024-09-14T05:30:08.738 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.738 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.738 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.738 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70397 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4958208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.739 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.370202+0000", 2024-09-14T05:30:08.739 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.354327954999999, 2024-09-14T05:30:08.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030143445000000001, 2024-09-14T05:30:08.739 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.739 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.739 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.739 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.739 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.739 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.739 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70397 2024-09-14T05:30:08.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.739 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370202+0000", 2024-09-14T05:30:08.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.740 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.740 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370202+0000", 2024-09-14T05:30:08.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.740 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.740 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370202+0000", 2024-09-14T05:30:08.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3000000000000002e-07 2024-09-14T05:30:08.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.741 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.741 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370204+0000", 2024-09-14T05:30:08.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7799999999999999e-06 2024-09-14T05:30:08.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.741 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.741 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370209+0000", 2024-09-14T05:30:08.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.169e-06 2024-09-14T05:30:08.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.741 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.741 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370214+0000", 2024-09-14T05:30:08.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2299999999999999e-06 2024-09-14T05:30:08.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.742 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400201+0000", 2024-09-14T05:30:08.742 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029986987999999999 2024-09-14T05:30:08.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.742 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400266+0000", 2024-09-14T05:30:08.742 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5061999999999995e-05 2024-09-14T05:30:08.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400345+0000", 2024-09-14T05:30:08.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8886000000000003e-05 2024-09-14T05:30:08.743 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.743 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.743 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.743 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70400 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4990976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.743 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.370229+0000", 2024-09-14T05:30:08.743 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.354300645999999, 2024-09-14T05:30:08.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030287287, 2024-09-14T05:30:08.743 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.743 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.744 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.744 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.744 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.744 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.744 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70400 2024-09-14T05:30:08.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.744 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.744 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370229+0000", 2024-09-14T05:30:08.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.745 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370229+0000", 2024-09-14T05:30:08.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.745 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370230+0000", 2024-09-14T05:30:08.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3000000000000001e-07 2024-09-14T05:30:08.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.746 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.746 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370232+0000", 2024-09-14T05:30:08.746 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0209999999999998e-06 2024-09-14T05:30:08.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.746 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.746 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370238+0000", 2024-09-14T05:30:08.746 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9340000000000003e-06 2024-09-14T05:30:08.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.747 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.747 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370242+0000", 2024-09-14T05:30:08.747 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.566e-06 2024-09-14T05:30:08.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.747 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.747 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400373+0000", 2024-09-14T05:30:08.747 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030131116999999999 2024-09-14T05:30:08.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.747 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.747 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400439+0000", 2024-09-14T05:30:08.747 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6038999999999996e-05 2024-09-14T05:30:08.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.748 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.748 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400516+0000", 2024-09-14T05:30:08.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7180000000000003e-05 2024-09-14T05:30:08.748 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.748 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.748 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.748 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70413 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4960256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.748 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.370249+0000", 2024-09-14T05:30:08.748 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.354280795000001, 2024-09-14T05:30:08.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030421191, 2024-09-14T05:30:08.749 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.749 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.749 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.749 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.749 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.749 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.749 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70413 2024-09-14T05:30:08.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.749 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370249+0000", 2024-09-14T05:30:08.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370249+0000", 2024-09-14T05:30:08.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370249+0000", 2024-09-14T05:30:08.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9400000000000001e-07 2024-09-14T05:30:08.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370251+0000", 2024-09-14T05:30:08.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.013e-06 2024-09-14T05:30:08.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.751 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.751 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370258+0000", 2024-09-14T05:30:08.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0809999999999999e-06 2024-09-14T05:30:08.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.751 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.751 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370262+0000", 2024-09-14T05:30:08.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7230000000000004e-06 2024-09-14T05:30:08.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.751 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400543+0000", 2024-09-14T05:30:08.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030281129 2024-09-14T05:30:08.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400593+0000", 2024-09-14T05:30:08.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9645000000000003e-05 2024-09-14T05:30:08.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400670+0000", 2024-09-14T05:30:08.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7206000000000006e-05 2024-09-14T05:30:08.753 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.753 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.753 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.753 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70416 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4993024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.753 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.370268+0000", 2024-09-14T05:30:08.753 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.354262015, 2024-09-14T05:30:08.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030571867999999999, 2024-09-14T05:30:08.753 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.753 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.754 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.754 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.754 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.754 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.754 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70416 2024-09-14T05:30:08.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.754 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370268+0000", 2024-09-14T05:30:08.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.755 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370268+0000", 2024-09-14T05:30:08.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.755 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370268+0000", 2024-09-14T05:30:08.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7e-07 2024-09-14T05:30:08.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.755 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370270+0000", 2024-09-14T05:30:08.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8279999999999999e-06 2024-09-14T05:30:08.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.756 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.756 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370275+0000", 2024-09-14T05:30:08.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4110000000000004e-06 2024-09-14T05:30:08.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.756 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.756 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370280+0000", 2024-09-14T05:30:08.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6940000000000003e-06 2024-09-14T05:30:08.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.756 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.756 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400698+0000", 2024-09-14T05:30:08.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030418207999999999 2024-09-14T05:30:08.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400764+0000", 2024-09-14T05:30:08.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5717999999999997e-05 2024-09-14T05:30:08.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.757 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.757 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400840+0000", 2024-09-14T05:30:08.757 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5638999999999999e-05 2024-09-14T05:30:08.757 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.757 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.758 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.758 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70429 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4962304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.758 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.370286+0000", 2024-09-14T05:30:08.758 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.354244242, 2024-09-14T05:30:08.758 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030741230000000001, 2024-09-14T05:30:08.758 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.758 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.758 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.758 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.758 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.758 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.758 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70429 2024-09-14T05:30:08.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.759 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370286+0000", 2024-09-14T05:30:08.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370286+0000", 2024-09-14T05:30:08.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370286+0000", 2024-09-14T05:30:08.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5600000000000001e-07 2024-09-14T05:30:08.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370301+0000", 2024-09-14T05:30:08.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5268999999999999e-05 2024-09-14T05:30:08.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370308+0000", 2024-09-14T05:30:08.761 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3430000000000001e-06 2024-09-14T05:30:08.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.761 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.761 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370313+0000", 2024-09-14T05:30:08.761 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0579999999999997e-06 2024-09-14T05:30:08.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.761 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.761 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400868+0000", 2024-09-14T05:30:08.761 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030555045999999999 2024-09-14T05:30:08.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.761 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400940+0000", 2024-09-14T05:30:08.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2304000000000004e-05 2024-09-14T05:30:08.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401027+0000", 2024-09-14T05:30:08.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6854000000000006e-05 2024-09-14T05:30:08.762 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.762 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.762 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.763 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70432 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4995072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.763 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.370319+0000", 2024-09-14T05:30:08.763 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.354210351999999, 2024-09-14T05:30:08.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030873629, 2024-09-14T05:30:08.763 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.763 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.763 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.763 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.763 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.763 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.763 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70432 2024-09-14T05:30:08.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.763 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370319+0000", 2024-09-14T05:30:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370319+0000", 2024-09-14T05:30:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370320+0000", 2024-09-14T05:30:08.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2399999999999999e-07 2024-09-14T05:30:08.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370322+0000", 2024-09-14T05:30:08.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.824e-06 2024-09-14T05:30:08.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370327+0000", 2024-09-14T05:30:08.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6219999999999997e-06 2024-09-14T05:30:08.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370332+0000", 2024-09-14T05:30:08.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3080000000000001e-06 2024-09-14T05:30:08.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401054+0000", 2024-09-14T05:30:08.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030722791999999999 2024-09-14T05:30:08.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401120+0000", 2024-09-14T05:30:08.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5309999999999996e-05 2024-09-14T05:30:08.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401193+0000", 2024-09-14T05:30:08.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3349000000000005e-05 2024-09-14T05:30:08.767 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.767 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.767 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.767 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70446 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4947968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.767 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.371258+0000", 2024-09-14T05:30:08.767 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.353271589999999, 2024-09-14T05:30:08.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030101513999999999, 2024-09-14T05:30:08.768 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.768 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.768 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.768 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.768 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.768 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.768 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70446 2024-09-14T05:30:08.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.768 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371258+0000", 2024-09-14T05:30:08.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.769 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.769 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371258+0000", 2024-09-14T05:30:08.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.769 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.769 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371259+0000", 2024-09-14T05:30:08.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2200000000000003e-07 2024-09-14T05:30:08.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.769 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.769 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371263+0000", 2024-09-14T05:30:08.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8940000000000003e-06 2024-09-14T05:30:08.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.770 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.770 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371272+0000", 2024-09-14T05:30:08.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6449999999999994e-06 2024-09-14T05:30:08.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.770 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.770 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371278+0000", 2024-09-14T05:30:08.770 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6660000000000002e-06 2024-09-14T05:30:08.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.770 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401221+0000", 2024-09-14T05:30:08.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029942368 2024-09-14T05:30:08.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401285+0000", 2024-09-14T05:30:08.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4710999999999998e-05 2024-09-14T05:30:08.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.771 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.771 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401360+0000", 2024-09-14T05:30:08.771 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4407999999999996e-05 2024-09-14T05:30:08.771 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.772 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.772 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.772 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70448 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4980736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.772 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.371289+0000", 2024-09-14T05:30:08.772 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.353240381999999, 2024-09-14T05:30:08.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030246810999999998, 2024-09-14T05:30:08.772 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.772 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.772 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.772 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.772 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.773 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.773 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70448 2024-09-14T05:30:08.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.773 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371289+0000", 2024-09-14T05:30:08.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371289+0000", 2024-09-14T05:30:08.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.774 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371290+0000", 2024-09-14T05:30:08.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2800000000000002e-07 2024-09-14T05:30:08.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.774 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371292+0000", 2024-09-14T05:30:08.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0049999999999999e-06 2024-09-14T05:30:08.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371297+0000", 2024-09-14T05:30:08.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6250000000000004e-06 2024-09-14T05:30:08.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371302+0000", 2024-09-14T05:30:08.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9960000000000001e-06 2024-09-14T05:30:08.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401387+0000", 2024-09-14T05:30:08.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030084818999999999 2024-09-14T05:30:08.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401451+0000", 2024-09-14T05:30:08.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3331999999999996e-05 2024-09-14T05:30:08.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401536+0000", 2024-09-14T05:30:08.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5606000000000007e-05 2024-09-14T05:30:08.776 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.777 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.777 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.777 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70461 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4950016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.777 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.371310+0000", 2024-09-14T05:30:08.777 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.35322, 2024-09-14T05:30:08.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030393710000000001, 2024-09-14T05:30:08.777 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.777 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.777 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.777 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.777 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.777 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.778 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70461 2024-09-14T05:30:08.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.778 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.778 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.778 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371310+0000", 2024-09-14T05:30:08.778 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371310+0000", 2024-09-14T05:30:08.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371310+0000", 2024-09-14T05:30:08.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4000000000000002e-07 2024-09-14T05:30:08.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.779 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.779 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371312+0000", 2024-09-14T05:30:08.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.04e-06 2024-09-14T05:30:08.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.780 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371318+0000", 2024-09-14T05:30:08.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7089999999999999e-06 2024-09-14T05:30:08.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.780 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.780 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371323+0000", 2024-09-14T05:30:08.780 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5820000000000003e-06 2024-09-14T05:30:08.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401563+0000", 2024-09-14T05:30:08.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030240466000000001 2024-09-14T05:30:08.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401629+0000", 2024-09-14T05:30:08.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5631999999999998e-05 2024-09-14T05:30:08.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401704+0000", 2024-09-14T05:30:08.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4840999999999997e-05 2024-09-14T05:30:08.782 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.782 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.782 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.782 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70464 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4982784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.782 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.371328+0000", 2024-09-14T05:30:08.782 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.353201473999999, 2024-09-14T05:30:08.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030558068000000001, 2024-09-14T05:30:08.782 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.782 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.782 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.782 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70464 2024-09-14T05:30:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371328+0000", 2024-09-14T05:30:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371328+0000", 2024-09-14T05:30:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371329+0000", 2024-09-14T05:30:08.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0900000000000002e-07 2024-09-14T05:30:08.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371331+0000", 2024-09-14T05:30:08.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.792e-06 2024-09-14T05:30:08.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371335+0000", 2024-09-14T05:30:08.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9300000000000002e-06 2024-09-14T05:30:08.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371340+0000", 2024-09-14T05:30:08.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.605e-06 2024-09-14T05:30:08.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401730+0000", 2024-09-14T05:30:08.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030390225 2024-09-14T05:30:08.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401800+0000", 2024-09-14T05:30:08.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0166000000000005e-05 2024-09-14T05:30:08.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401886+0000", 2024-09-14T05:30:08.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5940999999999996e-05 2024-09-14T05:30:08.787 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.787 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.788 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.788 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70609 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4972544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.788 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.375484+0000", 2024-09-14T05:30:08.788 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.349045459999999, 2024-09-14T05:30:08.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.031716407000000002, 2024-09-14T05:30:08.788 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.789 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.789 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.789 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.789 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.789 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.789 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70609 2024-09-14T05:30:08.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.789 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375484+0000", 2024-09-14T05:30:08.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.790 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.790 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375484+0000", 2024-09-14T05:30:08.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375485+0000", 2024-09-14T05:30:08.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6100000000000001e-07 2024-09-14T05:30:08.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375487+0000", 2024-09-14T05:30:08.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3700000000000002e-06 2024-09-14T05:30:08.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375494+0000", 2024-09-14T05:30:08.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.737e-06 2024-09-14T05:30:08.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375499+0000", 2024-09-14T05:30:08.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4500000000000003e-06 2024-09-14T05:30:08.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405175+0000", 2024-09-14T05:30:08.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029676022999999999 2024-09-14T05:30:08.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405243+0000", 2024-09-14T05:30:08.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7620000000000006e-05 2024-09-14T05:30:08.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407201+0000", 2024-09-14T05:30:08.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019577459999999998 2024-09-14T05:30:08.793 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.793 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.793 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.794 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70612 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 5005312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.794 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.375507+0000", 2024-09-14T05:30:08.794 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.349023142, 2024-09-14T05:30:08.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030049329999999999, 2024-09-14T05:30:08.794 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.794 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.794 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.794 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.794 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.794 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.794 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70612 2024-09-14T05:30:08.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.794 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375507+0000", 2024-09-14T05:30:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375507+0000", 2024-09-14T05:30:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375507+0000", 2024-09-14T05:30:08.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:08.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375509+0000", 2024-09-14T05:30:08.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.967e-06 2024-09-14T05:30:08.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375516+0000", 2024-09-14T05:30:08.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.691e-06 2024-09-14T05:30:08.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375521+0000", 2024-09-14T05:30:08.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2510000000000002e-06 2024-09-14T05:30:08.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405380+0000", 2024-09-14T05:30:08.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029858787000000001 2024-09-14T05:30:08.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405446+0000", 2024-09-14T05:30:08.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5996999999999998e-05 2024-09-14T05:30:08.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405556+0000", 2024-09-14T05:30:08.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011021100000000001 2024-09-14T05:30:08.799 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.799 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.799 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.800 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70625 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4974592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.800 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.375529+0000", 2024-09-14T05:30:08.800 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.349000996000001, 2024-09-14T05:30:08.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030229536000000001, 2024-09-14T05:30:08.800 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.800 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.801 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.801 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.801 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.801 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.801 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70625 2024-09-14T05:30:08.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.802 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375529+0000", 2024-09-14T05:30:08.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375529+0000", 2024-09-14T05:30:08.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375529+0000", 2024-09-14T05:30:08.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8299999999999998e-07 2024-09-14T05:30:08.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375531+0000", 2024-09-14T05:30:08.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9659999999999999e-06 2024-09-14T05:30:08.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375538+0000", 2024-09-14T05:30:08.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5089999999999999e-06 2024-09-14T05:30:08.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375543+0000", 2024-09-14T05:30:08.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3959999999999997e-06 2024-09-14T05:30:08.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405581+0000", 2024-09-14T05:30:08.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030037828999999999 2024-09-14T05:30:08.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405648+0000", 2024-09-14T05:30:08.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7194999999999996e-05 2024-09-14T05:30:08.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405758+0000", 2024-09-14T05:30:08.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000110258 2024-09-14T05:30:08.806 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.806 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.806 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.806 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70628 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 5007360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.806 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.375620+0000", 2024-09-14T05:30:08.806 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.348909850999998, 2024-09-14T05:30:08.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030354611, 2024-09-14T05:30:08.806 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.807 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.807 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.807 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.807 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.807 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.807 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70628 2024-09-14T05:30:08.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.807 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375620+0000", 2024-09-14T05:30:08.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.808 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.808 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375620+0000", 2024-09-14T05:30:08.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375621+0000", 2024-09-14T05:30:08.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9800000000000003e-07 2024-09-14T05:30:08.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375624+0000", 2024-09-14T05:30:08.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4429999999999999e-06 2024-09-14T05:30:08.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375633+0000", 2024-09-14T05:30:08.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6370000000000005e-06 2024-09-14T05:30:08.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375639+0000", 2024-09-14T05:30:08.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3149999999999997e-06 2024-09-14T05:30:08.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405784+0000", 2024-09-14T05:30:08.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030145110999999999 2024-09-14T05:30:08.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405851+0000", 2024-09-14T05:30:08.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7279000000000004e-05 2024-09-14T05:30:08.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405975+0000", 2024-09-14T05:30:08.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012312799999999999 2024-09-14T05:30:08.812 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.812 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.813 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.813 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70641 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4976640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.813 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.375666+0000", 2024-09-14T05:30:08.813 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.34886427, 2024-09-14T05:30:08.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030512713, 2024-09-14T05:30:08.813 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.813 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.813 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.813 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.813 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.814 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.814 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70641 2024-09-14T05:30:08.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.814 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375666+0000", 2024-09-14T05:30:08.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375666+0000", 2024-09-14T05:30:08.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375666+0000", 2024-09-14T05:30:08.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2500000000000005e-07 2024-09-14T05:30:08.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375669+0000", 2024-09-14T05:30:08.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9220000000000001e-06 2024-09-14T05:30:08.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375677+0000", 2024-09-14T05:30:08.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0809999999999994e-06 2024-09-14T05:30:08.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375683+0000", 2024-09-14T05:30:08.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6069999999999998e-06 2024-09-14T05:30:08.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.816 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.816 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.406000+0000", 2024-09-14T05:30:08.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030316777 2024-09-14T05:30:08.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.817 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.406072+0000", 2024-09-14T05:30:08.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1983000000000005e-05 2024-09-14T05:30:08.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.817 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.406178+0000", 2024-09-14T05:30:08.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000106718 2024-09-14T05:30:08.817 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.817 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.818 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.818 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70644 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 5009408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.818 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.375729+0000", 2024-09-14T05:30:08.818 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.348801041000002, 2024-09-14T05:30:08.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030657856000000001, 2024-09-14T05:30:08.818 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.818 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.818 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.819 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.819 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.819 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.819 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70644 2024-09-14T05:30:08.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.819 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375729+0000", 2024-09-14T05:30:08.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375729+0000", 2024-09-14T05:30:08.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375729+0000", 2024-09-14T05:30:08.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6599999999999996e-07 2024-09-14T05:30:08.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375732+0000", 2024-09-14T05:30:08.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8499999999999998e-06 2024-09-14T05:30:08.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375743+0000", 2024-09-14T05:30:08.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0352e-05 2024-09-14T05:30:08.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375748+0000", 2024-09-14T05:30:08.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7899999999999996e-06 2024-09-14T05:30:08.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.406203+0000", 2024-09-14T05:30:08.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030454108000000001 2024-09-14T05:30:08.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.406272+0000", 2024-09-14T05:30:08.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8979999999999993e-05 2024-09-14T05:30:08.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.406387+0000", 2024-09-14T05:30:08.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011511 2024-09-14T05:30:08.823 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.823 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.823 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.824 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70689 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4966400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.824 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.377471+0000", 2024-09-14T05:30:08.824 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.347058879999999, 2024-09-14T05:30:08.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029995048999999999, 2024-09-14T05:30:08.824 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.825 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.825 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.825 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.825 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.825 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.826 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70689 2024-09-14T05:30:08.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.826 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377471+0000", 2024-09-14T05:30:08.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377471+0000", 2024-09-14T05:30:08.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377471+0000", 2024-09-14T05:30:08.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8099999999999998e-07 2024-09-14T05:30:08.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377474+0000", 2024-09-14T05:30:08.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1780000000000002e-06 2024-09-14T05:30:08.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377480+0000", 2024-09-14T05:30:08.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0769999999999996e-06 2024-09-14T05:30:08.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377484+0000", 2024-09-14T05:30:08.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8040000000000001e-06 2024-09-14T05:30:08.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407257+0000", 2024-09-14T05:30:08.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029773079000000001 2024-09-14T05:30:08.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407324+0000", 2024-09-14T05:30:08.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6721e-05 2024-09-14T05:30:08.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407466+0000", 2024-09-14T05:30:08.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014180900000000001 2024-09-14T05:30:08.829 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.829 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.829 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.830 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70692 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4999168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:08.830 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.377495+0000", 2024-09-14T05:30:08.830 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.347035238, 2024-09-14T05:30:08.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030228279, 2024-09-14T05:30:08.830 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:08.830 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:08.830 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:08.830 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:08.830 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:08.830 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:08.830 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70692 2024-09-14T05:30:08.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.830 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:08.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:08.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377495+0000", 2024-09-14T05:30:08.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:08.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377495+0000", 2024-09-14T05:30:08.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:08.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:08.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377495+0000", 2024-09-14T05:30:08.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7700000000000005e-07 2024-09-14T05:30:08.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:08.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377497+0000", 2024-09-14T05:30:08.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.181e-06 2024-09-14T05:30:08.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:08.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377503+0000", 2024-09-14T05:30:08.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2550000000000003e-06 2024-09-14T05:30:08.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:08.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377508+0000", 2024-09-14T05:30:08.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.967e-06 2024-09-14T05:30:08.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:08.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407517+0000", 2024-09-14T05:30:08.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030008970999999999 2024-09-14T05:30:08.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:08.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407585+0000", 2024-09-14T05:30:08.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7452000000000004e-05 2024-09-14T05:30:08.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:08.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:08.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:08.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407723+0000", 2024-09-14T05:30:08.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013797599999999999 2024-09-14T05:30:08.834 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.834 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.834 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.834 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:08.834 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:08.834 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:08.835 INFO:teuthology.orchestra.run.smithi071.stdout:3727 trunc from 0xdde6050 to 0x5ce91b2 2024-09-14T05:30:08.835 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.836 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:30:08.838 INFO:teuthology.orchestra.run.smithi071.stdout:3731 trunc from 0x5ce91b2 to 0xe1a382b 2024-09-14T05:30:08.838 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.838 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.839 INFO:teuthology.orchestra.run.smithi071.stdout:3735 read 0xc41dc2b thru 0xc427931 (0x9d07 bytes) 2024-09-14T05:30:08.839 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.839 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.839 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.839 INFO:teuthology.orchestra.run.smithi071.stdout:3739 write 0x55d106f thru 0x55da8cc (0x985e bytes) 2024-09-14T05:30:08.839 INFO:teuthology.orchestra.run.smithi071.stdout:3740 read 0xd2b6517 thru 0xd2b87da (0x22c4 bytes) 2024-09-14T05:30:08.839 INFO:teuthology.orchestra.run.smithi071.stdout:3744 punch from 0xcbdde72 to 0xcbebd34, (0xdec2 bytes) 2024-09-14T05:30:08.839 INFO:teuthology.orchestra.run.smithi071.stdout:3745 writesame 0x1b08488 thru 0x1b18400 data_size 0x1(0xff79 bytes) 2024-09-14T05:30:08.839 INFO:teuthology.orchestra.run.smithi071.stdout:3746 write 0x38cb2e8 thru 0x38d80ea (0xce03 bytes) 2024-09-14T05:30:08.839 INFO:teuthology.orchestra.run.smithi071.stdout:3747 read 0x21d627f thru 0x21e5e7c (0xfbfe bytes) 2024-09-14T05:30:08.839 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.839 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.839 INFO:teuthology.orchestra.run.smithi071.stdout:3752 trunc from 0xe1a382b to 0x79a22ac 2024-09-14T05:30:08.840 INFO:teuthology.orchestra.run.smithi071.stdout:3753 punch from 0x33020a3 to 0x33093fb, (0x7358 bytes) 2024-09-14T05:30:08.840 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.840 INFO:teuthology.orchestra.run.smithi071.stdout:3755 writesame 0x5cc0f6d thru 0x5cc401e data_size 0x17(0x30b2 bytes) 2024-09-14T05:30:08.840 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.840 INFO:teuthology.orchestra.run.smithi071.stdout:3757 writesame 0xcb3cdfb thru 0xcb42166 data_size 0x26(0x536c bytes) 2024-09-14T05:30:08.840 INFO:teuthology.orchestra.run.smithi071.stdout:3759 trunc from 0xcb42167 to 0x809c3bc 2024-09-14T05:30:08.840 INFO:teuthology.orchestra.run.smithi071.stdout:3760 write 0x8e8158f thru 0x8e8c50b (0xaf7d bytes) 2024-09-14T05:30:08.840 INFO:teuthology.orchestra.run.smithi071.stdout:3761 flatten 2024-09-14T05:30:08.841 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:08.837+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:08.895 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:08.901 INFO:teuthology.orchestra.run.smithi071.stdout:3763 writesame 0x975a4d6 thru 0x9768268 data_size 0x1(0xdd93 bytes) 2024-09-14T05:30:08.901 INFO:teuthology.orchestra.run.smithi071.stdout:3764 read 0x3f12f50 thru 0x3f1bc2f (0x8ce0 bytes) 2024-09-14T05:30:08.903 INFO:teuthology.orchestra.run.smithi071.stdout:3765 trunc from 0x9768269 to 0xb53bf07 2024-09-14T05:30:08.908 INFO:teuthology.orchestra.run.smithi071.stdout:3766 punch from 0x94e2dbe to 0x94e57de, (0x2a20 bytes) 2024-09-14T05:30:08.908 INFO:teuthology.orchestra.run.smithi071.stdout:3767 read 0xa2a3f20 thru 0xa2ae4a5 (0xa586 bytes) 2024-09-14T05:30:08.908 INFO:teuthology.orchestra.run.smithi071.stdout:3768 write 0x816f730 thru 0x817103d (0x190e bytes) 2024-09-14T05:30:08.915 INFO:teuthology.orchestra.run.smithi071.stdout:3769 write 0x9203466 thru 0x920e8d9 (0xb474 bytes) 2024-09-14T05:30:08.934 INFO:teuthology.orchestra.run.smithi071.stdout:3770 trunc from 0xb53bf07 to 0x10dbef 2024-09-14T05:30:08.939 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:08.938+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:08.960 INFO:teuthology.orchestra.run.smithi071.stdout:3774 read 0x1091d4 thru 0x10dbee (0x4a1b bytes) 2024-09-14T05:30:08.960 INFO:teuthology.orchestra.run.smithi071.stdout:3775 read 0x537e thru 0xd593 (0x8216 bytes) 2024-09-14T05:30:08.976 INFO:teuthology.orchestra.run.smithi071.stdout:3777 trunc from 0x10dbef to 0xc1b13ba 2024-09-14T05:30:08.976 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:08.976 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:08.976 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:08.976 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:08.981 INFO:teuthology.orchestra.run.smithi071.stdout:3778 read 0xb67caf0 thru 0xb67e7ac (0x1cbd bytes) 2024-09-14T05:30:08.987 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:30:08.988 INFO:teuthology.orchestra.run.smithi071.stdout:3780 write 0xe0652a2 thru 0xe06a736 (0x5495 bytes) 2024-09-14T05:30:08.993 INFO:teuthology.orchestra.run.smithi071.stdout:3781 write 0xb1a7fcf thru 0xb1ac323 (0x4355 bytes) 2024-09-14T05:30:08.998 INFO:teuthology.orchestra.run.smithi071.stdout:3782 write 0x19429aa thru 0x194ae73 (0x84ca bytes) 2024-09-14T05:30:09.004 INFO:teuthology.orchestra.run.smithi071.stdout:3783 write 0xca064c9 thru 0xca0c031 (0x5b69 bytes) 2024-09-14T05:30:09.013 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.013 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.013 INFO:teuthology.orchestra.run.smithi071.stdout:3786 read 0x3712441 thru 0x3714aa1 (0x2661 bytes) 2024-09-14T05:30:09.013 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.013 INFO:teuthology.orchestra.run.smithi071.stdout:3788 writesame 0x6d22eb9 thru 0x6d2d705 data_size 0x7(0xa84d bytes) 2024-09-14T05:30:09.029 INFO:teuthology.orchestra.run.smithi071.stdout:3789 write 0xeb53297 thru 0xeb53b04 (0x86e bytes) 2024-09-14T05:30:09.032 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.032 INFO:teuthology.orchestra.run.smithi071.stdout:3791 write 0x3f6b0a5 thru 0x3f6f477 (0x43d3 bytes) 2024-09-14T05:30:09.038 INFO:teuthology.orchestra.run.smithi071.stdout:3792 write 0x843330c thru 0x843c1ad (0x8ea2 bytes) 2024-09-14T05:30:09.040 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:09.039+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:09.047 INFO:teuthology.orchestra.run.smithi071.stdout:3793 read 0x79d991c thru 0x79e83ea (0xeacf bytes) 2024-09-14T05:30:09.048 INFO:teuthology.orchestra.run.smithi071.stdout:3794 writesame 0xd849c65 thru 0xd852d60 data_size 0x9(0x90fc bytes) 2024-09-14T05:30:09.056 INFO:teuthology.orchestra.run.smithi071.stdout:3796 punch from 0x9c90022 to 0x9c9bb89, (0xbb67 bytes) 2024-09-14T05:30:09.056 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.057 INFO:teuthology.orchestra.run.smithi071.stdout:3799 punch from 0x130909e to 0x1312907, (0x9869 bytes) 2024-09-14T05:30:09.057 INFO:teuthology.orchestra.run.smithi071.stdout:3800 write 0xcde6f19 thru 0xcdf0687 (0x976f bytes) 2024-09-14T05:30:09.062 INFO:teuthology.orchestra.run.smithi071.stdout:3801 punch from 0xaae061e to 0xaae45de, (0x3fc0 bytes) 2024-09-14T05:30:09.062 INFO:teuthology.orchestra.run.smithi071.stdout:3802 write 0x550dc8d thru 0x551840e (0xa782 bytes) 2024-09-14T05:30:09.078 INFO:teuthology.orchestra.run.smithi071.stdout:3803 trunc from 0xeb53b05 to 0x7c782ad 2024-09-14T05:30:09.092 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:30:09.101 INFO:teuthology.orchestra.run.smithi071.stdout:3804 trunc from 0x7c782ad to 0x8e785f 2024-09-14T05:30:09.113 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:09.113 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:09.113 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:09.113 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:09.113 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:09.126 INFO:teuthology.orchestra.run.smithi071.stdout:3805 trunc from 0x8e785f to 0xba14c03 2024-09-14T05:30:09.127 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:30:09.131 INFO:teuthology.orchestra.run.smithi071.stdout:3806 writesame 0x2f638c3 thru 0x2f665d8 data_size 0x1d(0x2d16 bytes) 2024-09-14T05:30:09.137 INFO:teuthology.orchestra.run.smithi071.stdout:3807 trunc from 0xba14c03 to 0xd3ff05a 2024-09-14T05:30:09.140 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:09.139+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:09.149 INFO:teuthology.orchestra.run.smithi071.stdout:3808 trunc from 0xd3ff05a to 0x897beb5 2024-09-14T05:30:09.164 INFO:teuthology.orchestra.run.smithi071.stdout:3809 writesame 0xa744a4a thru 0xa74beb4 data_size 0x1(0x746b bytes) 2024-09-14T05:30:09.164 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.164 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.164 INFO:teuthology.orchestra.run.smithi071.stdout:3813 write 0x577656c thru 0x5776876 (0x30b bytes) 2024-09-14T05:30:09.165 INFO:teuthology.orchestra.run.smithi071.stdout:3814 writesame 0xb35d2d thru 0xb3a952 data_size 0x39(0x4c26 bytes) 2024-09-14T05:30:09.172 INFO:teuthology.orchestra.run.smithi071.stdout:3815 write 0xa2027ec thru 0xa208fd2 (0x67e7 bytes) 2024-09-14T05:30:09.178 INFO:teuthology.orchestra.run.smithi071.stdout:3816 write 0xa19e7e0 thru 0xa1a631a (0x7b3b bytes) 2024-09-14T05:30:09.187 INFO:teuthology.orchestra.run.smithi071.stdout:3818 punch from 0x505668a to 0x50606fa, (0xa070 bytes) 2024-09-14T05:30:09.187 INFO:teuthology.orchestra.run.smithi071.stdout:3819 clone 30 order 21 su 8192 sc 3 2024-09-14T05:30:09.234 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:09.234 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:09.241 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:09.240+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:09.246 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T05:30:09.252 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:09.252 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73757 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2816000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:30:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.691048015, 2024-09-14T05:30:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044614540000000001, 2024-09-14T05:30:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73757 2024-09-14T05:30:09.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.254 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.254 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:30:09.254 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.254 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:30:09.254 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.254 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559679+0000", 2024-09-14T05:30:09.255 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4000000000000002e-07 2024-09-14T05:30:09.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.255 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.255 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559681+0000", 2024-09-14T05:30:09.255 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1780000000000002e-06 2024-09-14T05:30:09.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.255 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.255 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559687+0000", 2024-09-14T05:30:09.255 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0270000000000001e-06 2024-09-14T05:30:09.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.255 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.256 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559692+0000", 2024-09-14T05:30:09.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1030000000000001e-06 2024-09-14T05:30:09.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.256 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.256 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604180+0000", 2024-09-14T05:30:09.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044487480000000003 2024-09-14T05:30:09.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.256 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.256 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604224+0000", 2024-09-14T05:30:09.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4357999999999998e-05 2024-09-14T05:30:09.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.257 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604293+0000", 2024-09-14T05:30:09.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8954000000000004e-05 2024-09-14T05:30:09.257 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.257 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.257 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.257 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73760 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2848768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.257 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:09.257 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.691027909999999, 2024-09-14T05:30:09.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.04469911, 2024-09-14T05:30:09.257 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.258 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.258 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.258 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.258 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.258 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.258 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73760 2024-09-14T05:30:09.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.258 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:09.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.259 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.259 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:09.259 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.259 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.259 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:09.259 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:09.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.259 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.259 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559701+0000", 2024-09-14T05:30:09.259 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1909999999999999e-06 2024-09-14T05:30:09.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.260 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559708+0000", 2024-09-14T05:30:09.260 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7279999999999998e-06 2024-09-14T05:30:09.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.260 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559713+0000", 2024-09-14T05:30:09.260 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8389999999999997e-06 2024-09-14T05:30:09.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.260 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604310+0000", 2024-09-14T05:30:09.260 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044597664000000002 2024-09-14T05:30:09.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.261 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.261 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604352+0000", 2024-09-14T05:30:09.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1236000000000003e-05 2024-09-14T05:30:09.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.261 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.261 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604398+0000", 2024-09-14T05:30:09.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6091000000000003e-05 2024-09-14T05:30:09.261 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.261 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.261 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.262 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73773 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2801664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.262 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:09.262 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.691007741, 2024-09-14T05:30:09.262 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941026000000002, 2024-09-14T05:30:09.262 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.262 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.262 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.262 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.262 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.262 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.262 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73773 2024-09-14T05:30:09.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.263 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:09.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:09.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:09.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8599999999999999e-07 2024-09-14T05:30:09.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559721+0000", 2024-09-14T05:30:09.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0540000000000002e-06 2024-09-14T05:30:09.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559729+0000", 2024-09-14T05:30:09.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1359999999999997e-06 2024-09-14T05:30:09.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.265 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.265 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559734+0000", 2024-09-14T05:30:09.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2140000000000004e-06 2024-09-14T05:30:09.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.265 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.265 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604441+0000", 2024-09-14T05:30:09.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044706828999999997 2024-09-14T05:30:09.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.265 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.265 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604522+0000", 2024-09-14T05:30:09.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0603000000000001e-05 2024-09-14T05:30:09.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.266 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604660+0000", 2024-09-14T05:30:09.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000137804 2024-09-14T05:30:09.266 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.266 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.266 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.266 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73777 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2834432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.267 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:09.267 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.690985340000001, 2024-09-14T05:30:09.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045116270999999999, 2024-09-14T05:30:09.267 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.267 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.267 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.267 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.267 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.267 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.268 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73777 2024-09-14T05:30:09.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.268 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:09.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:09.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.269 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.269 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559742+0000", 2024-09-14T05:30:09.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4099999999999999e-07 2024-09-14T05:30:09.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.269 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.269 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559744+0000", 2024-09-14T05:30:09.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0040000000000002e-06 2024-09-14T05:30:09.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.269 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.269 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559749+0000", 2024-09-14T05:30:09.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9739999999999999e-06 2024-09-14T05:30:09.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.270 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.270 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559754+0000", 2024-09-14T05:30:09.270 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9540000000000003e-06 2024-09-14T05:30:09.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.270 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.270 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604695+0000", 2024-09-14T05:30:09.270 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941003 2024-09-14T05:30:09.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.270 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.270 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604766+0000", 2024-09-14T05:30:09.270 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0296999999999995e-05 2024-09-14T05:30:09.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.271 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.271 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604857+0000", 2024-09-14T05:30:09.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1598e-05 2024-09-14T05:30:09.271 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.271 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.271 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.271 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73790 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.271 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:09.271 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.690965405, 2024-09-14T05:30:09.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316437000000001, 2024-09-14T05:30:09.271 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.272 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.272 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.272 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.272 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.272 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.272 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73790 2024-09-14T05:30:09.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.272 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.272 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:09.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:09.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:09.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7599999999999998e-07 2024-09-14T05:30:09.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.274 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.274 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559763+0000", 2024-09-14T05:30:09.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0880000000000002e-06 2024-09-14T05:30:09.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.274 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.274 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559769+0000", 2024-09-14T05:30:09.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6740000000000002e-06 2024-09-14T05:30:09.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.274 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.274 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559774+0000", 2024-09-14T05:30:09.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6110000000000004e-06 2024-09-14T05:30:09.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.275 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.275 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604905+0000", 2024-09-14T05:30:09.275 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045130827999999998 2024-09-14T05:30:09.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.275 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.275 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604982+0000", 2024-09-14T05:30:09.275 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7410999999999995e-05 2024-09-14T05:30:09.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.275 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.275 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605077+0000", 2024-09-14T05:30:09.275 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5449e-05 2024-09-14T05:30:09.276 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.276 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.276 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.276 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73793 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2836480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.276 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:09.276 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.690946200999999, 2024-09-14T05:30:09.276 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045441212000000002, 2024-09-14T05:30:09.276 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.276 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.276 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.276 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.276 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.277 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.277 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73793 2024-09-14T05:30:09.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.277 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:09.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:09.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559781+0000", 2024-09-14T05:30:09.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2899999999999999e-07 2024-09-14T05:30:09.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.279 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559783+0000", 2024-09-14T05:30:09.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.075e-06 2024-09-14T05:30:09.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.279 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.279 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559788+0000", 2024-09-14T05:30:09.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5369999999999997e-06 2024-09-14T05:30:09.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.279 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.279 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559793+0000", 2024-09-14T05:30:09.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:30:09.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.280 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.280 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605109+0000", 2024-09-14T05:30:09.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316229999999999 2024-09-14T05:30:09.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.280 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.280 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605168+0000", 2024-09-14T05:30:09.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8949000000000001e-05 2024-09-14T05:30:09.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.280 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.280 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605221+0000", 2024-09-14T05:30:09.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3260000000000002e-05 2024-09-14T05:30:09.280 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.281 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.281 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.281 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73806 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.281 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:09.281 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.690927575, 2024-09-14T05:30:09.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045526189000000002, 2024-09-14T05:30:09.281 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.281 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.281 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.281 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.281 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.281 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.282 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73806 2024-09-14T05:30:09.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.282 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:09.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:09.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:09.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:30:09.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559801+0000", 2024-09-14T05:30:09.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.995e-06 2024-09-14T05:30:09.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559807+0000", 2024-09-14T05:30:09.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5450000000000003e-06 2024-09-14T05:30:09.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559811+0000", 2024-09-14T05:30:09.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5090000000000004e-06 2024-09-14T05:30:09.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605237+0000", 2024-09-14T05:30:09.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045426133 2024-09-14T05:30:09.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605278+0000", 2024-09-14T05:30:09.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1084999999999997e-05 2024-09-14T05:30:09.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605325+0000", 2024-09-14T05:30:09.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6582999999999998e-05 2024-09-14T05:30:09.285 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.285 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.285 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.285 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73809 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2838528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.285 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:09.285 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.690904069999998, 2024-09-14T05:30:09.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045596062999999999, 2024-09-14T05:30:09.286 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.286 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.286 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.286 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.286 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.286 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.286 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73809 2024-09-14T05:30:09.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.286 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:09.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:09.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559823+0000", 2024-09-14T05:30:09.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.08e-07 2024-09-14T05:30:09.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.288 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.288 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559825+0000", 2024-09-14T05:30:09.288 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.26e-06 2024-09-14T05:30:09.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.288 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.288 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559831+0000", 2024-09-14T05:30:09.288 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6690000000000003e-06 2024-09-14T05:30:09.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.288 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.288 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559835+0000", 2024-09-14T05:30:09.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5120000000000002e-06 2024-09-14T05:30:09.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605342+0000", 2024-09-14T05:30:09.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045506957000000001 2024-09-14T05:30:09.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605375+0000", 2024-09-14T05:30:09.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3250999999999997e-05 2024-09-14T05:30:09.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.290 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605418+0000", 2024-09-14T05:30:09.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3006000000000001e-05 2024-09-14T05:30:09.290 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.290 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.290 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.290 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73822 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2807808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.290 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:09.290 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.690876416999998, 2024-09-14T05:30:09.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045661405000000002, 2024-09-14T05:30:09.290 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.290 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.290 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.291 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.291 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.291 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.291 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73822 2024-09-14T05:30:09.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.291 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.291 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:09.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.292 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:09.292 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.292 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559851+0000", 2024-09-14T05:30:09.292 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4000000000000002e-07 2024-09-14T05:30:09.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.292 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559853+0000", 2024-09-14T05:30:09.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5380000000000001e-06 2024-09-14T05:30:09.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.293 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.293 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559859+0000", 2024-09-14T05:30:09.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0619999999999997e-06 2024-09-14T05:30:09.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.293 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.293 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559864+0000", 2024-09-14T05:30:09.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.977e-06 2024-09-14T05:30:09.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.293 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.294 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605435+0000", 2024-09-14T05:30:09.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045570672 2024-09-14T05:30:09.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.294 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.294 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605467+0000", 2024-09-14T05:30:09.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2486e-05 2024-09-14T05:30:09.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.294 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.294 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605511+0000", 2024-09-14T05:30:09.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4129999999999999e-05 2024-09-14T05:30:09.294 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.294 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.294 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.295 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73825 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2840576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.295 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:09.295 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.690853053000001, 2024-09-14T05:30:09.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045728694, 2024-09-14T05:30:09.295 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.295 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.295 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.295 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.295 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.295 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.295 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73825 2024-09-14T05:30:09.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.296 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:09.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:09.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559874+0000", 2024-09-14T05:30:09.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5400000000000002e-07 2024-09-14T05:30:09.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.297 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.297 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559876+0000", 2024-09-14T05:30:09.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.131e-06 2024-09-14T05:30:09.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.297 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.297 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559888+0000", 2024-09-14T05:30:09.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2216e-05 2024-09-14T05:30:09.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.297 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.298 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559893+0000", 2024-09-14T05:30:09.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6539999999999998e-06 2024-09-14T05:30:09.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.298 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.298 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605526+0000", 2024-09-14T05:30:09.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045633119 2024-09-14T05:30:09.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.298 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.298 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605560+0000", 2024-09-14T05:30:09.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3825999999999998e-05 2024-09-14T05:30:09.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.299 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605602+0000", 2024-09-14T05:30:09.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2293999999999999e-05 2024-09-14T05:30:09.299 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.299 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.299 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.299 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74023 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2863104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.299 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:09.299 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.689667077999999, 2024-09-14T05:30:09.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044635385999999999, 2024-09-14T05:30:09.299 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.299 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.299 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.300 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.300 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.300 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.300 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74023 2024-09-14T05:30:09.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.300 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:09.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:09.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561060+0000", 2024-09-14T05:30:09.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0699999999999998e-07 2024-09-14T05:30:09.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561072+0000", 2024-09-14T05:30:09.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2261e-05 2024-09-14T05:30:09.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561079+0000", 2024-09-14T05:30:09.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8399999999999997e-06 2024-09-14T05:30:09.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561084+0000", 2024-09-14T05:30:09.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7380000000000002e-06 2024-09-14T05:30:09.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605620+0000", 2024-09-14T05:30:09.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044536146999999998 2024-09-14T05:30:09.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.303 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.303 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605653+0000", 2024-09-14T05:30:09.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2969000000000002e-05 2024-09-14T05:30:09.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.303 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.303 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605695+0000", 2024-09-14T05:30:09.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2024000000000003e-05 2024-09-14T05:30:09.303 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.303 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.303 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.304 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74026 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2895872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.304 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:09.304 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.689635616, 2024-09-14T05:30:09.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044692717, 2024-09-14T05:30:09.304 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.304 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.304 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.304 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.304 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.304 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.304 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74026 2024-09-14T05:30:09.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.304 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561093+0000", 2024-09-14T05:30:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-14T05:30:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561099+0000", 2024-09-14T05:30:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0399999999999998e-06 2024-09-14T05:30:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561104+0000", 2024-09-14T05:30:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8180000000000003e-06 2024-09-14T05:30:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605710+0000", 2024-09-14T05:30:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044605657 2024-09-14T05:30:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605742+0000", 2024-09-14T05:30:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1949000000000002e-05 2024-09-14T05:30:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605784+0000", 2024-09-14T05:30:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1884000000000001e-05 2024-09-14T05:30:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74039 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.689509017999999, 2024-09-14T05:30:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044654905000000002, 2024-09-14T05:30:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74039 2024-09-14T05:30:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561218+0000", 2024-09-14T05:30:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4000000000000001e-07 2024-09-14T05:30:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561221+0000", 2024-09-14T05:30:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9730000000000002e-06 2024-09-14T05:30:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561229+0000", 2024-09-14T05:30:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1759999999999993e-06 2024-09-14T05:30:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561235+0000", 2024-09-14T05:30:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0109999999999997e-06 2024-09-14T05:30:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605798+0000", 2024-09-14T05:30:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044563285000000001 2024-09-14T05:30:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605831+0000", 2024-09-14T05:30:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2054999999999997e-05 2024-09-14T05:30:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605872+0000", 2024-09-14T05:30:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1764999999999997e-05 2024-09-14T05:30:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74042 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.689460565000001, 2024-09-14T05:30:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044773234000000002, 2024-09-14T05:30:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74042 2024-09-14T05:30:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4099999999999998e-07 2024-09-14T05:30:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561269+0000", 2024-09-14T05:30:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7930000000000002e-06 2024-09-14T05:30:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561278+0000", 2024-09-14T05:30:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5150000000000006e-06 2024-09-14T05:30:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561284+0000", 2024-09-14T05:30:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2029999999999997e-06 2024-09-14T05:30:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605934+0000", 2024-09-14T05:30:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044650449000000002 2024-09-14T05:30:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605983+0000", 2024-09-14T05:30:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8943999999999998e-05 2024-09-14T05:30:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606039+0000", 2024-09-14T05:30:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5689000000000002e-05 2024-09-14T05:30:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74052 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.689433690000001, 2024-09-14T05:30:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044830474000000002, 2024-09-14T05:30:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74052 2024-09-14T05:30:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1099999999999996e-07 2024-09-14T05:30:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561295+0000", 2024-09-14T05:30:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7770000000000001e-06 2024-09-14T05:30:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561303+0000", 2024-09-14T05:30:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6860000000000006e-06 2024-09-14T05:30:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561308+0000", 2024-09-14T05:30:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.023e-06 2024-09-14T05:30:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605893+0000", 2024-09-14T05:30:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044585458000000001 2024-09-14T05:30:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605933+0000", 2024-09-14T05:30:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9551999999999997e-05 2024-09-14T05:30:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606123+0000", 2024-09-14T05:30:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019046699999999999 2024-09-14T05:30:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74054 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.689411663000001, 2024-09-14T05:30:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093610999999999, 2024-09-14T05:30:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74054 2024-09-14T05:30:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9699999999999996e-07 2024-09-14T05:30:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561317+0000", 2024-09-14T05:30:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9740000000000001e-06 2024-09-14T05:30:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561324+0000", 2024-09-14T05:30:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4590000000000003e-06 2024-09-14T05:30:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561329+0000", 2024-09-14T05:30:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5969999999999999e-06 2024-09-14T05:30:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606142+0000", 2024-09-14T05:30:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044812347000000002 2024-09-14T05:30:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606184+0000", 2024-09-14T05:30:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2441999999999998e-05 2024-09-14T05:30:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606408+0000", 2024-09-14T05:30:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022429500000000001 2024-09-14T05:30:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74056 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.68921696, 2024-09-14T05:30:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045191685000000002, 2024-09-14T05:30:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74056 2024-09-14T05:30:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561510+0000", 2024-09-14T05:30:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2399999999999998e-07 2024-09-14T05:30:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561513+0000", 2024-09-14T05:30:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7219999999999999e-06 2024-09-14T05:30:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561524+0000", 2024-09-14T05:30:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1357e-05 2024-09-14T05:30:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561530+0000", 2024-09-14T05:30:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2890000000000003e-06 2024-09-14T05:30:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606438+0000", 2024-09-14T05:30:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044907326999999997 2024-09-14T05:30:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606520+0000", 2024-09-14T05:30:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1924999999999999e-05 2024-09-14T05:30:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606701+0000", 2024-09-14T05:30:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00018144100000000001 2024-09-14T05:30:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74058 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.688738287, 2024-09-14T05:30:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093368000000002, 2024-09-14T05:30:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74058 2024-09-14T05:30:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561989+0000", 2024-09-14T05:30:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3300000000000001e-07 2024-09-14T05:30:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561993+0000", 2024-09-14T05:30:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1790000000000001e-06 2024-09-14T05:30:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562007+0000", 2024-09-14T05:30:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3434e-05 2024-09-14T05:30:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562014+0000", 2024-09-14T05:30:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.537e-06 2024-09-14T05:30:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606736+0000", 2024-09-14T05:30:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044722117999999998 2024-09-14T05:30:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606829+0000", 2024-09-14T05:30:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2410999999999993e-05 2024-09-14T05:30:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607082+0000", 2024-09-14T05:30:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025285599999999998 2024-09-14T05:30:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74059 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.688680296000001, 2024-09-14T05:30:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045479928000000003, 2024-09-14T05:30:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74059 2024-09-14T05:30:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562047+0000", 2024-09-14T05:30:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6899999999999997e-07 2024-09-14T05:30:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562050+0000", 2024-09-14T05:30:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.943e-06 2024-09-14T05:30:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562058+0000", 2024-09-14T05:30:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0120000000000005e-06 2024-09-14T05:30:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562063+0000", 2024-09-14T05:30:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:30:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607186+0000", 2024-09-14T05:30:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045122348999999999 2024-09-14T05:30:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607265+0000", 2024-09-14T05:30:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9462999999999995e-05 2024-09-14T05:30:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607526+0000", 2024-09-14T05:30:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026090400000000001 2024-09-14T05:30:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74060 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2850816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.688654919000001, 2024-09-14T05:30:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045200092999999997, 2024-09-14T05:30:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74060 2024-09-14T05:30:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5200000000000002e-07 2024-09-14T05:30:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562075+0000", 2024-09-14T05:30:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5200000000000002e-06 2024-09-14T05:30:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562082+0000", 2024-09-14T05:30:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8929999999999999e-06 2024-09-14T05:30:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562088+0000", 2024-09-14T05:30:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5570000000000003e-06 2024-09-14T05:30:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607111+0000", 2024-09-14T05:30:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045023464999999999 2024-09-14T05:30:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607190+0000", 2024-09-14T05:30:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8721999999999996e-05 2024-09-14T05:30:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607272+0000", 2024-09-14T05:30:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1483999999999994e-05 2024-09-14T05:30:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:09.343 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:09.344 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:09.345 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:09.345 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T05:30:09.346 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:30:09.347 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:09.347 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:09.347 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:09.343+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:09.415 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-14T05:30:09.417 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.417 INFO:teuthology.orchestra.run.smithi071.stdout:3821 punch from 0x8da13b0 to 0x8dad881, (0xc4d1 bytes) 2024-09-14T05:30:09.420 INFO:teuthology.orchestra.run.smithi071.stdout:3822 punch from 0x1bf4caf to 0x1bfb25a, (0x65ab bytes) 2024-09-14T05:30:09.420 INFO:teuthology.orchestra.run.smithi071.stdout:3824 punch from 0x85e4f6b to 0x85ecaa1, (0x7b36 bytes) 2024-09-14T05:30:09.427 INFO:teuthology.orchestra.run.smithi071.stdout:3825 write 0xbaaed7b thru 0xbabbcfd (0xcf83 bytes) 2024-09-14T05:30:09.432 INFO:teuthology.orchestra.run.smithi071.stdout:3826 punch from 0x2441eb0 to 0x2451c16, (0xfd66 bytes) 2024-09-14T05:30:09.434 INFO:teuthology.orchestra.run.smithi071.stdout:3831 trunc from 0xbabbcfe to 0xd61c201 2024-09-14T05:30:09.440 INFO:teuthology.orchestra.run.smithi071.stdout:3833 punch from 0x3c128a4 to 0x3c19673, (0x6dcf bytes) 2024-09-14T05:30:09.440 INFO:teuthology.orchestra.run.smithi071.stdout:3834 writesame 0xaee6dde thru 0xaef6414 data_size 0x1(0xf637 bytes) 2024-09-14T05:30:09.445 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:09.444+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:09.458 INFO:teuthology.orchestra.run.smithi071.stdout:3835 read 0x5646e5d thru 0x564edb6 (0x7f5a bytes) 2024-09-14T05:30:09.459 INFO:teuthology.orchestra.run.smithi071.stdout:3836 read 0xcbe5b66 thru 0xcbe9059 (0x34f4 bytes) 2024-09-14T05:30:09.459 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.459 INFO:teuthology.orchestra.run.smithi071.stdout:3838 punch from 0x4c3cf27 to 0x4c4068a, (0x3763 bytes) 2024-09-14T05:30:09.459 INFO:teuthology.orchestra.run.smithi071.stdout:3839 read 0xc7d6a70 thru 0xc7d73ef (0x980 bytes) 2024-09-14T05:30:09.459 INFO:teuthology.orchestra.run.smithi071.stdout:3840 read 0xc5ac95c thru 0xc5aee70 (0x2515 bytes) 2024-09-14T05:30:09.459 INFO:teuthology.orchestra.run.smithi071.stdout:3842 punch from 0x2fb43f to 0x2fdeb7, (0x2a78 bytes) 2024-09-14T05:30:09.459 INFO:teuthology.orchestra.run.smithi071.stdout:3843 read 0x5d70f84 thru 0x5d7197c (0x9f9 bytes) 2024-09-14T05:30:09.469 INFO:teuthology.orchestra.run.smithi071.stdout:3844 trunc from 0xd61c201 to 0x4d52e27 2024-09-14T05:30:09.492 INFO:teuthology.orchestra.run.smithi071.stdout:3845 trunc from 0x4d52e27 to 0x53c1d0b 2024-09-14T05:30:09.495 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:09.495 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:09.498 INFO:teuthology.orchestra.run.smithi071.stdout:3846 punch from 0x15802e3 to 0x158b5ef, (0xb30c bytes) 2024-09-14T05:30:09.498 INFO:teuthology.orchestra.run.smithi071.stdout:3847 read 0x18f8408 thru 0x1903704 (0xb2fd bytes) 2024-09-14T05:30:09.498 INFO:teuthology.orchestra.run.smithi071.stdout:3849 trunc from 0x53c1d0b to 0x50912d9 2024-09-14T05:30:09.508 INFO:teuthology.orchestra.run.smithi071.stdout:3851 trunc from 0x50912d9 to 0x7cd6e58 2024-09-14T05:30:09.511 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:09.511 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:09.520 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:09.521 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:09.522 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:30:09.529 INFO:teuthology.orchestra.run.smithi071.stdout:3852 writesame 0xec7196d thru 0xec750e2 data_size 0x2(0x3776 bytes) 2024-09-14T05:30:09.529 INFO:teuthology.orchestra.run.smithi071.stdout:3853 read 0xd873d81 thru 0xd87d1bb (0x943b bytes) 2024-09-14T05:30:09.530 INFO:teuthology.orchestra.run.smithi071.stdout:3855 write 0xaf5b334 thru 0xaf5e20d (0x2eda bytes) 2024-09-14T05:30:09.533 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.533 INFO:teuthology.orchestra.run.smithi071.stdout:3858 read 0x2317133 thru 0x231eb5a (0x7a28 bytes) 2024-09-14T05:30:09.533 INFO:teuthology.orchestra.run.smithi071.stdout:3859 write 0x1e9d201 thru 0x1ea4cd1 (0x7ad1 bytes) 2024-09-14T05:30:09.538 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.538 INFO:teuthology.orchestra.run.smithi071.stdout:3861 read 0x9d2e04d thru 0x9d2fbf8 (0x1bac bytes) 2024-09-14T05:30:09.538 INFO:teuthology.orchestra.run.smithi071.stdout:3864 punch from 0x7441d95 to 0x7442d83, (0xfee bytes) 2024-09-14T05:30:09.538 INFO:teuthology.orchestra.run.smithi071.stdout:3865 write 0x117feba thru 0x118c82d (0xc974 bytes) 2024-09-14T05:30:09.541 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:09.542 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:09.546 INFO:teuthology.orchestra.run.smithi071.stdout:3866 write 0x64a7fc0 thru 0x64ae8b8 (0x68f9 bytes) 2024-09-14T05:30:09.546 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:09.544+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:09.555 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.555 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.555 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.555 INFO:teuthology.orchestra.run.smithi071.stdout:3871 write 0x549df11 thru 0x549fea9 (0x1f99 bytes) 2024-09-14T05:30:09.561 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:09.561 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:09.562 INFO:teuthology.orchestra.run.smithi071.stdout:3873 read 0x1ce0d68 thru 0x1cf0bf2 (0xfe8b bytes) 2024-09-14T05:30:09.562 INFO:teuthology.orchestra.run.smithi071.stdout:3874 punch from 0x93a9d73 to 0x93ad2d5, (0x3562 bytes) 2024-09-14T05:30:09.563 INFO:teuthology.orchestra.run.smithi071.stdout:3877 punch from 0x85cbe3f to 0x85d2b16, (0x6cd7 bytes) 2024-09-14T05:30:09.563 INFO:teuthology.orchestra.run.smithi071.stdout:3879 write 0x6356cb2 thru 0x635ea80 (0x7dcf bytes) 2024-09-14T05:30:09.566 INFO:teuthology.orchestra.run.smithi071.stdout:3880 trunc from 0xec750e3 to 0xcdea6af 2024-09-14T05:30:09.572 INFO:teuthology.orchestra.run.smithi071.stdout:3881 read 0xa5873b1 thru 0xa590331 (0x8f81 bytes) 2024-09-14T05:30:09.580 INFO:teuthology.orchestra.run.smithi071.stdout:3884 write 0xe412569 thru 0xe41416d (0x1c05 bytes) 2024-09-14T05:30:09.582 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:09.582 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:09.593 INFO:teuthology.orchestra.run.smithi071.stdout:3885 write 0xe982ef2 thru 0xe990796 (0xd8a5 bytes) 2024-09-14T05:30:09.596 INFO:teuthology.orchestra.run.smithi071.stdout:3886 write 0x3170104 thru 0x317adde (0xacdb bytes) 2024-09-14T05:30:09.607 INFO:teuthology.orchestra.run.smithi071.stdout:3887 trunc from 0xe990797 to 0xacb61ef 2024-09-14T05:30:09.617 INFO:teuthology.orchestra.run.smithi071.stdout:3888 read 0xa48eeea thru 0xa49df83 (0xf09a bytes) 2024-09-14T05:30:09.618 INFO:teuthology.orchestra.run.smithi071.stdout:3889 writesame 0x187dc16 thru 0x1886e4c data_size 0x3(0x9237 bytes) 2024-09-14T05:30:09.635 INFO:teuthology.orchestra.run.smithi071.stdout:3892 trunc from 0xacb61ef to 0x1eaa83f 2024-09-14T05:30:09.647 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:09.647+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:09.652 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:09.652 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:09.661 INFO:teuthology.orchestra.run.smithi071.stdout:3893 trunc from 0x1eaa83f to 0xce59300 2024-09-14T05:30:09.666 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:30:09.667 INFO:teuthology.orchestra.run.smithi071.stdout:3894 punch from 0x9c0937 to 0x9c190a, (0xfd3 bytes) 2024-09-14T05:30:09.667 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.667 INFO:teuthology.orchestra.run.smithi071.stdout:3896 read 0x2c3e107 thru 0x2c42832 (0x472c bytes) 2024-09-14T05:30:09.667 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.667 INFO:teuthology.orchestra.run.smithi071.stdout:3898 write 0x16a3482 thru 0x16af2ed (0xbe6c bytes) 2024-09-14T05:30:09.679 INFO:teuthology.orchestra.run.smithi071.stdout:3900 trunc from 0xce59300 to 0x6f051a0 2024-09-14T05:30:09.684 INFO:teuthology.orchestra.run.smithi071.stdout:3901 read 0x44b8a0c thru 0x44be5ba (0x5baf bytes) 2024-09-14T05:30:09.684 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.685 INFO:teuthology.orchestra.run.smithi071.stdout:3903 read 0x5ee99df thru 0x5ef106e (0x7690 bytes) 2024-09-14T05:30:09.700 INFO:teuthology.orchestra.run.smithi071.stdout:3904 write 0xe09d9fc thru 0xe0a7e4a (0xa44f bytes) 2024-09-14T05:30:09.704 INFO:teuthology.orchestra.run.smithi071.stdout:3906 writesame 0xa607127 thru 0xa60fa43 data_size 0x1(0x891d bytes) 2024-09-14T05:30:09.716 INFO:teuthology.orchestra.run.smithi071.stdout:3907 write 0x4e29c8b thru 0x4e29ea5 (0x21b bytes) 2024-09-14T05:30:09.719 INFO:teuthology.orchestra.run.smithi071.stdout:3908 read 0x60a0148 thru 0x60a0573 (0x42c bytes) 2024-09-14T05:30:09.720 INFO:teuthology.orchestra.run.smithi071.stdout:3909 write 0xeb87c0 thru 0xebaf64 (0x27a5 bytes) 2024-09-14T05:30:09.721 INFO:teuthology.orchestra.run.smithi071.stdout:3911 punch from 0xb03034 to 0xb09a30, (0x69fc bytes) 2024-09-14T05:30:09.721 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.721 INFO:teuthology.orchestra.run.smithi071.stdout:3914 write 0xfdd39f thru 0xfde2cc (0xf2e bytes) 2024-09-14T05:30:09.722 INFO:teuthology.orchestra.run.smithi071.stdout:3915 write 0xa546317 thru 0xa54a0e1 (0x3dcb bytes) 2024-09-14T05:30:09.724 INFO:teuthology.orchestra.run.smithi071.stdout:3918 write 0x471789 thru 0x47aa65 (0x92dd bytes) 2024-09-14T05:30:09.748 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:09.747+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:09.751 INFO:teuthology.orchestra.run.smithi071.stdout:3919 read 0xaeb12e1 thru 0xaeb2036 (0xd56 bytes) 2024-09-14T05:30:09.751 INFO:teuthology.orchestra.run.smithi071.stdout:3920 read 0x903e765 thru 0x9041e5f (0x36fb bytes) 2024-09-14T05:30:09.756 INFO:teuthology.orchestra.run.smithi071.stdout:3921 trunc from 0xe0a7e4b to 0x9b43446 2024-09-14T05:30:09.766 INFO:teuthology.orchestra.run.smithi071.stdout:3922 write 0x7eef892 thru 0x7efa855 (0xafc4 bytes) 2024-09-14T05:30:09.771 INFO:teuthology.orchestra.run.smithi071.stdout:3923 read 0x76aaba thru 0x76e018 (0x355f bytes) 2024-09-14T05:30:09.771 INFO:teuthology.orchestra.run.smithi071.stdout:3924 write 0x27aec42 thru 0x27afb22 (0xee1 bytes) 2024-09-14T05:30:09.774 INFO:teuthology.orchestra.run.smithi071.stdout:3925 write 0x238046b thru 0x2387558 (0x70ee bytes) 2024-09-14T05:30:09.775 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.776 INFO:teuthology.orchestra.run.smithi071.stdout:3928 punch from 0x220196 to 0x225085, (0x4eef bytes) 2024-09-14T05:30:09.776 INFO:teuthology.orchestra.run.smithi071.stdout:3931 read 0x72277c thru 0x725dbc (0x3641 bytes) 2024-09-14T05:30:09.776 INFO:teuthology.orchestra.run.smithi071.stdout:3932 punch from 0x5ee6fd5 to 0x5ef2d96, (0xbdc1 bytes) 2024-09-14T05:30:09.776 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:09.779 INFO:teuthology.orchestra.run.smithi071.stdout:3935 trunc from 0x9b43446 to 0x6de2af8 2024-09-14T05:30:09.789 INFO:teuthology.orchestra.run.smithi071.stdout:3936 read 0x67aa9c5 thru 0x67b5a81 (0xb0bd bytes) 2024-09-14T05:30:09.789 INFO:teuthology.orchestra.run.smithi071.stdout:3938 punch from 0x4dfe1d5 to 0x4e01f9c, (0x3dc7 bytes) 2024-09-14T05:30:09.789 INFO:teuthology.orchestra.run.smithi071.stdout:3941 read 0x55fa923 thru 0x5604c69 (0xa347 bytes) 2024-09-14T05:30:09.789 INFO:teuthology.orchestra.run.smithi071.stdout:3943 clone 31 order 18 su 65536 sc 6 2024-09-14T05:30:09.796 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:09.796 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:09.805 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:30:09.850 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:09.849+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:09.909 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:09.909 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:09.917 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:30:09.951 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:09.950+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:10.020 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:10.020 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:10.029 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:30:10.051 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:10.051+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:10.131 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:10.131 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:10.140 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:30:10.152 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:10.151+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:10.243 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:10.243 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:10.253 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:30:10.253 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:10.251+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:10.258 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T05:30:10.273 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T05:30:10.292 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T05:30:10.312 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone30 from 0x6de2af8 (overlap 0x1eaa83f) to 0x3110d3 2024-09-14T05:30:10.355 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:10.355+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:10.384 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-14T05:30:10.406 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:10.406 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:10.414 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:30:10.455 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:10.454+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:10.521 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:10.521 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:10.530 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:30:10.574 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:10.572+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:10.590 INFO:teuthology.orchestra.run.smithi071.stdout:3944 punch from 0x58ad9bc to 0x58b959c, (0xbbe0 bytes) 2024-09-14T05:30:10.594 INFO:teuthology.orchestra.run.smithi071.stdout:3946 read 0x2493b2f thru 0x24990ae (0x5580 bytes) 2024-09-14T05:30:10.604 INFO:teuthology.orchestra.run.smithi071.stdout:3947 write 0x9d0da7e thru 0x9d15398 (0x791b bytes) 2024-09-14T05:30:10.618 INFO:teuthology.orchestra.run.smithi071.stdout:3948 writesame 0xd47b981 thru 0xd47e930 data_size 0x38(0x2fb0 bytes) 2024-09-14T05:30:10.618 INFO:teuthology.orchestra.run.smithi071.stdout:3949 read 0xcb3f95c thru 0xcb44715 (0x4dba bytes) 2024-09-14T05:30:10.618 INFO:teuthology.orchestra.run.smithi071.stdout:3950 read 0xc53b947 thru 0xc546719 (0xadd3 bytes) 2024-09-14T05:30:10.618 INFO:teuthology.orchestra.run.smithi071.stdout:3951 write 0xb9914e1 thru 0xb99484c (0x336c bytes) 2024-09-14T05:30:10.620 INFO:teuthology.orchestra.run.smithi071.stdout:3952 write 0xac9169c thru 0xac9a18c (0x8af1 bytes) 2024-09-14T05:30:10.624 INFO:teuthology.orchestra.run.smithi071.stdout:3953 read 0x64e638d thru 0x64f33da (0xd04e bytes) 2024-09-14T05:30:10.631 INFO:teuthology.orchestra.run.smithi071.stdout:3954 write 0xee2e3e0 thru 0xee2fdb0 (0x19d1 bytes) 2024-09-14T05:30:10.634 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:10.634 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:10.635 INFO:teuthology.orchestra.run.smithi071.stdout:3955 read 0xb72cdc8 thru 0xb72f004 (0x223d bytes) 2024-09-14T05:30:10.635 INFO:teuthology.orchestra.run.smithi071.stdout:3956 writesame 0xee1b295 thru 0xee230f6 data_size 0x7(0x7e62 bytes) 2024-09-14T05:30:10.643 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:30:10.656 INFO:teuthology.orchestra.run.smithi071.stdout:3959 trunc from 0xee2fdb1 to 0x4612d5 2024-09-14T05:30:10.684 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:10.682+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:10.690 INFO:teuthology.orchestra.run.smithi071.stdout:3961 writesame 0xc403567 thru 0xc4064fd data_size 0x1f(0x2f97 bytes) 2024-09-14T05:30:10.690 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:10.690 INFO:teuthology.orchestra.run.smithi071.stdout:3963 read 0x1345f51 thru 0x134d59d (0x764d bytes) 2024-09-14T05:30:10.690 INFO:teuthology.orchestra.run.smithi071.stdout:3964 write 0x115883d thru 0x11644ca (0xbc8e bytes) 2024-09-14T05:30:10.693 INFO:teuthology.orchestra.run.smithi071.stdout:3965 writesame 0xbfa9b52 thru 0xbfab08d data_size 0x12(0x153c bytes) 2024-09-14T05:30:10.696 INFO:teuthology.orchestra.run.smithi071.stdout:3966 punch from 0x1ef8553 to 0x1eff758, (0x7205 bytes) 2024-09-14T05:30:10.696 INFO:teuthology.orchestra.run.smithi071.stdout:3967 read 0xb3cef91 thru 0xb3d9891 (0xa901 bytes) 2024-09-14T05:30:10.696 INFO:teuthology.orchestra.run.smithi071.stdout:3969 write 0xa966cc1 thru 0xa96efe6 (0x8326 bytes) 2024-09-14T05:30:10.699 INFO:teuthology.orchestra.run.smithi071.stdout:3970 write 0x26b3dd2 thru 0x26be9ad (0xabdc bytes) 2024-09-14T05:30:10.702 INFO:teuthology.orchestra.run.smithi071.stdout:3971 punch from 0xaa93629 to 0xaa94e4b, (0x1822 bytes) 2024-09-14T05:30:10.702 INFO:teuthology.orchestra.run.smithi071.stdout:3973 read 0x1711fa thru 0x1800f1 (0xeef8 bytes) 2024-09-14T05:30:10.702 INFO:teuthology.orchestra.run.smithi071.stdout:3974 read 0x25736f3 thru 0x2578d77 (0x5685 bytes) 2024-09-14T05:30:10.709 INFO:teuthology.orchestra.run.smithi071.stdout:3975 trunc from 0xc4064fe to 0xea09b37 2024-09-14T05:30:10.710 INFO:teuthology.orchestra.run.smithi071.stdout:3976 read 0x5f66dc5 thru 0x5f679a3 (0xbdf bytes) 2024-09-14T05:30:10.710 INFO:teuthology.orchestra.run.smithi071.stdout:3977 writesame 0x5118e79 thru 0x5121804 data_size 0x2(0x898c bytes) 2024-09-14T05:30:10.718 INFO:teuthology.orchestra.run.smithi071.stdout:3978 writesame 0x298a996 thru 0x298ede6 data_size 0x1(0x4451 bytes) 2024-09-14T05:30:10.729 INFO:teuthology.orchestra.run.smithi071.stdout:3979 read 0xdd918c5 thru 0xdd97bd1 (0x630d bytes) 2024-09-14T05:30:10.730 INFO:teuthology.orchestra.run.smithi071.stdout:3980 punch from 0x15f22a7 to 0x16014b6, (0xf20f bytes) 2024-09-14T05:30:10.730 INFO:teuthology.orchestra.run.smithi071.stdout:3981 write 0x36c4b2a thru 0x36c5afb (0xfd2 bytes) 2024-09-14T05:30:10.734 INFO:teuthology.orchestra.run.smithi071.stdout:3982 read 0x9daf83b thru 0x9db2305 (0x2acb bytes) 2024-09-14T05:30:10.743 INFO:teuthology.orchestra.run.smithi071.stdout:3985 trunc from 0xea09b37 to 0x68aa571 2024-09-14T05:30:10.753 INFO:teuthology.orchestra.run.smithi071.stdout:3986 read 0x45c1de8 thru 0x45d1068 (0xf281 bytes) 2024-09-14T05:30:10.758 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:10.756+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:10.767 INFO:teuthology.orchestra.run.smithi071.stdout:3987 writesame 0xdbf133c thru 0xdbfcbcf data_size 0x2(0xb894 bytes) 2024-09-14T05:30:10.767 INFO:teuthology.orchestra.run.smithi071.stdout:3988 read 0x8cad825 thru 0x8cb80aa (0xa886 bytes) 2024-09-14T05:30:10.767 INFO:teuthology.orchestra.run.smithi071.stdout:3989 clone 32 order 21 su 524288 sc 2 2024-09-14T05:30:10.788 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:10.789 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:10.797 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:30:10.881 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:10.880+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:10.941 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:10.941 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:10.949 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:30:10.959 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:10.958+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:11.060 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:11.060+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:11.093 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:11.094 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:11.102 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:30:11.161 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:11.160+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:11.207 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:11.207 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:11.216 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:30:11.254 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T05:30:11.254 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-14T05:30:11.262 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:11.261+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:11.276 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T05:30:11.330 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone31 from 0xdbfcbd0 (overlap 0x4612d5) to 0x1505a6 2024-09-14T05:30:11.363 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:11.361+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:11.366 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:11.366 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:11.376 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:30:11.415 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-14T05:30:11.422 INFO:teuthology.orchestra.run.smithi071.stdout:3990 write 0x698a3b thru 0x6a7f8e (0xf554 bytes) 2024-09-14T05:30:11.431 INFO:teuthology.orchestra.run.smithi071.stdout:3991 read 0x8c0ca53 thru 0x8c11018 (0x45c6 bytes) 2024-09-14T05:30:11.431 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:11.431 INFO:teuthology.orchestra.run.smithi071.stdout:3994 read 0x14104d thru 0x14645d (0x5411 bytes) 2024-09-14T05:30:11.431 INFO:teuthology.orchestra.run.smithi071.stdout:3995 read 0x9e5c4f2 thru 0x9e61339 (0x4e48 bytes) 2024-09-14T05:30:11.431 INFO:teuthology.orchestra.run.smithi071.stdout:3996 write 0x535929e thru 0x53638ba (0xa61d bytes) 2024-09-14T05:30:11.436 INFO:teuthology.orchestra.run.smithi071.stdout:3998 writesame 0xc85ff83 thru 0xc864797 data_size 0x1(0x4815 bytes) 2024-09-14T05:30:11.447 INFO:teuthology.orchestra.run.smithi071.stdout:3999 write 0x99b04a4 thru 0x99b5de3 (0x5940 bytes) 2024-09-14T05:30:11.451 INFO:teuthology.orchestra.run.smithi071.stdout:4000 read 0x8b77bbe thru 0x8b83fb9 (0xc3fc bytes) 2024-09-14T05:30:11.451 INFO:teuthology.orchestra.run.smithi071.stdout:4002 punch from 0xb2edb52 to 0xb2eea73, (0xf21 bytes) 2024-09-14T05:30:11.451 INFO:teuthology.orchestra.run.smithi071.stdout:4003 read 0xd735854 thru 0xd73b662 (0x5e0f bytes) 2024-09-14T05:30:11.451 INFO:teuthology.orchestra.run.smithi071.stdout:4005 writesame 0x69da09e thru 0x69e05ae data_size 0x1(0x6511 bytes) 2024-09-14T05:30:11.463 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:11.463+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:11.465 INFO:teuthology.orchestra.run.smithi071.stdout:4006 read 0xf6238e thru 0xf678c8 (0x553b bytes) 2024-09-14T05:30:11.466 INFO:teuthology.orchestra.run.smithi071.stdout:4007 trunc from 0xdbfcbd0 to 0xd8715dd 2024-09-14T05:30:11.474 INFO:teuthology.orchestra.run.smithi071.stdout:4008 write 0x1dd5b06 thru 0x1dd6877 (0xd72 bytes) 2024-09-14T05:30:11.478 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:11.478 INFO:teuthology.orchestra.run.smithi071.stdout:4010 read 0x4b69f6d thru 0x4b6bbab (0x1c3f bytes) 2024-09-14T05:30:11.478 INFO:teuthology.orchestra.run.smithi071.stdout:4011 read 0x8d17414 thru 0x8d18258 (0xe45 bytes) 2024-09-14T05:30:11.478 INFO:teuthology.orchestra.run.smithi071.stdout:4013 punch from 0x794726b to 0x7954823, (0xd5b8 bytes) 2024-09-14T05:30:11.492 INFO:teuthology.orchestra.run.smithi071.stdout:4014 trunc from 0xd8715dd to 0x35264fe 2024-09-14T05:30:11.508 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:11.508 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:11.512 INFO:teuthology.orchestra.run.smithi071.stdout:4017 read 0x20d2244 thru 0x20d9bf8 (0x79b5 bytes) 2024-09-14T05:30:11.512 INFO:teuthology.orchestra.run.smithi071.stdout:4019 write 0x63abb1 thru 0x640b32 (0x5f82 bytes) 2024-09-14T05:30:11.518 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:30:11.519 INFO:teuthology.orchestra.run.smithi071.stdout:4020 read 0xc2fb54 thru 0xc3cae3 (0xcf90 bytes) 2024-09-14T05:30:11.519 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:11.519 INFO:teuthology.orchestra.run.smithi071.stdout:4022 writesame 0x26d3eb6 thru 0x26d9d1b data_size 0x2b(0x5e66 bytes) 2024-09-14T05:30:11.532 INFO:teuthology.orchestra.run.smithi071.stdout:4023 write 0x551ae99 thru 0x5529f80 (0xf0e8 bytes) 2024-09-14T05:30:11.535 INFO:teuthology.orchestra.run.smithi071.stdout:4024 read 0x189e9ff thru 0x18ad819 (0xee1b bytes) 2024-09-14T05:30:11.535 INFO:teuthology.orchestra.run.smithi071.stdout:4026 read 0x27dda4c thru 0x27e9e25 (0xc3da bytes) 2024-09-14T05:30:11.535 INFO:teuthology.orchestra.run.smithi071.stdout:4027 punch from 0x29f0e42 to 0x29f5838, (0x49f6 bytes) 2024-09-14T05:30:11.535 INFO:teuthology.orchestra.run.smithi071.stdout:4028 read 0x331b7c8 thru 0x331f682 (0x3ebb bytes) 2024-09-14T05:30:11.535 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:11.541 INFO:teuthology.orchestra.run.smithi071.stdout:4032 trunc from 0x5529f81 to 0x385ec9 2024-09-14T05:30:11.565 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:11.564+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:11.565 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:11.565 INFO:teuthology.orchestra.run.smithi071.stdout:4035 read 0x54842 thru 0x5c2a8 (0x7a67 bytes) 2024-09-14T05:30:11.587 INFO:teuthology.orchestra.run.smithi071.stdout:4036 write 0xdef2711 thru 0xdefe9b9 (0xc2a9 bytes) 2024-09-14T05:30:11.590 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:11.590 INFO:teuthology.orchestra.run.smithi071.stdout:4038 write 0x51cf068 thru 0x51d55b4 (0x654d bytes) 2024-09-14T05:30:11.598 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:30:11.601 INFO:teuthology.orchestra.run.smithi071.stdout:4039 write 0xe2f2da7 thru 0xe2fe612 (0xb86c bytes) 2024-09-14T05:30:11.603 INFO:teuthology.orchestra.run.smithi071.stdout:4041 read 0x94ad2ac thru 0x94aee16 (0x1b6b bytes) 2024-09-14T05:30:11.603 INFO:teuthology.orchestra.run.smithi071.stdout:4042 read 0xb2d504e thru 0xb2d687c (0x182f bytes) 2024-09-14T05:30:11.603 INFO:teuthology.orchestra.run.smithi071.stdout:4043 read 0xdd8fcef thru 0xdd98564 (0x8876 bytes) 2024-09-14T05:30:11.604 INFO:teuthology.orchestra.run.smithi071.stdout:4044 write 0x1e81760 thru 0x1e91327 (0xfbc8 bytes) 2024-09-14T05:30:11.608 INFO:teuthology.orchestra.run.smithi071.stdout:4045 write 0x65cbd5b thru 0x65d910e (0xd3b4 bytes) 2024-09-14T05:30:11.611 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:11.611 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:11.624 INFO:teuthology.orchestra.run.smithi071.stdout:4050 trunc from 0xe2fe613 to 0x315a079 2024-09-14T05:30:11.636 INFO:teuthology.orchestra.run.smithi071.stdout:4051 read 0x20e04ef thru 0x20e82f1 (0x7e03 bytes) 2024-09-14T05:30:11.636 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:11.636 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:11.636 INFO:teuthology.orchestra.run.smithi071.stdout:4054 punch from 0x2715a4d to 0x27164f6, (0xaa9 bytes) 2024-09-14T05:30:11.649 INFO:teuthology.orchestra.run.smithi071.stdout:4056 trunc from 0x315a079 to 0xe252ab3 2024-09-14T05:30:11.656 INFO:teuthology.orchestra.run.smithi071.stdout:4057 write 0x46dcd8f thru 0x46e6e1f (0xa091 bytes) 2024-09-14T05:30:11.657 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:11.657 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:11.659 INFO:teuthology.orchestra.run.smithi071.stdout:4058 write 0x71695f thru 0x719fd6 (0x3678 bytes) 2024-09-14T05:30:11.662 INFO:teuthology.orchestra.run.smithi071.stdout:4059 writesame 0x8aefd4f thru 0x8af1c68 data_size 0x3(0x1f1a bytes) 2024-09-14T05:30:11.666 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:11.665+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:11.668 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:30:11.675 INFO:teuthology.orchestra.run.smithi071.stdout:4060 trunc from 0xe252ab3 to 0x6b9cf7d 2024-09-14T05:30:11.691 INFO:teuthology.orchestra.run.smithi071.stdout:4061 trunc from 0x6b9cf7d to 0x8bf5df8 2024-09-14T05:30:11.707 INFO:teuthology.orchestra.run.smithi071.stdout:4062 write 0xb803f87 thru 0xb804cdc (0xd56 bytes) 2024-09-14T05:30:11.710 INFO:teuthology.orchestra.run.smithi071.stdout:4063 write 0x5e6bcc0 thru 0x5e766be (0xa9ff bytes) 2024-09-14T05:30:11.713 INFO:teuthology.orchestra.run.smithi071.stdout:4064 write 0xadc939b thru 0xadcfcbf (0x6925 bytes) 2024-09-14T05:30:11.716 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:11.717 INFO:teuthology.orchestra.run.smithi071.stdout:4066 write 0x48f3a92 thru 0x48fa3e0 (0x694f bytes) 2024-09-14T05:30:11.724 INFO:teuthology.orchestra.run.smithi071.stdout:4067 trunc from 0xb804cdd to 0xee367c3 2024-09-14T05:30:11.743 INFO:teuthology.orchestra.run.smithi071.stdout:4068 trunc from 0xee367c3 to 0x3e89599 2024-09-14T05:30:11.744 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:11.744 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:11.752 INFO:teuthology.orchestra.run.smithi071.stdout:4069 write 0x7905f thru 0x85252 (0xc1f4 bytes) 2024-09-14T05:30:11.767 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:11.765+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:11.767 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:11.767 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:11.782 INFO:teuthology.orchestra.run.smithi071.stdout:4070 writesame 0x983b21f thru 0x984762e data_size 0x8(0xc410 bytes) 2024-09-14T05:30:11.782 INFO:teuthology.orchestra.run.smithi071.stdout:4071 read 0x89edd08 thru 0x89fa47a (0xc773 bytes) 2024-09-14T05:30:11.782 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:11.782 INFO:teuthology.orchestra.run.smithi071.stdout:4077 writesame 0x6c4cf3f thru 0x6c56340 data_size 0xa(0x9402 bytes) 2024-09-14T05:30:11.786 INFO:teuthology.orchestra.run.smithi071.stdout:4078 read 0x22d4756 thru 0x22d7a9a (0x3345 bytes) 2024-09-14T05:30:11.786 INFO:teuthology.orchestra.run.smithi071.stdout:4079 punch from 0x5fa6b4a to 0x5fb6abe, (0xff74 bytes) 2024-09-14T05:30:11.786 INFO:teuthology.orchestra.run.smithi071.stdout:4082 read 0x6c0f85c thru 0x6c13fd5 (0x477a bytes) 2024-09-14T05:30:11.786 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:11.787 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:11.788 INFO:teuthology.orchestra.run.smithi071.stdout:4083 read 0x4c57d73 thru 0x4c60d20 (0x8fae bytes) 2024-09-14T05:30:11.788 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:11.788 INFO:teuthology.orchestra.run.smithi071.stdout:4085 punch from 0x266458f to 0x26702c7, (0xbd38 bytes) 2024-09-14T05:30:11.788 INFO:teuthology.orchestra.run.smithi071.stdout:4086 clone 33 order 21 su 16384 sc 13 2024-09-14T05:30:11.800 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:11.800 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:11.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:11.801 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:11.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.801 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74187 3.f 3:f707b2d9:::rbd_data.112d46b1ca94.0000000000000061:11 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.801 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:11.801 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.230698555, 2024-09-14T05:30:11.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072367828999999995, 2024-09-14T05:30:11.801 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.801 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.801 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.801 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.801 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.802 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.802 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74187 2024-09-14T05:30:11.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.802 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:11.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:11.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:11.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0900000000000002e-07 2024-09-14T05:30:11.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568166+0000", 2024-09-14T05:30:11.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.683e-06 2024-09-14T05:30:11.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568172+0000", 2024-09-14T05:30:11.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0639999999999999e-06 2024-09-14T05:30:11.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568177+0000", 2024-09-14T05:30:11.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.967e-06 2024-09-14T05:30:11.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640193+0000", 2024-09-14T05:30:11.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072016284 2024-09-14T05:30:11.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640282+0000", 2024-09-14T05:30:11.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9011999999999994e-05 2024-09-14T05:30:11.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.805 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.805 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640532+0000", 2024-09-14T05:30:11.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024940999999999999 2024-09-14T05:30:11.806 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.806 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.806 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.806 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74188 3.f 3:f707b2d9:::rbd_data.112d46b1ca94.0000000000000061:11 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.806 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:11.806 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.230679181999999, 2024-09-14T05:30:11.806 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072600425999999996, 2024-09-14T05:30:11.806 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.806 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.806 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.806 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.806 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.807 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.807 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74188 2024-09-14T05:30:11.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.807 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:11.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:11.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.808 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.808 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568184+0000", 2024-09-14T05:30:11.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8700000000000001e-07 2024-09-14T05:30:11.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.808 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.808 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568200+0000", 2024-09-14T05:30:11.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6603e-05 2024-09-14T05:30:11.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568208+0000", 2024-09-14T05:30:11.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4800000000000004e-06 2024-09-14T05:30:11.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568213+0000", 2024-09-14T05:30:11.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6860000000000003e-06 2024-09-14T05:30:11.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640563+0000", 2024-09-14T05:30:11.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072349438000000002 2024-09-14T05:30:11.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640607+0000", 2024-09-14T05:30:11.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4014999999999998e-05 2024-09-14T05:30:11.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640784+0000", 2024-09-14T05:30:11.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017681700000000001 2024-09-14T05:30:11.811 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.811 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.811 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.811 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74194 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.811 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:11.811 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.230641049999999, 2024-09-14T05:30:11.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072795199000000005, 2024-09-14T05:30:11.812 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.812 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.812 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.812 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.812 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.812 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.812 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74194 2024-09-14T05:30:11.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.812 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:11.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:11.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568222+0000", 2024-09-14T05:30:11.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1300000000000001e-07 2024-09-14T05:30:11.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568224+0000", 2024-09-14T05:30:11.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0660000000000002e-06 2024-09-14T05:30:11.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568230+0000", 2024-09-14T05:30:11.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4470000000000003e-06 2024-09-14T05:30:11.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568235+0000", 2024-09-14T05:30:11.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1220000000000003e-06 2024-09-14T05:30:11.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640824+0000", 2024-09-14T05:30:11.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072588707000000002 2024-09-14T05:30:11.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640915+0000", 2024-09-14T05:30:11.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1008999999999995e-05 2024-09-14T05:30:11.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641017+0000", 2024-09-14T05:30:11.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010143499999999999 2024-09-14T05:30:11.816 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.816 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.816 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.816 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74208 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2693120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.816 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:11.816 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.230620473999998, 2024-09-14T05:30:11.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072905555999999996, 2024-09-14T05:30:11.816 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.816 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.816 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.817 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.817 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.817 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.817 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74208 2024-09-14T05:30:11.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.817 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:11.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:11.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:11.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9799999999999999e-07 2024-09-14T05:30:11.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568244+0000", 2024-09-14T05:30:11.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0339999999999999e-06 2024-09-14T05:30:11.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568251+0000", 2024-09-14T05:30:11.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4060000000000002e-06 2024-09-14T05:30:11.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568255+0000", 2024-09-14T05:30:11.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6589999999999998e-06 2024-09-14T05:30:11.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641048+0000", 2024-09-14T05:30:11.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072792647000000002 2024-09-14T05:30:11.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641086+0000", 2024-09-14T05:30:11.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8031999999999998e-05 2024-09-14T05:30:11.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641147+0000", 2024-09-14T05:30:11.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1379999999999998e-05 2024-09-14T05:30:11.821 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.822 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.822 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.822 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74236 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2662400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.822 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:11.822 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.229986811, 2024-09-14T05:30:11.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072517776000000006, 2024-09-14T05:30:11.822 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.822 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.822 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.822 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.822 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.823 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.823 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74236 2024-09-14T05:30:11.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.823 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:11.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:11.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568881+0000", 2024-09-14T05:30:11.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2340000000000002e-06 2024-09-14T05:30:11.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568886+0000", 2024-09-14T05:30:11.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9450000000000001e-06 2024-09-14T05:30:11.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568898+0000", 2024-09-14T05:30:11.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2150000000000001e-05 2024-09-14T05:30:11.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568907+0000", 2024-09-14T05:30:11.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9239999999999996e-06 2024-09-14T05:30:11.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641185+0000", 2024-09-14T05:30:11.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.07227807 2024-09-14T05:30:11.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.826 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641265+0000", 2024-09-14T05:30:11.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0575999999999996e-05 2024-09-14T05:30:11.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.826 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641393+0000", 2024-09-14T05:30:11.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012787700000000001 2024-09-14T05:30:11.827 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.827 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.827 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.827 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74239 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2695168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.827 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:11.827 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.229942418, 2024-09-14T05:30:11.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072593816000000005, 2024-09-14T05:30:11.828 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.828 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.828 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.828 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.828 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.828 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.828 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74239 2024-09-14T05:30:11.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.828 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:11.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:11.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:11.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4300000000000003e-07 2024-09-14T05:30:11.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568923+0000", 2024-09-14T05:30:11.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2019999999999998e-06 2024-09-14T05:30:11.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568933+0000", 2024-09-14T05:30:11.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0376e-05 2024-09-14T05:30:11.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568938+0000", 2024-09-14T05:30:11.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2850000000000002e-06 2024-09-14T05:30:11.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641424+0000", 2024-09-14T05:30:11.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072485230999999997 2024-09-14T05:30:11.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641464+0000", 2024-09-14T05:30:11.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0574000000000002e-05 2024-09-14T05:30:11.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641514+0000", 2024-09-14T05:30:11.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9604999999999997e-05 2024-09-14T05:30:11.833 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.834 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.834 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.834 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74252 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2664448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.834 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:11.834 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.229762504, 2024-09-14T05:30:11.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072631237000000001, 2024-09-14T05:30:11.834 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.834 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.834 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.834 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.834 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.834 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.835 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74252 2024-09-14T05:30:11.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.835 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:11.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:11.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569101+0000", 2024-09-14T05:30:11.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8700000000000005e-07 2024-09-14T05:30:11.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569104+0000", 2024-09-14T05:30:11.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.163e-06 2024-09-14T05:30:11.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569115+0000", 2024-09-14T05:30:11.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1335000000000001e-05 2024-09-14T05:30:11.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569121+0000", 2024-09-14T05:30:11.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0800000000000002e-06 2024-09-14T05:30:11.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641552+0000", 2024-09-14T05:30:11.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072430743000000006 2024-09-14T05:30:11.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641631+0000", 2024-09-14T05:30:11.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9388999999999995e-05 2024-09-14T05:30:11.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641731+0000", 2024-09-14T05:30:11.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9840000000000006e-05 2024-09-14T05:30:11.838 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.839 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.839 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.839 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74256 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2697216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.839 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:11.839 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.229731447999999, 2024-09-14T05:30:11.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072721090000000002, 2024-09-14T05:30:11.839 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.839 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.839 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.839 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.840 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.840 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.840 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74256 2024-09-14T05:30:11.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.840 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:11.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:11.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:11.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0299999999999999e-07 2024-09-14T05:30:11.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569133+0000", 2024-09-14T05:30:11.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.046e-06 2024-09-14T05:30:11.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569139+0000", 2024-09-14T05:30:11.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0279999999999997e-06 2024-09-14T05:30:11.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569144+0000", 2024-09-14T05:30:11.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6929999999999998e-06 2024-09-14T05:30:11.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641762+0000", 2024-09-14T05:30:11.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072617904999999996 2024-09-14T05:30:11.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641801+0000", 2024-09-14T05:30:11.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8902999999999997e-05 2024-09-14T05:30:11.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641852+0000", 2024-09-14T05:30:11.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1011999999999999e-05 2024-09-14T05:30:11.844 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.844 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.844 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.844 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74285 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2672640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.844 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:11.844 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.228886012, 2024-09-14T05:30:11.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072422069000000006, 2024-09-14T05:30:11.844 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.844 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.844 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.845 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.845 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.845 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.845 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74285 2024-09-14T05:30:11.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.845 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:11.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:11.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569977+0000", 2024-09-14T05:30:11.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3099999999999998e-07 2024-09-14T05:30:11.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569978+0000", 2024-09-14T05:30:11.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.598e-06 2024-09-14T05:30:11.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569984+0000", 2024-09-14T05:30:11.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4689999999999996e-06 2024-09-14T05:30:11.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569989+0000", 2024-09-14T05:30:11.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8409999999999999e-06 2024-09-14T05:30:11.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642229+0000", 2024-09-14T05:30:11.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072240046000000002 2024-09-14T05:30:11.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642306+0000", 2024-09-14T05:30:11.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7347000000000003e-05 2024-09-14T05:30:11.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642398+0000", 2024-09-14T05:30:11.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2336999999999994e-05 2024-09-14T05:30:11.848 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.849 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.849 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.849 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74292 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2705408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.849 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:11.849 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.228631304, 2024-09-14T05:30:11.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072361018999999999, 2024-09-14T05:30:11.849 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.849 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.849 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.849 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.850 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.850 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.850 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74292 2024-09-14T05:30:11.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.850 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:11.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:11.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570232+0000", 2024-09-14T05:30:11.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.06e-07 2024-09-14T05:30:11.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570235+0000", 2024-09-14T05:30:11.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3189999999999999e-06 2024-09-14T05:30:11.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570248+0000", 2024-09-14T05:30:11.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3006e-05 2024-09-14T05:30:11.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570254+0000", 2024-09-14T05:30:11.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.908e-06 2024-09-14T05:30:11.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642433+0000", 2024-09-14T05:30:11.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072179192000000003 2024-09-14T05:30:11.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642500+0000", 2024-09-14T05:30:11.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7080000000000001e-05 2024-09-14T05:30:11.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642592+0000", 2024-09-14T05:30:11.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1807999999999999e-05 2024-09-14T05:30:11.854 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.854 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.854 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.854 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74304 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2707456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.854 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:11.854 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.228274637999998, 2024-09-14T05:30:11.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072444960000000003, 2024-09-14T05:30:11.855 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.855 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.855 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.855 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.855 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.855 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.855 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74304 2024-09-14T05:30:11.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.855 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:11.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:11.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:11.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9800000000000004e-07 2024-09-14T05:30:11.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570590+0000", 2024-09-14T05:30:11.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0810000000000001e-06 2024-09-14T05:30:11.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570596+0000", 2024-09-14T05:30:11.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9789999999999998e-06 2024-09-14T05:30:11.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570601+0000", 2024-09-14T05:30:11.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6750000000000001e-06 2024-09-14T05:30:11.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642835+0000", 2024-09-14T05:30:11.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.07223359 2024-09-14T05:30:11.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642917+0000", 2024-09-14T05:30:11.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2013000000000002e-05 2024-09-14T05:30:11.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643033+0000", 2024-09-14T05:30:11.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116124 2024-09-14T05:30:11.859 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.859 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.859 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.859 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74321 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2709504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.859 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:11.859 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.227792648000001, 2024-09-14T05:30:11.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072350611999999995, 2024-09-14T05:30:11.860 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.860 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.860 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.860 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.860 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.860 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.860 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74321 2024-09-14T05:30:11.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.860 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:11.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:11.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:11.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3e-07 2024-09-14T05:30:11.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571073+0000", 2024-09-14T05:30:11.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6879999999999999e-06 2024-09-14T05:30:11.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571080+0000", 2024-09-14T05:30:11.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0319999999999998e-06 2024-09-14T05:30:11.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571085+0000", 2024-09-14T05:30:11.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2109999999999997e-06 2024-09-14T05:30:11.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643309+0000", 2024-09-14T05:30:11.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072223812999999998 2024-09-14T05:30:11.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643351+0000", 2024-09-14T05:30:11.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2239e-05 2024-09-14T05:30:11.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643420+0000", 2024-09-14T05:30:11.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8998999999999994e-05 2024-09-14T05:30:11.864 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.864 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.865 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.865 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74332 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2678784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.865 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:11.865 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.227469485, 2024-09-14T05:30:11.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072259062999999998, 2024-09-14T05:30:11.865 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.866 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.866 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.866 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.866 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.866 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.866 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74332 2024-09-14T05:30:11.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.867 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:11.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:11.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571394+0000", 2024-09-14T05:30:11.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:30:11.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571397+0000", 2024-09-14T05:30:11.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8469999999999998e-06 2024-09-14T05:30:11.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571411+0000", 2024-09-14T05:30:11.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.34e-05 2024-09-14T05:30:11.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571418+0000", 2024-09-14T05:30:11.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9149999999999999e-06 2024-09-14T05:30:11.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643458+0000", 2024-09-14T05:30:11.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072039881 2024-09-14T05:30:11.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.870 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.870 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643539+0000", 2024-09-14T05:30:11.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1650000000000006e-05 2024-09-14T05:30:11.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.870 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.870 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643652+0000", 2024-09-14T05:30:11.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000112593 2024-09-14T05:30:11.871 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.871 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.871 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.871 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74336 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2711552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.871 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:11.871 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.227433035000001, 2024-09-14T05:30:11.871 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072354038999999995, 2024-09-14T05:30:11.871 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.871 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.871 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.871 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.872 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.872 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.872 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74336 2024-09-14T05:30:11.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.872 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.872 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:11.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.872 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:11.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.873 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.873 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571430+0000", 2024-09-14T05:30:11.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8999999999999996e-07 2024-09-14T05:30:11.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.873 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.873 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571432+0000", 2024-09-14T05:30:11.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4159999999999998e-06 2024-09-14T05:30:11.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571439+0000", 2024-09-14T05:30:11.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.657e-06 2024-09-14T05:30:11.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571444+0000", 2024-09-14T05:30:11.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1329999999999998e-06 2024-09-14T05:30:11.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643690+0000", 2024-09-14T05:30:11.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072246088 2024-09-14T05:30:11.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.875 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.875 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643731+0000", 2024-09-14T05:30:11.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0311000000000001e-05 2024-09-14T05:30:11.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.875 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.875 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643783+0000", 2024-09-14T05:30:11.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2843999999999998e-05 2024-09-14T05:30:11.875 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.875 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.875 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.876 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74349 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.876 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:11.876 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.227412026, 2024-09-14T05:30:11.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072554297000000004, 2024-09-14T05:30:11.876 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.876 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.876 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.876 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.876 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.876 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.876 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74349 2024-09-14T05:30:11.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.877 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:11.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:11.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571451+0000", 2024-09-14T05:30:11.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.41e-07 2024-09-14T05:30:11.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571453+0000", 2024-09-14T05:30:11.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8360000000000001e-06 2024-09-14T05:30:11.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571475+0000", 2024-09-14T05:30:11.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2452000000000001e-05 2024-09-14T05:30:11.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.880 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571480+0000", 2024-09-14T05:30:11.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4430000000000002e-06 2024-09-14T05:30:11.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.880 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643821+0000", 2024-09-14T05:30:11.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072340347999999999 2024-09-14T05:30:11.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643904+0000", 2024-09-14T05:30:11.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3534000000000004e-05 2024-09-14T05:30:11.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644005+0000", 2024-09-14T05:30:11.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100343 2024-09-14T05:30:11.881 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.881 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.881 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.882 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74357 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2713600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.882 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:11.882 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.227367993000001, 2024-09-14T05:30:11.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072651144000000001, 2024-09-14T05:30:11.882 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.882 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.882 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.882 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.882 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.882 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.882 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74357 2024-09-14T05:30:11.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.883 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:11.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:11.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571495+0000", 2024-09-14T05:30:11.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8899999999999999e-07 2024-09-14T05:30:11.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571498+0000", 2024-09-14T05:30:11.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5840000000000002e-06 2024-09-14T05:30:11.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571504+0000", 2024-09-14T05:30:11.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6549999999999998e-06 2024-09-14T05:30:11.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571509+0000", 2024-09-14T05:30:11.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6500000000000004e-06 2024-09-14T05:30:11.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644043+0000", 2024-09-14T05:30:11.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072533832000000006 2024-09-14T05:30:11.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644090+0000", 2024-09-14T05:30:11.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7426000000000003e-05 2024-09-14T05:30:11.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644146+0000", 2024-09-14T05:30:11.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5408000000000002e-05 2024-09-14T05:30:11.887 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.887 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.887 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.887 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74364 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2682880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.887 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:11.887 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.227031221000001, 2024-09-14T05:30:11.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072531227000000004, 2024-09-14T05:30:11.888 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.888 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.888 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.888 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.888 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.888 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.888 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74364 2024-09-14T05:30:11.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.888 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:11.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:11.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571832+0000", 2024-09-14T05:30:11.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4e-07 2024-09-14T05:30:11.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571838+0000", 2024-09-14T05:30:11.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0519999999999998e-06 2024-09-14T05:30:11.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571865+0000", 2024-09-14T05:30:11.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7109000000000001e-05 2024-09-14T05:30:11.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571883+0000", 2024-09-14T05:30:11.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7428e-05 2024-09-14T05:30:11.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644182+0000", 2024-09-14T05:30:11.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072299551000000004 2024-09-14T05:30:11.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644260+0000", 2024-09-14T05:30:11.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7551000000000003e-05 2024-09-14T05:30:11.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644362+0000", 2024-09-14T05:30:11.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010259600000000001 2024-09-14T05:30:11.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.893 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.893 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74367 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2715648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.893 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:11.894 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.226667914, 2024-09-14T05:30:11.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072368259000000004, 2024-09-14T05:30:11.894 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.894 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.894 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.894 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74367 2024-09-14T05:30:11.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.895 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:11.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:11.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572195+0000", 2024-09-14T05:30:11.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.78e-07 2024-09-14T05:30:11.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572200+0000", 2024-09-14T05:30:11.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2520000000000001e-06 2024-09-14T05:30:11.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572213+0000", 2024-09-14T05:30:11.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3618000000000001e-05 2024-09-14T05:30:11.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572226+0000", 2024-09-14T05:30:11.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3040999999999999e-05 2024-09-14T05:30:11.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644409+0000", 2024-09-14T05:30:11.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072182982000000007 2024-09-14T05:30:11.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644489+0000", 2024-09-14T05:30:11.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9429000000000001e-05 2024-09-14T05:30:11.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644563+0000", 2024-09-14T05:30:11.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4059000000000004e-05 2024-09-14T05:30:11.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.898 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.898 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74380 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2684928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.899 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:11.899 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.226534136000001, 2024-09-14T05:30:11.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072442180999999994, 2024-09-14T05:30:11.899 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.899 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.899 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.899 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.899 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.899 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.899 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74380 2024-09-14T05:30:11.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.900 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:11.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:11.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572329+0000", 2024-09-14T05:30:11.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3300000000000001e-07 2024-09-14T05:30:11.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572333+0000", 2024-09-14T05:30:11.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8e-06 2024-09-14T05:30:11.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572346+0000", 2024-09-14T05:30:11.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.307e-05 2024-09-14T05:30:11.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572353+0000", 2024-09-14T05:30:11.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9e-06 2024-09-14T05:30:11.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644597+0000", 2024-09-14T05:30:11.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072244770999999999 2024-09-14T05:30:11.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.903 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.903 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644672+0000", 2024-09-14T05:30:11.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4945999999999997e-05 2024-09-14T05:30:11.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.903 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.903 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644770+0000", 2024-09-14T05:30:11.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7961000000000006e-05 2024-09-14T05:30:11.904 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.904 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.904 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.904 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74383 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2717696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:11.904 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:11.904 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.226278142000002, 2024-09-14T05:30:11.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072397258000000006, 2024-09-14T05:30:11.905 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:11.905 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:11.905 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:11.905 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:11.905 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:11.905 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:11.905 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74383 2024-09-14T05:30:11.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.906 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:11.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:11.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:11.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:11.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:11.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:11.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:11.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572585+0000", 2024-09-14T05:30:11.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:30:11.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:11.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572589+0000", 2024-09-14T05:30:11.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:30:11.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:11.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572602+0000", 2024-09-14T05:30:11.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2917e-05 2024-09-14T05:30:11.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:11.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572610+0000", 2024-09-14T05:30:11.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.943e-06 2024-09-14T05:30:11.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:11.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644791+0000", 2024-09-14T05:30:11.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072181113000000005 2024-09-14T05:30:11.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:11.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644861+0000", 2024-09-14T05:30:11.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0650999999999998e-05 2024-09-14T05:30:11.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:11.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:11.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:11.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644981+0000", 2024-09-14T05:30:11.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000120185 2024-09-14T05:30:11.910 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.910 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.910 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.910 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:11.910 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:11.910 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:11.911 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:11.911 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:11.912 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:30:11.913 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:11.913 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:11.913 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:11.913 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:11.914 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:11.914 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:11.914 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:11.915 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:11.915 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:11.910+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:12.012 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:12.011+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:12.057 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:12.057 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:12.057 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:12.057 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:12.066 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:30:12.114 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:12.113+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:12.168 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:12.168 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:12.169 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:12.169 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:12.169 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:12.177 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:30:12.218 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:12.217+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:12.270 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T05:30:12.287 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T05:30:12.315 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:12.313+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:12.333 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:12.333 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:12.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:12.333 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:12.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.333 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:30:12.333 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:30:12.333 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 64.555101730999993, 2024-09-14T05:30:12.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.24562984099999999, 2024-09-14T05:30:12.334 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.334 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.334 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:12.334 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.334 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:30:12.334 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:30:12.334 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1 2024-09-14T05:30:12.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.334 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.334 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.334 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:30:12.334 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:30:12.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776565+0000", 2024-09-14T05:30:12.335 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4780000000000001e-06 2024-09-14T05:30:12.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.335 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.335 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776571+0000", 2024-09-14T05:30:12.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7669999999999998e-06 2024-09-14T05:30:12.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776580+0000", 2024-09-14T05:30:12.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9830000000000002e-06 2024-09-14T05:30:12.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.336 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.336 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776589+0000", 2024-09-14T05:30:12.336 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7579999999999998e-06 2024-09-14T05:30:12.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776662+0000", 2024-09-14T05:30:12.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3332999999999997e-05 2024-09-14T05:30:12.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:30:12.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776670+0000", 2024-09-14T05:30:12.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9850000000000006e-06 2024-09-14T05:30:12.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.337 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.337 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.020867+0000", 2024-09-14T05:30:12.337 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.244196265 2024-09-14T05:30:12.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021022+0000", 2024-09-14T05:30:12.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000155699 2024-09-14T05:30:12.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:12.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021169+0000", 2024-09-14T05:30:12.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014661700000000001 2024-09-14T05:30:12.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.338 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:30:12.338 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021891+0000", 2024-09-14T05:30:12.338 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072146700000000005 2024-09-14T05:30:12.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:12.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022124+0000", 2024-09-14T05:30:12.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023354199999999999 2024-09-14T05:30:12.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.339 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:30:12.339 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022164+0000", 2024-09-14T05:30:12.339 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9897000000000001e-05 2024-09-14T05:30:12.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.340 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.340 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022193+0000", 2024-09-14T05:30:12.340 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9050000000000001e-05 2024-09-14T05:30:12.340 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.340 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.340 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.340 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:1989 3.1f e30/24 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head v 30'29, mlcod=30'27)", 2024-09-14T05:30:12.340 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:30:12.340 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.738330744000002, 2024-09-14T05:30:12.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.064242492999999998, 2024-09-14T05:30:12.341 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.341 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.341 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:12.341 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.341 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:30:12.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.341 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.341 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:30:12.341 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593336+0000", 2024-09-14T05:30:12.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3909999999999999e-06 2024-09-14T05:30:12.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616632+0000", 2024-09-14T05:30:12.342 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.023296598000000002 2024-09-14T05:30:12.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.342 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.342 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616657+0000", 2024-09-14T05:30:12.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5228e-05 2024-09-14T05:30:12.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616684+0000", 2024-09-14T05:30:12.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6441999999999999e-05 2024-09-14T05:30:12.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.343 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616755+0000", 2024-09-14T05:30:12.343 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1452000000000006e-05 2024-09-14T05:30:12.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.343 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616780+0000", 2024-09-14T05:30:12.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4436000000000001e-05 2024-09-14T05:30:12.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:30:12.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657527+0000", 2024-09-14T05:30:12.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.040747323000000002 2024-09-14T05:30:12.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.344 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.344 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657577+0000", 2024-09-14T05:30:12.344 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9623000000000002e-05 2024-09-14T05:30:12.344 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.344 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.345 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.345 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70397 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4958208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.345 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.370202+0000", 2024-09-14T05:30:12.345 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.961463028000001, 2024-09-14T05:30:12.345 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030143445000000001, 2024-09-14T05:30:12.345 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.345 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.345 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.345 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.345 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.345 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.346 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70397 2024-09-14T05:30:12.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.346 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.346 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370202+0000", 2024-09-14T05:30:12.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.346 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.346 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370202+0000", 2024-09-14T05:30:12.346 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.347 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.347 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370202+0000", 2024-09-14T05:30:12.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3000000000000002e-07 2024-09-14T05:30:12.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.347 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.347 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370204+0000", 2024-09-14T05:30:12.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7799999999999999e-06 2024-09-14T05:30:12.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.347 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.347 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370209+0000", 2024-09-14T05:30:12.347 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.169e-06 2024-09-14T05:30:12.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.348 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370214+0000", 2024-09-14T05:30:12.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2299999999999999e-06 2024-09-14T05:30:12.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.348 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400201+0000", 2024-09-14T05:30:12.348 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029986987999999999 2024-09-14T05:30:12.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.348 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.348 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400266+0000", 2024-09-14T05:30:12.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5061999999999995e-05 2024-09-14T05:30:12.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.349 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.349 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400345+0000", 2024-09-14T05:30:12.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8886000000000003e-05 2024-09-14T05:30:12.349 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.349 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.349 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.349 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70400 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4990976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.349 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.370229+0000", 2024-09-14T05:30:12.349 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.961435719000001, 2024-09-14T05:30:12.349 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030287287, 2024-09-14T05:30:12.350 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.350 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.350 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.350 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.350 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.350 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.350 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70400 2024-09-14T05:30:12.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.350 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.350 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.350 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370229+0000", 2024-09-14T05:30:12.350 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.351 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.351 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370229+0000", 2024-09-14T05:30:12.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.351 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.351 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370230+0000", 2024-09-14T05:30:12.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3000000000000001e-07 2024-09-14T05:30:12.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.351 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.351 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370232+0000", 2024-09-14T05:30:12.351 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0209999999999998e-06 2024-09-14T05:30:12.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370238+0000", 2024-09-14T05:30:12.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9340000000000003e-06 2024-09-14T05:30:12.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370242+0000", 2024-09-14T05:30:12.352 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.566e-06 2024-09-14T05:30:12.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.352 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.352 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400373+0000", 2024-09-14T05:30:12.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030131116999999999 2024-09-14T05:30:12.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400439+0000", 2024-09-14T05:30:12.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6038999999999996e-05 2024-09-14T05:30:12.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.353 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.353 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400516+0000", 2024-09-14T05:30:12.353 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7180000000000003e-05 2024-09-14T05:30:12.353 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.353 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.353 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.354 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70413 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4960256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.354 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.370249+0000", 2024-09-14T05:30:12.354 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.961415868, 2024-09-14T05:30:12.354 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030421191, 2024-09-14T05:30:12.354 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.354 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.354 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.354 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.354 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.354 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.354 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70413 2024-09-14T05:30:12.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.355 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370249+0000", 2024-09-14T05:30:12.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370249+0000", 2024-09-14T05:30:12.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.355 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.355 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370249+0000", 2024-09-14T05:30:12.355 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9400000000000001e-07 2024-09-14T05:30:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370251+0000", 2024-09-14T05:30:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.013e-06 2024-09-14T05:30:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370258+0000", 2024-09-14T05:30:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0809999999999999e-06 2024-09-14T05:30:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370262+0000", 2024-09-14T05:30:12.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7230000000000004e-06 2024-09-14T05:30:12.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400543+0000", 2024-09-14T05:30:12.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030281129 2024-09-14T05:30:12.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.357 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400593+0000", 2024-09-14T05:30:12.357 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9645000000000003e-05 2024-09-14T05:30:12.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.357 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.358 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400670+0000", 2024-09-14T05:30:12.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7206000000000006e-05 2024-09-14T05:30:12.358 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.358 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.358 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.358 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70416 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4993024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.358 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.370268+0000", 2024-09-14T05:30:12.358 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.961397087999998, 2024-09-14T05:30:12.358 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030571867999999999, 2024-09-14T05:30:12.358 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.358 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.358 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.359 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.359 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.359 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.359 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70416 2024-09-14T05:30:12.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.359 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370268+0000", 2024-09-14T05:30:12.359 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.359 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.359 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370268+0000", 2024-09-14T05:30:12.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370268+0000", 2024-09-14T05:30:12.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7e-07 2024-09-14T05:30:12.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.360 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.360 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370270+0000", 2024-09-14T05:30:12.360 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8279999999999999e-06 2024-09-14T05:30:12.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370275+0000", 2024-09-14T05:30:12.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4110000000000004e-06 2024-09-14T05:30:12.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370280+0000", 2024-09-14T05:30:12.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6940000000000003e-06 2024-09-14T05:30:12.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.361 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.361 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400698+0000", 2024-09-14T05:30:12.361 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030418207999999999 2024-09-14T05:30:12.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.362 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.362 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400764+0000", 2024-09-14T05:30:12.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5717999999999997e-05 2024-09-14T05:30:12.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.362 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.362 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400840+0000", 2024-09-14T05:30:12.362 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5638999999999999e-05 2024-09-14T05:30:12.362 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.362 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.362 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.363 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70429 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4962304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.363 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.370286+0000", 2024-09-14T05:30:12.363 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.961379314999999, 2024-09-14T05:30:12.363 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030741230000000001, 2024-09-14T05:30:12.363 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.363 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.363 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.363 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.363 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.363 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.363 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70429 2024-09-14T05:30:12.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.363 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370286+0000", 2024-09-14T05:30:12.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370286+0000", 2024-09-14T05:30:12.364 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.364 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.364 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370286+0000", 2024-09-14T05:30:12.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5600000000000001e-07 2024-09-14T05:30:12.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370301+0000", 2024-09-14T05:30:12.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5268999999999999e-05 2024-09-14T05:30:12.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.365 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370308+0000", 2024-09-14T05:30:12.365 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3430000000000001e-06 2024-09-14T05:30:12.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.365 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370313+0000", 2024-09-14T05:30:12.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0579999999999997e-06 2024-09-14T05:30:12.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400868+0000", 2024-09-14T05:30:12.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030555045999999999 2024-09-14T05:30:12.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.366 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.366 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.400940+0000", 2024-09-14T05:30:12.366 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2304000000000004e-05 2024-09-14T05:30:12.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.367 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.367 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401027+0000", 2024-09-14T05:30:12.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6854000000000006e-05 2024-09-14T05:30:12.367 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.367 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.367 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.367 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70432 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4995072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.367 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.370319+0000", 2024-09-14T05:30:12.367 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.961345425000001, 2024-09-14T05:30:12.367 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030873629, 2024-09-14T05:30:12.367 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.367 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.368 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.368 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.368 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.368 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.368 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70432 2024-09-14T05:30:12.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.368 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.368 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.368 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370319+0000", 2024-09-14T05:30:12.368 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370319+0000", 2024-09-14T05:30:12.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370320+0000", 2024-09-14T05:30:12.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2399999999999999e-07 2024-09-14T05:30:12.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.369 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.369 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370322+0000", 2024-09-14T05:30:12.369 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.824e-06 2024-09-14T05:30:12.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.370 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370327+0000", 2024-09-14T05:30:12.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6219999999999997e-06 2024-09-14T05:30:12.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.370 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.370332+0000", 2024-09-14T05:30:12.370 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3080000000000001e-06 2024-09-14T05:30:12.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.370 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.370 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401054+0000", 2024-09-14T05:30:12.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030722791999999999 2024-09-14T05:30:12.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401120+0000", 2024-09-14T05:30:12.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5309999999999996e-05 2024-09-14T05:30:12.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.371 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.371 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401193+0000", 2024-09-14T05:30:12.371 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3349000000000005e-05 2024-09-14T05:30:12.371 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.371 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.372 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.372 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70446 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4947968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.372 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.371258+0000", 2024-09-14T05:30:12.372 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.960406663000001, 2024-09-14T05:30:12.372 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030101513999999999, 2024-09-14T05:30:12.372 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.372 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.372 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.372 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.372 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.372 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.372 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70446 2024-09-14T05:30:12.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.373 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371258+0000", 2024-09-14T05:30:12.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.373 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371258+0000", 2024-09-14T05:30:12.373 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.373 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371259+0000", 2024-09-14T05:30:12.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2200000000000003e-07 2024-09-14T05:30:12.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371263+0000", 2024-09-14T05:30:12.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8940000000000003e-06 2024-09-14T05:30:12.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.374 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.374 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371272+0000", 2024-09-14T05:30:12.374 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6449999999999994e-06 2024-09-14T05:30:12.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371278+0000", 2024-09-14T05:30:12.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6660000000000002e-06 2024-09-14T05:30:12.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401221+0000", 2024-09-14T05:30:12.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029942368 2024-09-14T05:30:12.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.375 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.375 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401285+0000", 2024-09-14T05:30:12.375 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4710999999999998e-05 2024-09-14T05:30:12.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.376 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.376 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401360+0000", 2024-09-14T05:30:12.376 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4407999999999996e-05 2024-09-14T05:30:12.376 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.376 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.376 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.377 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70448 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4980736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.377 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.371289+0000", 2024-09-14T05:30:12.377 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.960375455000001, 2024-09-14T05:30:12.377 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030246810999999998, 2024-09-14T05:30:12.377 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.377 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.377 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.377 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.377 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.377 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.377 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70448 2024-09-14T05:30:12.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.378 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371289+0000", 2024-09-14T05:30:12.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371289+0000", 2024-09-14T05:30:12.378 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.378 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.378 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371290+0000", 2024-09-14T05:30:12.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2800000000000002e-07 2024-09-14T05:30:12.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371292+0000", 2024-09-14T05:30:12.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0049999999999999e-06 2024-09-14T05:30:12.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371297+0000", 2024-09-14T05:30:12.379 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6250000000000004e-06 2024-09-14T05:30:12.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.379 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.379 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371302+0000", 2024-09-14T05:30:12.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9960000000000001e-06 2024-09-14T05:30:12.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401387+0000", 2024-09-14T05:30:12.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030084818999999999 2024-09-14T05:30:12.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.380 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.380 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401451+0000", 2024-09-14T05:30:12.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3331999999999996e-05 2024-09-14T05:30:12.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401536+0000", 2024-09-14T05:30:12.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5606000000000007e-05 2024-09-14T05:30:12.381 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.381 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.381 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.381 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70461 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4950016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.381 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.371310+0000", 2024-09-14T05:30:12.381 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.960355072999999, 2024-09-14T05:30:12.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030393710000000001, 2024-09-14T05:30:12.381 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.381 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.382 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.382 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.382 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.382 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.382 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70461 2024-09-14T05:30:12.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.382 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371310+0000", 2024-09-14T05:30:12.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371310+0000", 2024-09-14T05:30:12.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371310+0000", 2024-09-14T05:30:12.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4000000000000002e-07 2024-09-14T05:30:12.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371312+0000", 2024-09-14T05:30:12.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.04e-06 2024-09-14T05:30:12.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371318+0000", 2024-09-14T05:30:12.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7089999999999999e-06 2024-09-14T05:30:12.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371323+0000", 2024-09-14T05:30:12.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5820000000000003e-06 2024-09-14T05:30:12.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401563+0000", 2024-09-14T05:30:12.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030240466000000001 2024-09-14T05:30:12.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401629+0000", 2024-09-14T05:30:12.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5631999999999998e-05 2024-09-14T05:30:12.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.385 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.385 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401704+0000", 2024-09-14T05:30:12.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4840999999999997e-05 2024-09-14T05:30:12.385 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.385 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.386 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.386 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70464 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4982784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.386 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.371328+0000", 2024-09-14T05:30:12.386 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.960336547000001, 2024-09-14T05:30:12.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030558068000000001, 2024-09-14T05:30:12.386 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.386 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.386 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.386 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.386 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.386 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.386 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70464 2024-09-14T05:30:12.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.387 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371328+0000", 2024-09-14T05:30:12.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371328+0000", 2024-09-14T05:30:12.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371329+0000", 2024-09-14T05:30:12.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0900000000000002e-07 2024-09-14T05:30:12.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371331+0000", 2024-09-14T05:30:12.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.792e-06 2024-09-14T05:30:12.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371335+0000", 2024-09-14T05:30:12.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9300000000000002e-06 2024-09-14T05:30:12.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.371340+0000", 2024-09-14T05:30:12.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.605e-06 2024-09-14T05:30:12.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401730+0000", 2024-09-14T05:30:12.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030390225 2024-09-14T05:30:12.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401800+0000", 2024-09-14T05:30:12.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0166000000000005e-05 2024-09-14T05:30:12.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.401886+0000", 2024-09-14T05:30:12.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5940999999999996e-05 2024-09-14T05:30:12.390 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.390 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.390 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.391 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70609 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4972544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.391 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.375484+0000", 2024-09-14T05:30:12.391 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.956180533000001, 2024-09-14T05:30:12.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.031716407000000002, 2024-09-14T05:30:12.391 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.391 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.391 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.391 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.391 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.391 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.391 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70609 2024-09-14T05:30:12.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.391 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375484+0000", 2024-09-14T05:30:12.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375484+0000", 2024-09-14T05:30:12.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375485+0000", 2024-09-14T05:30:12.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6100000000000001e-07 2024-09-14T05:30:12.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375487+0000", 2024-09-14T05:30:12.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.3700000000000002e-06 2024-09-14T05:30:12.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375494+0000", 2024-09-14T05:30:12.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.737e-06 2024-09-14T05:30:12.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375499+0000", 2024-09-14T05:30:12.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4500000000000003e-06 2024-09-14T05:30:12.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405175+0000", 2024-09-14T05:30:12.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029676022999999999 2024-09-14T05:30:12.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.394 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405243+0000", 2024-09-14T05:30:12.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7620000000000006e-05 2024-09-14T05:30:12.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407201+0000", 2024-09-14T05:30:12.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0019577459999999998 2024-09-14T05:30:12.395 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.395 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.395 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.395 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70612 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 5005312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.395 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.375507+0000", 2024-09-14T05:30:12.395 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.956158214999999, 2024-09-14T05:30:12.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030049329999999999, 2024-09-14T05:30:12.395 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.395 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.395 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.396 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.396 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.396 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.396 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70612 2024-09-14T05:30:12.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.396 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375507+0000", 2024-09-14T05:30:12.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.397 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375507+0000", 2024-09-14T05:30:12.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.397 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375507+0000", 2024-09-14T05:30:12.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:12.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.397 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375509+0000", 2024-09-14T05:30:12.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.967e-06 2024-09-14T05:30:12.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375516+0000", 2024-09-14T05:30:12.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.691e-06 2024-09-14T05:30:12.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375521+0000", 2024-09-14T05:30:12.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2510000000000002e-06 2024-09-14T05:30:12.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405380+0000", 2024-09-14T05:30:12.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029858787000000001 2024-09-14T05:30:12.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405446+0000", 2024-09-14T05:30:12.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5996999999999998e-05 2024-09-14T05:30:12.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.399 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.399 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405556+0000", 2024-09-14T05:30:12.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011021100000000001 2024-09-14T05:30:12.399 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.399 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.399 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.400 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70625 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4974592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.400 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.375529+0000", 2024-09-14T05:30:12.400 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.956136068999999, 2024-09-14T05:30:12.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030229536000000001, 2024-09-14T05:30:12.400 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.400 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.400 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.400 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.400 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.400 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.400 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70625 2024-09-14T05:30:12.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.400 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375529+0000", 2024-09-14T05:30:12.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375529+0000", 2024-09-14T05:30:12.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375529+0000", 2024-09-14T05:30:12.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8299999999999998e-07 2024-09-14T05:30:12.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375531+0000", 2024-09-14T05:30:12.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9659999999999999e-06 2024-09-14T05:30:12.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375538+0000", 2024-09-14T05:30:12.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5089999999999999e-06 2024-09-14T05:30:12.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375543+0000", 2024-09-14T05:30:12.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3959999999999997e-06 2024-09-14T05:30:12.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.403 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405581+0000", 2024-09-14T05:30:12.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030037828999999999 2024-09-14T05:30:12.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.403 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405648+0000", 2024-09-14T05:30:12.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7194999999999996e-05 2024-09-14T05:30:12.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.404 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.404 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405758+0000", 2024-09-14T05:30:12.404 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000110258 2024-09-14T05:30:12.404 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.404 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.404 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.404 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70628 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 5007360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.404 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.375620+0000", 2024-09-14T05:30:12.404 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.956044924, 2024-09-14T05:30:12.404 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030354611, 2024-09-14T05:30:12.404 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.405 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.405 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.405 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.405 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.405 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.405 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70628 2024-09-14T05:30:12.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.405 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375620+0000", 2024-09-14T05:30:12.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375620+0000", 2024-09-14T05:30:12.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375621+0000", 2024-09-14T05:30:12.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9800000000000003e-07 2024-09-14T05:30:12.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375624+0000", 2024-09-14T05:30:12.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4429999999999999e-06 2024-09-14T05:30:12.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375633+0000", 2024-09-14T05:30:12.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.6370000000000005e-06 2024-09-14T05:30:12.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375639+0000", 2024-09-14T05:30:12.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3149999999999997e-06 2024-09-14T05:30:12.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405784+0000", 2024-09-14T05:30:12.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030145110999999999 2024-09-14T05:30:12.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405851+0000", 2024-09-14T05:30:12.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7279000000000004e-05 2024-09-14T05:30:12.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.408 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.408 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.405975+0000", 2024-09-14T05:30:12.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012312799999999999 2024-09-14T05:30:12.408 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.408 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.408 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.409 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70641 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4976640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.409 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.375666+0000", 2024-09-14T05:30:12.409 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.955999342999998, 2024-09-14T05:30:12.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030512713, 2024-09-14T05:30:12.409 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.409 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.409 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.409 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.409 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.409 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.409 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70641 2024-09-14T05:30:12.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.409 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375666+0000", 2024-09-14T05:30:12.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375666+0000", 2024-09-14T05:30:12.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375666+0000", 2024-09-14T05:30:12.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2500000000000005e-07 2024-09-14T05:30:12.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375669+0000", 2024-09-14T05:30:12.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9220000000000001e-06 2024-09-14T05:30:12.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375677+0000", 2024-09-14T05:30:12.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0809999999999994e-06 2024-09-14T05:30:12.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375683+0000", 2024-09-14T05:30:12.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6069999999999998e-06 2024-09-14T05:30:12.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.412 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.406000+0000", 2024-09-14T05:30:12.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030316777 2024-09-14T05:30:12.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.412 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.406072+0000", 2024-09-14T05:30:12.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1983000000000005e-05 2024-09-14T05:30:12.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.406178+0000", 2024-09-14T05:30:12.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000106718 2024-09-14T05:30:12.413 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.413 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.413 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.413 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70644 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 5009408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.413 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.375729+0000", 2024-09-14T05:30:12.414 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.955936114, 2024-09-14T05:30:12.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030657856000000001, 2024-09-14T05:30:12.414 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.414 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.414 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.414 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.414 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.414 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.414 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70644 2024-09-14T05:30:12.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.414 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375729+0000", 2024-09-14T05:30:12.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.415 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375729+0000", 2024-09-14T05:30:12.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.415 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375729+0000", 2024-09-14T05:30:12.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6599999999999996e-07 2024-09-14T05:30:12.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375732+0000", 2024-09-14T05:30:12.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8499999999999998e-06 2024-09-14T05:30:12.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375743+0000", 2024-09-14T05:30:12.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0352e-05 2024-09-14T05:30:12.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.375748+0000", 2024-09-14T05:30:12.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7899999999999996e-06 2024-09-14T05:30:12.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.406203+0000", 2024-09-14T05:30:12.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030454108000000001 2024-09-14T05:30:12.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.406272+0000", 2024-09-14T05:30:12.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8979999999999993e-05 2024-09-14T05:30:12.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.406387+0000", 2024-09-14T05:30:12.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00011511 2024-09-14T05:30:12.418 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.418 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.418 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.418 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70689 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4966400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.418 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.377471+0000", 2024-09-14T05:30:12.418 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.954193953000001, 2024-09-14T05:30:12.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029995048999999999, 2024-09-14T05:30:12.418 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.418 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.418 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.418 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.419 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.419 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.419 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70689 2024-09-14T05:30:12.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.419 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377471+0000", 2024-09-14T05:30:12.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377471+0000", 2024-09-14T05:30:12.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377471+0000", 2024-09-14T05:30:12.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8099999999999998e-07 2024-09-14T05:30:12.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377474+0000", 2024-09-14T05:30:12.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1780000000000002e-06 2024-09-14T05:30:12.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.421 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377480+0000", 2024-09-14T05:30:12.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0769999999999996e-06 2024-09-14T05:30:12.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.421 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.421 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377484+0000", 2024-09-14T05:30:12.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8040000000000001e-06 2024-09-14T05:30:12.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.421 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.421 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407257+0000", 2024-09-14T05:30:12.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.029773079000000001 2024-09-14T05:30:12.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407324+0000", 2024-09-14T05:30:12.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6721e-05 2024-09-14T05:30:12.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407466+0000", 2024-09-14T05:30:12.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014180900000000001 2024-09-14T05:30:12.422 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.422 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.422 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.422 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:70692 3.1c 3:38554ada:::rbd_data.112d641bd628.0000000000000001:10 [read 4999168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.423 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.377495+0000", 2024-09-14T05:30:12.423 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.954170310999999, 2024-09-14T05:30:12.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030228279, 2024-09-14T05:30:12.423 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.423 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.423 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.423 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.423 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.423 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.423 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 70692 2024-09-14T05:30:12.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.423 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377495+0000", 2024-09-14T05:30:12.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377495+0000", 2024-09-14T05:30:12.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377495+0000", 2024-09-14T05:30:12.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7700000000000005e-07 2024-09-14T05:30:12.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377497+0000", 2024-09-14T05:30:12.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.181e-06 2024-09-14T05:30:12.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377503+0000", 2024-09-14T05:30:12.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2550000000000003e-06 2024-09-14T05:30:12.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.377508+0000", 2024-09-14T05:30:12.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.967e-06 2024-09-14T05:30:12.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.426 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407517+0000", 2024-09-14T05:30:12.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.030008970999999999 2024-09-14T05:30:12.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.426 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407585+0000", 2024-09-14T05:30:12.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7452000000000004e-05 2024-09-14T05:30:12.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.426 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.407723+0000", 2024-09-14T05:30:12.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013797599999999999 2024-09-14T05:30:12.426 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.427 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.427 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.427 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.427 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.427 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:12.428 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:30:12.429 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone32 from 0x984762f (overlap 0x385ec9) to 0x21d278 2024-09-14T05:30:12.429 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:12.427+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:12.430 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-14T05:30:12.437 INFO:teuthology.orchestra.run.smithi071.stdout:4088 read 0x72895f8 thru 0x728b1d9 (0x1be2 bytes) 2024-09-14T05:30:12.437 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:12.446 INFO:teuthology.orchestra.run.smithi071.stdout:4092 trunc from 0x984762f to 0x2ab5b3c 2024-09-14T05:30:12.487 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:12.488 INFO:teuthology.orchestra.run.smithi071.stdout:4095 trunc from 0x2ab5b3c to 0x3ed097a 2024-09-14T05:30:12.494 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:12.494 INFO:teuthology.orchestra.run.smithi071.stdout:4097 writesame 0x1a5d747 thru 0x1a62f55 data_size 0x1(0x580f bytes) 2024-09-14T05:30:12.513 INFO:teuthology.orchestra.run.smithi071.stdout:4098 write 0xab853a6 thru 0xab92f47 (0xdba2 bytes) 2024-09-14T05:30:12.522 INFO:teuthology.orchestra.run.smithi071.stdout:4102 read 0x54ed557 thru 0x54f97dd (0xc287 bytes) 2024-09-14T05:30:12.522 INFO:teuthology.orchestra.run.smithi071.stdout:4104 punch from 0x1a473ca to 0x1a4a106, (0x2d3c bytes) 2024-09-14T05:30:12.523 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:12.523 INFO:teuthology.orchestra.run.smithi071.stdout:4108 punch from 0x901b3b6 to 0x9027702, (0xc34c bytes) 2024-09-14T05:30:12.523 INFO:teuthology.orchestra.run.smithi071.stdout:4109 write 0x1436610 thru 0x143d555 (0x6f46 bytes) 2024-09-14T05:30:12.529 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:12.528+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:12.539 INFO:teuthology.orchestra.run.smithi071.stdout:4112 punch from 0x464032d to 0x464d36b, (0xd03e bytes) 2024-09-14T05:30:12.539 INFO:teuthology.orchestra.run.smithi071.stdout:4114 read 0x30ae9a4 thru 0x30bd314 (0xe971 bytes) 2024-09-14T05:30:12.540 INFO:teuthology.orchestra.run.smithi071.stdout:4115 trunc from 0xab92f48 to 0xb29f5e7 2024-09-14T05:30:12.545 INFO:teuthology.orchestra.run.smithi071.stdout:4117 write 0x30bce43 thru 0x30c8313 (0xb4d1 bytes) 2024-09-14T05:30:12.547 INFO:teuthology.orchestra.run.smithi071.stdout:4118 writesame 0x3be5935 thru 0x3beed45 data_size 0x5(0x9411 bytes) 2024-09-14T05:30:12.553 INFO:teuthology.orchestra.run.smithi071.stdout:4119 read 0x55ed851 thru 0x55f5057 (0x7807 bytes) 2024-09-14T05:30:12.553 INFO:teuthology.orchestra.run.smithi071.stdout:4120 punch from 0x3fcb728 to 0x3fd6229, (0xab01 bytes) 2024-09-14T05:30:12.553 INFO:teuthology.orchestra.run.smithi071.stdout:4122 write 0x557efa0 thru 0x558475b (0x57bc bytes) 2024-09-14T05:30:12.559 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:12.559 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:12.559 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:12.559 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:12.560 INFO:teuthology.orchestra.run.smithi071.stdout:4124 trunc from 0xb29f5e7 to 0xdeac73c 2024-09-14T05:30:12.566 INFO:teuthology.orchestra.run.smithi071.stdout:4125 read 0xa33b77 thru 0xa33bd9 (0x63 bytes) 2024-09-14T05:30:12.566 INFO:teuthology.orchestra.run.smithi071.stdout:4128 writesame 0x2d21292 thru 0x2d2540a data_size 0x25(0x4179 bytes) 2024-09-14T05:30:12.568 INFO:teuthology.orchestra.run.smithi071.stdout:4129 writesame 0x56cea6c thru 0x56d6b01 data_size 0x2(0x8096 bytes) 2024-09-14T05:30:12.570 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:30:12.576 INFO:teuthology.orchestra.run.smithi071.stdout:4130 read 0x2587437 thru 0x2593dea (0xc9b4 bytes) 2024-09-14T05:30:12.576 INFO:teuthology.orchestra.run.smithi071.stdout:4131 read 0x290c88a thru 0x29138de (0x7055 bytes) 2024-09-14T05:30:12.578 INFO:teuthology.orchestra.run.smithi071.stdout:4132 writesame 0x4401dca thru 0x441091e data_size 0x1(0xeb55 bytes) 2024-09-14T05:30:12.594 INFO:teuthology.orchestra.run.smithi071.stdout:4135 write 0x89be2cf thru 0x89c95dd (0xb30f bytes) 2024-09-14T05:30:12.601 INFO:teuthology.orchestra.run.smithi071.stdout:4137 read 0x43dcd3a thru 0x43e4e58 (0x811f bytes) 2024-09-14T05:30:12.601 INFO:teuthology.orchestra.run.smithi071.stdout:4138 read 0x7d31aaa thru 0x7d41971 (0xfec8 bytes) 2024-09-14T05:30:12.602 INFO:teuthology.orchestra.run.smithi071.stdout:4139 trunc from 0xdeac73c to 0xe968305 2024-09-14T05:30:12.607 INFO:teuthology.orchestra.run.smithi071.stdout:4141 punch from 0x106db5c to 0x1079182, (0xb626 bytes) 2024-09-14T05:30:12.607 INFO:teuthology.orchestra.run.smithi071.stdout:4142 punch from 0xe70d11 to 0xe789e9, (0x7cd8 bytes) 2024-09-14T05:30:12.607 INFO:teuthology.orchestra.run.smithi071.stdout:4143 writesame 0x24aabf2 thru 0x24b06f6 data_size 0x9(0x5b05 bytes) 2024-09-14T05:30:12.610 INFO:teuthology.orchestra.run.smithi071.stdout:4144 write 0x54b6b65 thru 0x54c68d8 (0xfd74 bytes) 2024-09-14T05:30:12.624 INFO:teuthology.orchestra.run.smithi071.stdout:4146 trunc from 0xe968305 to 0x6ce79c7 2024-09-14T05:30:12.630 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:12.629+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:12.648 INFO:teuthology.orchestra.run.smithi071.stdout:4147 write 0xd9acdd8 thru 0xd9bb3d1 (0xe5fa bytes) 2024-09-14T05:30:12.654 INFO:teuthology.orchestra.run.smithi071.stdout:4149 punch from 0xc3240 to 0xcd7f4, (0xa5b4 bytes) 2024-09-14T05:30:12.655 INFO:teuthology.orchestra.run.smithi071.stdout:4150 write 0x99d3e42 thru 0x99d9059 (0x5218 bytes) 2024-09-14T05:30:12.659 INFO:teuthology.orchestra.run.smithi071.stdout:4151 write 0x1a71b42 thru 0x1a755ee (0x3aad bytes) 2024-09-14T05:30:12.662 INFO:teuthology.orchestra.run.smithi071.stdout:4152 read 0x5ff43c8 thru 0x5ff62d6 (0x1f0f bytes) 2024-09-14T05:30:12.663 INFO:teuthology.orchestra.run.smithi071.stdout:4153 punch from 0x938ad66 to 0x939943a, (0xe6d4 bytes) 2024-09-14T05:30:12.663 INFO:teuthology.orchestra.run.smithi071.stdout:4155 write 0xab7f0c thru 0xac4bee (0xcce3 bytes) 2024-09-14T05:30:12.690 INFO:teuthology.orchestra.run.smithi071.stdout:4156 punch from 0x629fb0 to 0x63684c, (0xc89c bytes) 2024-09-14T05:30:12.690 INFO:teuthology.orchestra.run.smithi071.stdout:4157 punch from 0x5530e9a to 0x553bc38, (0xad9e bytes) 2024-09-14T05:30:12.690 INFO:teuthology.orchestra.run.smithi071.stdout:4159 read 0x1ada873 thru 0x1adcb74 (0x2302 bytes) 2024-09-14T05:30:12.690 INFO:teuthology.orchestra.run.smithi071.stdout:4160 read 0xa07f28a thru 0xa08c725 (0xd49c bytes) 2024-09-14T05:30:12.690 INFO:teuthology.orchestra.run.smithi071.stdout:4161 read 0x6192c2e thru 0x61a2670 (0xfa43 bytes) 2024-09-14T05:30:12.691 INFO:teuthology.orchestra.run.smithi071.stdout:4162 write 0x45ddb4 thru 0x46507c (0x72c9 bytes) 2024-09-14T05:30:12.693 INFO:teuthology.orchestra.run.smithi071.stdout:4163 read 0x9db8914 thru 0x9dbfa5d (0x714a bytes) 2024-09-14T05:30:12.693 INFO:teuthology.orchestra.run.smithi071.stdout:4164 read 0x80e1f91 thru 0x80e318a (0x11fa bytes) 2024-09-14T05:30:12.693 INFO:teuthology.orchestra.run.smithi071.stdout:4165 write 0xb07cd9e thru 0xb085e98 (0x90fb bytes) 2024-09-14T05:30:12.700 INFO:teuthology.orchestra.run.smithi071.stdout:4166 read 0xc4c0fa9 thru 0xc4c5b4b (0x4ba3 bytes) 2024-09-14T05:30:12.700 INFO:teuthology.orchestra.run.smithi071.stdout:4167 read 0x9131ce3 thru 0x913ff95 (0xe2b3 bytes) 2024-09-14T05:30:12.700 INFO:teuthology.orchestra.run.smithi071.stdout:4169 write 0x5d2147d thru 0x5d24dbc (0x3940 bytes) 2024-09-14T05:30:12.700 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:12.700 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:12.701 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:12.701 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:12.701 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:12.702 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:12.702 INFO:teuthology.orchestra.run.smithi071.stdout:4171 write 0x2a4c271 thru 0x2a4c914 (0x6a4 bytes) 2024-09-14T05:30:12.704 INFO:teuthology.orchestra.run.smithi071.stdout:4172 read 0x2f74a96 thru 0x2f825d3 (0xdb3e bytes) 2024-09-14T05:30:12.704 INFO:teuthology.orchestra.run.smithi071.stdout:4173 write 0x89a2a8a thru 0x89aaa31 (0x7fa8 bytes) 2024-09-14T05:30:12.709 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:12.709 INFO:teuthology.orchestra.run.smithi071.stdout:4175 write 0xba812f1 thru 0xba8b68e (0xa39e bytes) 2024-09-14T05:30:12.712 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:30:12.716 INFO:teuthology.orchestra.run.smithi071.stdout:4176 read 0x533f1ee thru 0x534275c (0x356f bytes) 2024-09-14T05:30:12.717 INFO:teuthology.orchestra.run.smithi071.stdout:4177 write 0x18ed885 thru 0x18f5fd2 (0x874e bytes) 2024-09-14T05:30:12.719 INFO:teuthology.orchestra.run.smithi071.stdout:4178 writesame 0xae4d568 thru 0xae4da17 data_size 0x19(0x4b0 bytes) 2024-09-14T05:30:12.721 INFO:teuthology.orchestra.run.smithi071.stdout:4179 writesame 0x84fcf8e thru 0x84fe1a3 data_size 0x5(0x1216 bytes) 2024-09-14T05:30:12.722 INFO:teuthology.orchestra.run.smithi071.stdout:4180 read 0x8512f3e thru 0x8519cca (0x6d8d bytes) 2024-09-14T05:30:12.723 INFO:teuthology.orchestra.run.smithi071.stdout:4181 punch from 0x7d1b19f to 0x7d265ee, (0xb44f bytes) 2024-09-14T05:30:12.728 INFO:teuthology.orchestra.run.smithi071.stdout:4182 trunc from 0xd9bb3d2 to 0x90966c7 2024-09-14T05:30:12.730 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:12.729+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:12.751 INFO:teuthology.orchestra.run.smithi071.stdout:4184 write 0xbab5366 thru 0xbac247b (0xd116 bytes) 2024-09-14T05:30:12.761 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:12.764 INFO:teuthology.orchestra.run.smithi071.stdout:4186 trunc from 0xbac247c to 0xd7f6463 2024-09-14T05:30:12.769 INFO:teuthology.orchestra.run.smithi071.stdout:4187 read 0x40c7d1f thru 0x40d74fb (0xf7dd bytes) 2024-09-14T05:30:12.770 INFO:teuthology.orchestra.run.smithi071.stdout:4190 read 0x508e8bb thru 0x5095442 (0x6b88 bytes) 2024-09-14T05:30:12.784 INFO:teuthology.orchestra.run.smithi071.stdout:4192 trunc from 0xd7f6463 to 0x20b50cd 2024-09-14T05:30:12.803 INFO:teuthology.orchestra.run.smithi071.stdout:4195 read 0x33530f thru 0x34148b (0xc17d bytes) 2024-09-14T05:30:12.805 INFO:teuthology.orchestra.run.smithi071.stdout:4196 punch from 0x1de3e0f to 0x1df2fe0, (0xf1d1 bytes) 2024-09-14T05:30:12.814 INFO:teuthology.orchestra.run.smithi071.stdout:4197 write 0x59d9d8e thru 0x59e92bd (0xf530 bytes) 2024-09-14T05:30:12.826 INFO:teuthology.orchestra.run.smithi071.stdout:4198 read 0x3cbb575 thru 0x3cc05a8 (0x5034 bytes) 2024-09-14T05:30:12.830 INFO:teuthology.orchestra.run.smithi071.stdout:4199 trunc from 0x59e92be to 0x29e3704 2024-09-14T05:30:12.830 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:12.830+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:12.842 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:12.843 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:12.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:12.843 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:12.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.843 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73757 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2816000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.843 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:30:12.843 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.281449886000001, 2024-09-14T05:30:12.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044614540000000001, 2024-09-14T05:30:12.843 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.843 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.843 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.843 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.844 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.844 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.844 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73757 2024-09-14T05:30:12.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.844 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:30:12.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:30:12.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559679+0000", 2024-09-14T05:30:12.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4000000000000002e-07 2024-09-14T05:30:12.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559681+0000", 2024-09-14T05:30:12.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1780000000000002e-06 2024-09-14T05:30:12.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559687+0000", 2024-09-14T05:30:12.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0270000000000001e-06 2024-09-14T05:30:12.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559692+0000", 2024-09-14T05:30:12.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1030000000000001e-06 2024-09-14T05:30:12.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604180+0000", 2024-09-14T05:30:12.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044487480000000003 2024-09-14T05:30:12.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604224+0000", 2024-09-14T05:30:12.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4357999999999998e-05 2024-09-14T05:30:12.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604293+0000", 2024-09-14T05:30:12.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8954000000000004e-05 2024-09-14T05:30:12.847 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.847 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.847 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.848 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73760 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2848768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.848 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:12.848 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.281429781, 2024-09-14T05:30:12.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.04469911, 2024-09-14T05:30:12.848 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.848 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.848 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.848 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.848 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.848 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.848 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73760 2024-09-14T05:30:12.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.848 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:12.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:12.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:12.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:12.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559701+0000", 2024-09-14T05:30:12.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1909999999999999e-06 2024-09-14T05:30:12.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559708+0000", 2024-09-14T05:30:12.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7279999999999998e-06 2024-09-14T05:30:12.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559713+0000", 2024-09-14T05:30:12.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8389999999999997e-06 2024-09-14T05:30:12.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604310+0000", 2024-09-14T05:30:12.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044597664000000002 2024-09-14T05:30:12.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604352+0000", 2024-09-14T05:30:12.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1236000000000003e-05 2024-09-14T05:30:12.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604398+0000", 2024-09-14T05:30:12.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6091000000000003e-05 2024-09-14T05:30:12.852 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.852 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.852 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.852 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73773 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2801664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.852 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:12.852 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.281409612000001, 2024-09-14T05:30:12.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941026000000002, 2024-09-14T05:30:12.853 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.853 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.853 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.853 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.853 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.853 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.853 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73773 2024-09-14T05:30:12.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.853 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:12.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:12.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:12.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8599999999999999e-07 2024-09-14T05:30:12.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559721+0000", 2024-09-14T05:30:12.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0540000000000002e-06 2024-09-14T05:30:12.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559729+0000", 2024-09-14T05:30:12.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1359999999999997e-06 2024-09-14T05:30:12.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559734+0000", 2024-09-14T05:30:12.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2140000000000004e-06 2024-09-14T05:30:12.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604441+0000", 2024-09-14T05:30:12.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044706828999999997 2024-09-14T05:30:12.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604522+0000", 2024-09-14T05:30:12.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0603000000000001e-05 2024-09-14T05:30:12.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604660+0000", 2024-09-14T05:30:12.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000137804 2024-09-14T05:30:12.856 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.856 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.856 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.857 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73777 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2834432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.857 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:12.857 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.281387210999998, 2024-09-14T05:30:12.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045116270999999999, 2024-09-14T05:30:12.857 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.857 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.857 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.857 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.857 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.857 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.857 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73777 2024-09-14T05:30:12.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.858 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:12.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:12.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559742+0000", 2024-09-14T05:30:12.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4099999999999999e-07 2024-09-14T05:30:12.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559744+0000", 2024-09-14T05:30:12.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0040000000000002e-06 2024-09-14T05:30:12.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559749+0000", 2024-09-14T05:30:12.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9739999999999999e-06 2024-09-14T05:30:12.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559754+0000", 2024-09-14T05:30:12.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9540000000000003e-06 2024-09-14T05:30:12.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604695+0000", 2024-09-14T05:30:12.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941003 2024-09-14T05:30:12.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604766+0000", 2024-09-14T05:30:12.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0296999999999995e-05 2024-09-14T05:30:12.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604857+0000", 2024-09-14T05:30:12.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1598e-05 2024-09-14T05:30:12.861 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.861 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.861 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.861 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73790 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.862 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:12.862 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.281367276000001, 2024-09-14T05:30:12.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316437000000001, 2024-09-14T05:30:12.862 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.862 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.862 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.862 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.862 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.862 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.862 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73790 2024-09-14T05:30:12.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.863 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:12.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:12.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:12.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7599999999999998e-07 2024-09-14T05:30:12.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559763+0000", 2024-09-14T05:30:12.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0880000000000002e-06 2024-09-14T05:30:12.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559769+0000", 2024-09-14T05:30:12.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6740000000000002e-06 2024-09-14T05:30:12.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559774+0000", 2024-09-14T05:30:12.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6110000000000004e-06 2024-09-14T05:30:12.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604905+0000", 2024-09-14T05:30:12.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045130827999999998 2024-09-14T05:30:12.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604982+0000", 2024-09-14T05:30:12.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7410999999999995e-05 2024-09-14T05:30:12.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605077+0000", 2024-09-14T05:30:12.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5449e-05 2024-09-14T05:30:12.866 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.866 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.866 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.866 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73793 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2836480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.866 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:12.866 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.281348072, 2024-09-14T05:30:12.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045441212000000002, 2024-09-14T05:30:12.867 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.867 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.867 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.867 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.867 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.867 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.867 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73793 2024-09-14T05:30:12.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.867 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:12.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:12.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559781+0000", 2024-09-14T05:30:12.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2899999999999999e-07 2024-09-14T05:30:12.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559783+0000", 2024-09-14T05:30:12.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.075e-06 2024-09-14T05:30:12.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559788+0000", 2024-09-14T05:30:12.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5369999999999997e-06 2024-09-14T05:30:12.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559793+0000", 2024-09-14T05:30:12.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:30:12.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.870 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605109+0000", 2024-09-14T05:30:12.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316229999999999 2024-09-14T05:30:12.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.870 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.870 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605168+0000", 2024-09-14T05:30:12.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8949000000000001e-05 2024-09-14T05:30:12.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.870 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.870 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605221+0000", 2024-09-14T05:30:12.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3260000000000002e-05 2024-09-14T05:30:12.870 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.870 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.871 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.871 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73806 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.871 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:12.871 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.281329446000001, 2024-09-14T05:30:12.871 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045526189000000002, 2024-09-14T05:30:12.871 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.871 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.871 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.871 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.871 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.871 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.872 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73806 2024-09-14T05:30:12.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.872 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.872 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:12.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.872 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:12.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.873 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.873 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:12.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:30:12.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.873 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.873 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559801+0000", 2024-09-14T05:30:12.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.995e-06 2024-09-14T05:30:12.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.873 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.873 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559807+0000", 2024-09-14T05:30:12.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5450000000000003e-06 2024-09-14T05:30:12.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559811+0000", 2024-09-14T05:30:12.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5090000000000004e-06 2024-09-14T05:30:12.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605237+0000", 2024-09-14T05:30:12.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045426133 2024-09-14T05:30:12.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.875 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605278+0000", 2024-09-14T05:30:12.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1084999999999997e-05 2024-09-14T05:30:12.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.875 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.875 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605325+0000", 2024-09-14T05:30:12.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6582999999999998e-05 2024-09-14T05:30:12.875 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.875 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.875 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.875 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73809 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2838528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.875 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:12.876 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.281305940999999, 2024-09-14T05:30:12.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045596062999999999, 2024-09-14T05:30:12.876 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.876 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.876 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.876 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.876 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.876 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.876 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73809 2024-09-14T05:30:12.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.876 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:12.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:12.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559823+0000", 2024-09-14T05:30:12.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.08e-07 2024-09-14T05:30:12.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559825+0000", 2024-09-14T05:30:12.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.26e-06 2024-09-14T05:30:12.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559831+0000", 2024-09-14T05:30:12.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6690000000000003e-06 2024-09-14T05:30:12.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559835+0000", 2024-09-14T05:30:12.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5120000000000002e-06 2024-09-14T05:30:12.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605342+0000", 2024-09-14T05:30:12.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045506957000000001 2024-09-14T05:30:12.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605375+0000", 2024-09-14T05:30:12.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3250999999999997e-05 2024-09-14T05:30:12.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.880 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605418+0000", 2024-09-14T05:30:12.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3006000000000001e-05 2024-09-14T05:30:12.880 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.880 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.880 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.880 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73822 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2807808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.880 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:12.880 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.281278287999999, 2024-09-14T05:30:12.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045661405000000002, 2024-09-14T05:30:12.880 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.880 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.881 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.881 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.881 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.881 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.881 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73822 2024-09-14T05:30:12.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.881 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:12.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:12.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559851+0000", 2024-09-14T05:30:12.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4000000000000002e-07 2024-09-14T05:30:12.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559853+0000", 2024-09-14T05:30:12.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5380000000000001e-06 2024-09-14T05:30:12.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559859+0000", 2024-09-14T05:30:12.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0619999999999997e-06 2024-09-14T05:30:12.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559864+0000", 2024-09-14T05:30:12.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.977e-06 2024-09-14T05:30:12.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605435+0000", 2024-09-14T05:30:12.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045570672 2024-09-14T05:30:12.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605467+0000", 2024-09-14T05:30:12.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2486e-05 2024-09-14T05:30:12.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605511+0000", 2024-09-14T05:30:12.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4129999999999999e-05 2024-09-14T05:30:12.884 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.884 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.884 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.885 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73825 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2840576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.885 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:12.885 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.281254923999999, 2024-09-14T05:30:12.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045728694, 2024-09-14T05:30:12.885 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.885 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.885 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.885 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.885 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.885 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.885 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73825 2024-09-14T05:30:12.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.885 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:12.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:12.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559874+0000", 2024-09-14T05:30:12.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5400000000000002e-07 2024-09-14T05:30:12.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559876+0000", 2024-09-14T05:30:12.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.131e-06 2024-09-14T05:30:12.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559888+0000", 2024-09-14T05:30:12.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2216e-05 2024-09-14T05:30:12.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559893+0000", 2024-09-14T05:30:12.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6539999999999998e-06 2024-09-14T05:30:12.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605526+0000", 2024-09-14T05:30:12.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045633119 2024-09-14T05:30:12.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605560+0000", 2024-09-14T05:30:12.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3825999999999998e-05 2024-09-14T05:30:12.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605602+0000", 2024-09-14T05:30:12.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2293999999999999e-05 2024-09-14T05:30:12.889 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.889 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.889 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.889 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74023 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2863104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.889 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:12.889 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.280068949, 2024-09-14T05:30:12.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044635385999999999, 2024-09-14T05:30:12.889 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.889 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.889 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.889 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.889 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.890 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.890 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74023 2024-09-14T05:30:12.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.890 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:12.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:12.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561060+0000", 2024-09-14T05:30:12.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0699999999999998e-07 2024-09-14T05:30:12.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561072+0000", 2024-09-14T05:30:12.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2261e-05 2024-09-14T05:30:12.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561079+0000", 2024-09-14T05:30:12.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8399999999999997e-06 2024-09-14T05:30:12.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561084+0000", 2024-09-14T05:30:12.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7380000000000002e-06 2024-09-14T05:30:12.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605620+0000", 2024-09-14T05:30:12.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044536146999999998 2024-09-14T05:30:12.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605653+0000", 2024-09-14T05:30:12.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2969000000000002e-05 2024-09-14T05:30:12.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605695+0000", 2024-09-14T05:30:12.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2024000000000003e-05 2024-09-14T05:30:12.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.893 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.893 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74026 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2895872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.893 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:12.894 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.280037487000001, 2024-09-14T05:30:12.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044692717, 2024-09-14T05:30:12.894 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.894 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.894 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.894 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74026 2024-09-14T05:30:12.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.894 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:12.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:12.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:12.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:12.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561093+0000", 2024-09-14T05:30:12.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-14T05:30:12.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561099+0000", 2024-09-14T05:30:12.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0399999999999998e-06 2024-09-14T05:30:12.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561104+0000", 2024-09-14T05:30:12.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8180000000000003e-06 2024-09-14T05:30:12.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605710+0000", 2024-09-14T05:30:12.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044605657 2024-09-14T05:30:12.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605742+0000", 2024-09-14T05:30:12.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1949000000000002e-05 2024-09-14T05:30:12.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605784+0000", 2024-09-14T05:30:12.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1884000000000001e-05 2024-09-14T05:30:12.897 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.898 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.898 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74039 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.898 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:12.898 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.279910889, 2024-09-14T05:30:12.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044654905000000002, 2024-09-14T05:30:12.898 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.898 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.898 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.898 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.898 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.898 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.898 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74039 2024-09-14T05:30:12.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.899 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:12.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:12.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561218+0000", 2024-09-14T05:30:12.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4000000000000001e-07 2024-09-14T05:30:12.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561221+0000", 2024-09-14T05:30:12.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9730000000000002e-06 2024-09-14T05:30:12.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561229+0000", 2024-09-14T05:30:12.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1759999999999993e-06 2024-09-14T05:30:12.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561235+0000", 2024-09-14T05:30:12.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0109999999999997e-06 2024-09-14T05:30:12.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605798+0000", 2024-09-14T05:30:12.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044563285000000001 2024-09-14T05:30:12.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605831+0000", 2024-09-14T05:30:12.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2054999999999997e-05 2024-09-14T05:30:12.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605872+0000", 2024-09-14T05:30:12.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1764999999999997e-05 2024-09-14T05:30:12.902 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.902 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.902 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.902 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74042 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.902 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:12.902 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.279862435999998, 2024-09-14T05:30:12.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044773234000000002, 2024-09-14T05:30:12.902 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.903 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.903 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.903 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.903 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.903 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.903 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74042 2024-09-14T05:30:12.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.903 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.903 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.903 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:12.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.903 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:12.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:12.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4099999999999998e-07 2024-09-14T05:30:12.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561269+0000", 2024-09-14T05:30:12.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7930000000000002e-06 2024-09-14T05:30:12.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561278+0000", 2024-09-14T05:30:12.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5150000000000006e-06 2024-09-14T05:30:12.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561284+0000", 2024-09-14T05:30:12.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2029999999999997e-06 2024-09-14T05:30:12.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605934+0000", 2024-09-14T05:30:12.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044650449000000002 2024-09-14T05:30:12.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605983+0000", 2024-09-14T05:30:12.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8943999999999998e-05 2024-09-14T05:30:12.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606039+0000", 2024-09-14T05:30:12.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5689000000000002e-05 2024-09-14T05:30:12.906 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.906 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.906 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.907 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74052 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.907 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:12.907 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.279835560999999, 2024-09-14T05:30:12.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044830474000000002, 2024-09-14T05:30:12.907 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.907 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.907 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.907 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.907 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.907 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.907 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74052 2024-09-14T05:30:12.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.907 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:12.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:12.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:12.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1099999999999996e-07 2024-09-14T05:30:12.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561295+0000", 2024-09-14T05:30:12.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7770000000000001e-06 2024-09-14T05:30:12.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561303+0000", 2024-09-14T05:30:12.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6860000000000006e-06 2024-09-14T05:30:12.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561308+0000", 2024-09-14T05:30:12.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.023e-06 2024-09-14T05:30:12.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605893+0000", 2024-09-14T05:30:12.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044585458000000001 2024-09-14T05:30:12.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605933+0000", 2024-09-14T05:30:12.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9551999999999997e-05 2024-09-14T05:30:12.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606123+0000", 2024-09-14T05:30:12.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019046699999999999 2024-09-14T05:30:12.911 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.911 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.911 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.911 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74054 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.911 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:12.911 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.279813533999999, 2024-09-14T05:30:12.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093610999999999, 2024-09-14T05:30:12.911 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.912 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.912 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.912 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.912 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.912 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.912 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74054 2024-09-14T05:30:12.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.912 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:12.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:12.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:12.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9699999999999996e-07 2024-09-14T05:30:12.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561317+0000", 2024-09-14T05:30:12.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9740000000000001e-06 2024-09-14T05:30:12.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.914 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561324+0000", 2024-09-14T05:30:12.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4590000000000003e-06 2024-09-14T05:30:12.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.914 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561329+0000", 2024-09-14T05:30:12.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5969999999999999e-06 2024-09-14T05:30:12.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606142+0000", 2024-09-14T05:30:12.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044812347000000002 2024-09-14T05:30:12.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606184+0000", 2024-09-14T05:30:12.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2441999999999998e-05 2024-09-14T05:30:12.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606408+0000", 2024-09-14T05:30:12.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022429500000000001 2024-09-14T05:30:12.916 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.916 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.916 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.916 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74056 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.916 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:12.916 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.279618831000001, 2024-09-14T05:30:12.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045191685000000002, 2024-09-14T05:30:12.916 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.916 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.916 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.916 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.916 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.917 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.917 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74056 2024-09-14T05:30:12.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.917 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:12.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:12.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561510+0000", 2024-09-14T05:30:12.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2399999999999998e-07 2024-09-14T05:30:12.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561513+0000", 2024-09-14T05:30:12.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7219999999999999e-06 2024-09-14T05:30:12.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561524+0000", 2024-09-14T05:30:12.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1357e-05 2024-09-14T05:30:12.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561530+0000", 2024-09-14T05:30:12.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2890000000000003e-06 2024-09-14T05:30:12.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606438+0000", 2024-09-14T05:30:12.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044907326999999997 2024-09-14T05:30:12.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.919 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.919 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606520+0000", 2024-09-14T05:30:12.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1924999999999999e-05 2024-09-14T05:30:12.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.920 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.920 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606701+0000", 2024-09-14T05:30:12.920 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00018144100000000001 2024-09-14T05:30:12.920 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.920 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.920 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.920 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74058 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.920 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:12.920 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.279140158000001, 2024-09-14T05:30:12.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093368000000002, 2024-09-14T05:30:12.921 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.921 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.921 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.921 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.921 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.921 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.921 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74058 2024-09-14T05:30:12.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.921 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:12.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.922 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.922 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:12.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.922 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.922 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561989+0000", 2024-09-14T05:30:12.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3300000000000001e-07 2024-09-14T05:30:12.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.922 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.922 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561993+0000", 2024-09-14T05:30:12.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1790000000000001e-06 2024-09-14T05:30:12.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562007+0000", 2024-09-14T05:30:12.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3434e-05 2024-09-14T05:30:12.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562014+0000", 2024-09-14T05:30:12.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.537e-06 2024-09-14T05:30:12.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606736+0000", 2024-09-14T05:30:12.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044722117999999998 2024-09-14T05:30:12.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.924 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606829+0000", 2024-09-14T05:30:12.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2410999999999993e-05 2024-09-14T05:30:12.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.924 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.924 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607082+0000", 2024-09-14T05:30:12.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025285599999999998 2024-09-14T05:30:12.924 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.924 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.924 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.925 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74059 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.925 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:12.925 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.279082166999999, 2024-09-14T05:30:12.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045479928000000003, 2024-09-14T05:30:12.925 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.925 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.925 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.925 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.925 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.925 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.925 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74059 2024-09-14T05:30:12.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.926 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:12.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:12.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562047+0000", 2024-09-14T05:30:12.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6899999999999997e-07 2024-09-14T05:30:12.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562050+0000", 2024-09-14T05:30:12.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.943e-06 2024-09-14T05:30:12.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562058+0000", 2024-09-14T05:30:12.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0120000000000005e-06 2024-09-14T05:30:12.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562063+0000", 2024-09-14T05:30:12.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:30:12.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607186+0000", 2024-09-14T05:30:12.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045122348999999999 2024-09-14T05:30:12.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607265+0000", 2024-09-14T05:30:12.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9462999999999995e-05 2024-09-14T05:30:12.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607526+0000", 2024-09-14T05:30:12.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026090400000000001 2024-09-14T05:30:12.929 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.929 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.929 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.929 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74060 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2850816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:12.929 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:12.930 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 23.279056789999999, 2024-09-14T05:30:12.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045200092999999997, 2024-09-14T05:30:12.930 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:12.930 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:12.930 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:12.930 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:12.930 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:12.930 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:12.930 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74060 2024-09-14T05:30:12.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.930 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:12.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:12.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:12.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:12.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:12.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:12.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:12.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:12.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5200000000000002e-07 2024-09-14T05:30:12.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:12.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562075+0000", 2024-09-14T05:30:12.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5200000000000002e-06 2024-09-14T05:30:12.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:12.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562082+0000", 2024-09-14T05:30:12.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8929999999999999e-06 2024-09-14T05:30:12.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:12.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562088+0000", 2024-09-14T05:30:12.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5570000000000003e-06 2024-09-14T05:30:12.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:12.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607111+0000", 2024-09-14T05:30:12.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045023464999999999 2024-09-14T05:30:12.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:12.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607190+0000", 2024-09-14T05:30:12.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8721999999999996e-05 2024-09-14T05:30:12.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:12.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:12.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:12.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607272+0000", 2024-09-14T05:30:12.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1483999999999994e-05 2024-09-14T05:30:12.934 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.934 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.934 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.934 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:12.934 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:12.934 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:12.935 INFO:teuthology.orchestra.run.smithi071.stdout:4200 writesame 0x2de337a thru 0x2de3878 data_size 0x1(0x4ff bytes) 2024-09-14T05:30:12.936 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:30:12.937 INFO:teuthology.orchestra.run.smithi071.stdout:4202 writesame 0x8205574 thru 0x8213e93 data_size 0x50(0xe920 bytes) 2024-09-14T05:30:12.937 INFO:teuthology.orchestra.run.smithi071.stdout:4203 writesame 0xdc109ab thru 0xdc16993 data_size 0x1(0x5fe9 bytes) 2024-09-14T05:30:12.937 INFO:teuthology.orchestra.run.smithi071.stdout:4205 write 0xd3ed6 thru 0xd5842 (0x196d bytes) 2024-09-14T05:30:12.937 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:12.937 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:12.937 INFO:teuthology.orchestra.run.smithi071.stdout:4208 write 0xd773a33 thru 0xd77de2e (0xa3fc bytes) 2024-09-14T05:30:12.938 INFO:teuthology.orchestra.run.smithi071.stdout:4209 write 0x3fcdb16 thru 0x3fcdde1 (0x2cc bytes) 2024-09-14T05:30:12.938 INFO:teuthology.orchestra.run.smithi071.stdout:4213 read 0xb9e80f0 thru 0xb9f01e0 (0x80f1 bytes) 2024-09-14T05:30:12.938 INFO:teuthology.orchestra.run.smithi071.stdout:4215 writesame 0x11ea1ec thru 0x11f152e data_size 0x1(0x7343 bytes) 2024-09-14T05:30:12.938 INFO:teuthology.orchestra.run.smithi071.stdout:4216 punch from 0x471814 to 0x477039, (0x5825 bytes) 2024-09-14T05:30:12.938 INFO:teuthology.orchestra.run.smithi071.stdout:4217 trunc from 0xdc16994 to 0x797232d 2024-09-14T05:30:12.938 INFO:teuthology.orchestra.run.smithi071.stdout:4218 write 0x62564b3 thru 0x62614fa (0xb048 bytes) 2024-09-14T05:30:12.938 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:12.934+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:12.944 INFO:teuthology.orchestra.run.smithi071.stdout:4220 writesame 0x79a33e4 thru 0x79a5698 data_size 0x1(0x22b5 bytes) 2024-09-14T05:30:12.944 INFO:teuthology.orchestra.run.smithi071.stdout:4221 write 0x49176fa thru 0x4925557 (0xde5e bytes) 2024-09-14T05:30:12.952 INFO:teuthology.orchestra.run.smithi071.stdout:4222 write 0x628d6d8 thru 0x628e04d (0x976 bytes) 2024-09-14T05:30:12.953 INFO:teuthology.orchestra.run.smithi071.stdout:4223 write 0x142f744 thru 0x1431867 (0x2124 bytes) 2024-09-14T05:30:12.961 INFO:teuthology.orchestra.run.smithi071.stdout:4225 write 0x74788a1 thru 0x7483dfe (0xb55e bytes) 2024-09-14T05:30:12.969 INFO:teuthology.orchestra.run.smithi071.stdout:4226 writesame 0x8c7df4 thru 0x8cb3dc data_size 0x1(0x35e9 bytes) 2024-09-14T05:30:12.985 INFO:teuthology.orchestra.run.smithi071.stdout:4227 trunc from 0x79a5699 to 0xe549a2a 2024-09-14T05:30:12.991 INFO:teuthology.orchestra.run.smithi071.stdout:4229 read 0x7266956 thru 0x726a59b (0x3c46 bytes) 2024-09-14T05:30:12.991 INFO:teuthology.orchestra.run.smithi071.stdout:4231 write 0x9f62590 thru 0x9f6b817 (0x9288 bytes) 2024-09-14T05:30:12.999 INFO:teuthology.orchestra.run.smithi071.stdout:4232 write 0x50d2970 thru 0x50d831f (0x59b0 bytes) 2024-09-14T05:30:13.003 INFO:teuthology.orchestra.run.smithi071.stdout:4233 writesame 0xda952e1 thru 0xdaa2ad8 data_size 0x4(0xd7f8 bytes) 2024-09-14T05:30:13.013 INFO:teuthology.orchestra.run.smithi071.stdout:4234 write 0x65aec9f thru 0x65b5e19 (0x717b bytes) 2024-09-14T05:30:13.017 INFO:teuthology.orchestra.run.smithi071.stdout:4236 punch from 0xc8cc6c7 to 0xc8ce73d, (0x2076 bytes) 2024-09-14T05:30:13.027 INFO:teuthology.orchestra.run.smithi071.stdout:4237 trunc from 0xe549a2a to 0x74b86e1 2024-09-14T05:30:13.036 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:13.035+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:13.046 INFO:teuthology.orchestra.run.smithi071.stdout:4239 trunc from 0x74b86e1 to 0xb9dab0 2024-09-14T05:30:13.064 INFO:teuthology.orchestra.run.smithi071.stdout:4240 punch from 0xa2e051 to 0xa34b44, (0x6af3 bytes) 2024-09-14T05:30:13.072 INFO:teuthology.orchestra.run.smithi071.stdout:4241 write 0x3528690 thru 0x352e109 (0x5a7a bytes) 2024-09-14T05:30:13.074 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:117319 3.9 3:90999c1d:::rbd_object_map.112d7803edfe:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-14T05:30:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:13.072597+0000", 2024-09-14T05:30:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0015702330000000001, 2024-09-14T05:30:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0015899379999999999, 2024-09-14T05:30:13.074 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "waiting for sub ops", 2024-09-14T05:30:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 117319 2024-09-14T05:30:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:13.072597+0000", 2024-09-14T05:30:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:13.072597+0000", 2024-09-14T05:30:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:13.072598+0000", 2024-09-14T05:30:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4900000000000001e-07 2024-09-14T05:30:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.076 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:13.072600+0000", 2024-09-14T05:30:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1610000000000001e-06 2024-09-14T05:30:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:13.072605+0000", 2024-09-14T05:30:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5090000000000004e-06 2024-09-14T05:30:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:13.072608+0000", 2024-09-14T05:30:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.196e-06 2024-09-14T05:30:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.077 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:13.073213+0000", 2024-09-14T05:30:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000605285 2024-09-14T05:30:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:13.073262+0000", 2024-09-14T05:30:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8948e-05 2024-09-14T05:30:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-14T05:30:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:13.073459+0000", 2024-09-14T05:30:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019708900000000001 2024-09-14T05:30:13.078 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:30:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:13.074012+0000", 2024-09-14T05:30:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00055278399999999996 2024-09-14T05:30:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:30:13.079 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 1 2024-09-14T05:30:13.079 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:13.082 INFO:teuthology.orchestra.run.smithi071.stdout:4243 write 0x4c91fd9 thru 0x4c94940 (0x2968 bytes) 2024-09-14T05:30:13.084 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:30:13.099 INFO:teuthology.orchestra.run.smithi071.stdout:4244 write 0xb6eb252 thru 0xb6f2cbe (0x7a6d bytes) 2024-09-14T05:30:13.104 INFO:teuthology.orchestra.run.smithi071.stdout:4245 read 0x564b43 thru 0x569edc (0x539a bytes) 2024-09-14T05:30:13.104 INFO:teuthology.orchestra.run.smithi071.stdout:4246 write 0x87d1ba7 thru 0x87dad98 (0x91f2 bytes) 2024-09-14T05:30:13.112 INFO:teuthology.orchestra.run.smithi071.stdout:4247 read 0x28096ae thru 0x280afb4 (0x1907 bytes) 2024-09-14T05:30:13.112 INFO:teuthology.orchestra.run.smithi071.stdout:4248 write 0x2b47544 thru 0x2b56011 (0xeace bytes) 2024-09-14T05:30:13.121 INFO:teuthology.orchestra.run.smithi071.stdout:4249 read 0x5c7862f thru 0x5c84353 (0xbd25 bytes) 2024-09-14T05:30:13.122 INFO:teuthology.orchestra.run.smithi071.stdout:4251 punch from 0x902ed79 to 0x902f324, (0x5ab bytes) 2024-09-14T05:30:13.122 INFO:teuthology.orchestra.run.smithi071.stdout:4252 read 0xa1838d4 thru 0xa187811 (0x3f3e bytes) 2024-09-14T05:30:13.122 INFO:teuthology.orchestra.run.smithi071.stdout:4254 writesame 0xbfb851 thru 0xbfedea data_size 0x3(0x359a bytes) 2024-09-14T05:30:13.124 INFO:teuthology.orchestra.run.smithi071.stdout:4255 punch from 0xad55b3b to 0xad62f8e, (0xd453 bytes) 2024-09-14T05:30:13.124 INFO:teuthology.orchestra.run.smithi071.stdout:4257 read 0x6c999a0 thru 0x6c9b48d (0x1aee bytes) 2024-09-14T05:30:13.124 INFO:teuthology.orchestra.run.smithi071.stdout:4261 trunc from 0xb6f2cbf to 0xb64c357 2024-09-14T05:30:13.130 INFO:teuthology.orchestra.run.smithi071.stdout:4262 punch from 0x472f7f3 to 0x473a152, (0xa95f bytes) 2024-09-14T05:30:13.134 INFO:teuthology.orchestra.run.smithi071.stdout:4264 trunc from 0xb64c357 to 0xdf1b3dc 2024-09-14T05:30:13.137 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:13.136+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:13.139 INFO:teuthology.orchestra.run.smithi071.stdout:4265 read 0xb308950 thru 0xb31183b (0x8eec bytes) 2024-09-14T05:30:13.139 INFO:teuthology.orchestra.run.smithi071.stdout:4266 writesame 0x2ec9cf6 thru 0x2ecf2cc data_size 0x19(0x55d7 bytes) 2024-09-14T05:30:13.142 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:13.142 INFO:teuthology.orchestra.run.smithi071.stdout:4269 write 0x951fea9 thru 0x95276cb (0x7823 bytes) 2024-09-14T05:30:13.146 INFO:teuthology.orchestra.run.smithi071.stdout:4270 clone 34 order 23 su 2097152 sc 12 2024-09-14T05:30:13.208 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:13.208 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:13.208 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:13.208 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:13.208 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:13.216 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:30:13.238 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:13.237+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:13.277 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T05:30:13.278 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T05:30:13.295 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T05:30:13.338 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:13.338+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:13.374 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone33 from 0xdf1b3dc (overlap 0xb9dab0) to 0x5ced58 2024-09-14T05:30:13.380 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4235.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-14T05:30:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:30:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.538120585999998, 2024-09-14T05:30:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12887758399999999, 2024-09-14T05:30:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.380 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:30:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4235", 2024-09-14T05:30:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/3211559243", 2024-09-14T05:30:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 2 2024-09-14T05:30:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:30:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:30:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839833+0000", 2024-09-14T05:30:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:30:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839838+0000", 2024-09-14T05:30:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2440000000000001e-06 2024-09-14T05:30:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.382 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839848+0000", 2024-09-14T05:30:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0030000000000001e-05 2024-09-14T05:30:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839854+0000", 2024-09-14T05:30:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3750000000000002e-06 2024-09-14T05:30:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839908+0000", 2024-09-14T05:30:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3965999999999998e-05 2024-09-14T05:30:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:30:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839914+0000", 2024-09-14T05:30:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6139999999999997e-06 2024-09-14T05:30:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968558+0000", 2024-09-14T05:30:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12864361199999999 2024-09-14T05:30:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968710+0000", 2024-09-14T05:30:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015177300000000001 2024-09-14T05:30:13.384 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27764 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 188416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:30:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.166794203999999, 2024-09-14T05:30:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080826920999999996, 2024-09-14T05:30:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.385 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27764 2024-09-14T05:30:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:30:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:30:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211159+0000", 2024-09-14T05:30:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1800000000000001e-07 2024-09-14T05:30:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211161+0000", 2024-09-14T05:30:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.968e-06 2024-09-14T05:30:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211169+0000", 2024-09-14T05:30:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4800000000000001e-06 2024-09-14T05:30:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211174+0000", 2024-09-14T05:30:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1880000000000001e-06 2024-09-14T05:30:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291670+0000", 2024-09-14T05:30:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080495199000000003 2024-09-14T05:30:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291775+0000", 2024-09-14T05:30:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000105375 2024-09-14T05:30:13.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291985+0000", 2024-09-14T05:30:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021029299999999999 2024-09-14T05:30:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27765 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:30:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.166771453999999, 2024-09-14T05:30:13.389 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081115197999999999, 2024-09-14T05:30:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27765 2024-09-14T05:30:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:30:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:30:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211182+0000", 2024-09-14T05:30:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3599999999999999e-07 2024-09-14T05:30:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211184+0000", 2024-09-14T05:30:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.03e-06 2024-09-14T05:30:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211192+0000", 2024-09-14T05:30:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2309999999999996e-06 2024-09-14T05:30:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211197+0000", 2024-09-14T05:30:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4199999999999998e-06 2024-09-14T05:30:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.392 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292017+0000", 2024-09-14T05:30:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080819516999999994 2024-09-14T05:30:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292087+0000", 2024-09-14T05:30:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0056999999999997e-05 2024-09-14T05:30:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.393 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292296+0000", 2024-09-14T05:30:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000209507 2024-09-14T05:30:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27766 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:30:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.16674824, 2024-09-14T05:30:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081359077000000002, 2024-09-14T05:30:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.394 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27766 2024-09-14T05:30:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:30:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.395 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:30:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211205+0000", 2024-09-14T05:30:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7e-07 2024-09-14T05:30:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211207+0000", 2024-09-14T05:30:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4540000000000001e-06 2024-09-14T05:30:13.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211216+0000", 2024-09-14T05:30:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3680000000000002e-06 2024-09-14T05:30:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211221+0000", 2024-09-14T05:30:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3499999999999996e-06 2024-09-14T05:30:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292329+0000", 2024-09-14T05:30:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081108240999999998 2024-09-14T05:30:13.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292370+0000", 2024-09-14T05:30:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.121e-05 2024-09-14T05:30:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292563+0000", 2024-09-14T05:30:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019298399999999999 2024-09-14T05:30:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27767 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.166724725999998, 2024-09-14T05:30:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081623119999999993, 2024-09-14T05:30:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27767 2024-09-14T05:30:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:13.400 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0499999999999999e-07 2024-09-14T05:30:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211230+0000", 2024-09-14T05:30:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0700000000000001e-06 2024-09-14T05:30:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211236+0000", 2024-09-14T05:30:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8429999999999998e-06 2024-09-14T05:30:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.401 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211241+0000", 2024-09-14T05:30:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:30:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292594+0000", 2024-09-14T05:30:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081352868999999994 2024-09-14T05:30:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292631+0000", 2024-09-14T05:30:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7083999999999999e-05 2024-09-14T05:30:13.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292851+0000", 2024-09-14T05:30:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022011700000000001 2024-09-14T05:30:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27768 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 274432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:30:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.166706189999999, 2024-09-14T05:30:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081875032, 2024-09-14T05:30:13.403 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27768 2024-09-14T05:30:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:30:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:30:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211247+0000", 2024-09-14T05:30:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.46e-07 2024-09-14T05:30:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211248+0000", 2024-09-14T05:30:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7349999999999999e-06 2024-09-14T05:30:13.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211253+0000", 2024-09-14T05:30:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.899e-06 2024-09-14T05:30:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211258+0000", 2024-09-14T05:30:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2810000000000002e-06 2024-09-14T05:30:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292888+0000", 2024-09-14T05:30:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081629955000000004 2024-09-14T05:30:13.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292926+0000", 2024-09-14T05:30:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7947000000000001e-05 2024-09-14T05:30:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293121+0000", 2024-09-14T05:30:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019586899999999999 2024-09-14T05:30:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27769 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 294912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:30:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.166688100999998, 2024-09-14T05:30:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082181899000000003, 2024-09-14T05:30:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27769 2024-09-14T05:30:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:30:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:30:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211265+0000", 2024-09-14T05:30:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2199999999999999e-07 2024-09-14T05:30:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211267+0000", 2024-09-14T05:30:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2239999999999998e-06 2024-09-14T05:30:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211273+0000", 2024-09-14T05:30:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6540000000000004e-06 2024-09-14T05:30:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211277+0000", 2024-09-14T05:30:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6299999999999997e-06 2024-09-14T05:30:13.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293150+0000", 2024-09-14T05:30:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081872575000000003 2024-09-14T05:30:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293245+0000", 2024-09-14T05:30:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5327000000000004e-05 2024-09-14T05:30:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293446+0000", 2024-09-14T05:30:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000201067 2024-09-14T05:30:13.411 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27770 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.166667939, 2024-09-14T05:30:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082663707000000003, 2024-09-14T05:30:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.412 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27770 2024-09-14T05:30:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1199999999999998e-07 2024-09-14T05:30:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211292+0000", 2024-09-14T05:30:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1609999999999997e-06 2024-09-14T05:30:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211299+0000", 2024-09-14T05:30:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3300000000000004e-06 2024-09-14T05:30:13.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211303+0000", 2024-09-14T05:30:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7859999999999996e-06 2024-09-14T05:30:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293628+0000", 2024-09-14T05:30:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082324283999999998 2024-09-14T05:30:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293694+0000", 2024-09-14T05:30:13.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6277000000000003e-05 2024-09-14T05:30:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293948+0000", 2024-09-14T05:30:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025445699999999998 2024-09-14T05:30:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27790 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:30:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.166641535, 2024-09-14T05:30:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082461006000000003, 2024-09-14T05:30:13.416 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27790 2024-09-14T05:30:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:30:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:30:13.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211312+0000", 2024-09-14T05:30:13.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9299999999999998e-07 2024-09-14T05:30:13.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211314+0000", 2024-09-14T05:30:13.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000002e-06 2024-09-14T05:30:13.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211320+0000", 2024-09-14T05:30:13.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1589999999999998e-06 2024-09-14T05:30:13.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211325+0000", 2024-09-14T05:30:13.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9609999999999996e-06 2024-09-14T05:30:13.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293474+0000", 2024-09-14T05:30:13.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082149080999999999 2024-09-14T05:30:13.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293542+0000", 2024-09-14T05:30:13.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8497000000000005e-05 2024-09-14T05:30:13.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293772+0000", 2024-09-14T05:30:13.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022969900000000001 2024-09-14T05:30:13.420 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.420 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.420 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.421 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27791 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.421 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:30:13.421 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.166621221, 2024-09-14T05:30:13.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082700838999999998, 2024-09-14T05:30:13.421 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.421 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.421 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.421 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.421 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.421 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.421 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27791 2024-09-14T05:30:13.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.422 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:30:13.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:30:13.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211332+0000", 2024-09-14T05:30:13.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7399999999999999e-07 2024-09-14T05:30:13.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211334+0000", 2024-09-14T05:30:13.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8500000000000001e-06 2024-09-14T05:30:13.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211340+0000", 2024-09-14T05:30:13.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3010000000000003e-06 2024-09-14T05:30:13.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211345+0000", 2024-09-14T05:30:13.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.938e-06 2024-09-14T05:30:13.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293797+0000", 2024-09-14T05:30:13.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082452152000000001 2024-09-14T05:30:13.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293869+0000", 2024-09-14T05:30:13.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2236000000000004e-05 2024-09-14T05:30:13.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294032+0000", 2024-09-14T05:30:13.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016298800000000001 2024-09-14T05:30:13.425 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.425 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.425 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.425 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27792 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.425 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:13.425 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.16660169, 2024-09-14T05:30:13.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082897304000000005, 2024-09-14T05:30:13.425 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.425 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.425 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.426 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.426 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.426 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.426 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27792 2024-09-14T05:30:13.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.426 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.426 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:13.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.426 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:13.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:13.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2000000000000001e-07 2024-09-14T05:30:13.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211353+0000", 2024-09-14T05:30:13.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.761e-06 2024-09-14T05:30:13.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211359+0000", 2024-09-14T05:30:13.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2430000000000002e-06 2024-09-14T05:30:13.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211364+0000", 2024-09-14T05:30:13.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0239999999999997e-06 2024-09-14T05:30:13.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294059+0000", 2024-09-14T05:30:13.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082694834999999994 2024-09-14T05:30:13.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294114+0000", 2024-09-14T05:30:13.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4823e-05 2024-09-14T05:30:13.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294248+0000", 2024-09-14T05:30:13.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013429800000000001 2024-09-14T05:30:13.429 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.429 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.429 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.429 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27793 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.430 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:13.430 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.166581891, 2024-09-14T05:30:13.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083153362999999994, 2024-09-14T05:30:13.430 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.430 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.430 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.430 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.430 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.430 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.430 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27793 2024-09-14T05:30:13.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.430 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:13.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:13.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:13.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5100000000000001e-07 2024-09-14T05:30:13.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211373+0000", 2024-09-14T05:30:13.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.001e-06 2024-09-14T05:30:13.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211379+0000", 2024-09-14T05:30:13.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4559999999999997e-06 2024-09-14T05:30:13.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211384+0000", 2024-09-14T05:30:13.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9749999999999998e-06 2024-09-14T05:30:13.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294279+0000", 2024-09-14T05:30:13.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082894282999999999 2024-09-14T05:30:13.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294327+0000", 2024-09-14T05:30:13.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8208000000000003e-05 2024-09-14T05:30:13.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294524+0000", 2024-09-14T05:30:13.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019708900000000001 2024-09-14T05:30:13.433 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.434 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.434 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.434 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27794 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.434 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:13.434 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.166561566999999, 2024-09-14T05:30:13.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588590000000004, 2024-09-14T05:30:13.434 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.434 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.434 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.434 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.434 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.434 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.434 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27794 2024-09-14T05:30:13.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.435 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:13.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:13.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:13.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4499999999999997e-07 2024-09-14T05:30:13.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.436 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211393+0000", 2024-09-14T05:30:13.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.815e-06 2024-09-14T05:30:13.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.436 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211400+0000", 2024-09-14T05:30:13.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2899999999999999e-06 2024-09-14T05:30:13.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211404+0000", 2024-09-14T05:30:13.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:30:13.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294560+0000", 2024-09-14T05:30:13.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083155534000000003 2024-09-14T05:30:13.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294631+0000", 2024-09-14T05:30:13.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1378999999999995e-05 2024-09-14T05:30:13.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294980+0000", 2024-09-14T05:30:13.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00034844900000000002 2024-09-14T05:30:13.438 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.438 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.438 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.438 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27795 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.438 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:30:13.438 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.166542208999999, 2024-09-14T05:30:13.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083970367000000004, 2024-09-14T05:30:13.438 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.439 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.439 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.439 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.439 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.439 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.439 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27795 2024-09-14T05:30:13.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.439 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.439 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:30:13.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:30:13.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211411+0000", 2024-09-14T05:30:13.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5499999999999999e-07 2024-09-14T05:30:13.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211413+0000", 2024-09-14T05:30:13.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:30:13.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.441 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211419+0000", 2024-09-14T05:30:13.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1720000000000004e-06 2024-09-14T05:30:13.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.441 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211424+0000", 2024-09-14T05:30:13.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0690000000000001e-06 2024-09-14T05:30:13.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295013+0000", 2024-09-14T05:30:13.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588898999999994 2024-09-14T05:30:13.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295102+0000", 2024-09-14T05:30:13.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9462000000000005e-05 2024-09-14T05:30:13.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295381+0000", 2024-09-14T05:30:13.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000278566 2024-09-14T05:30:13.442 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.443 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.443 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.443 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27796 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.443 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:13.443 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.166522480000001, 2024-09-14T05:30:13.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084379107999999994, 2024-09-14T05:30:13.443 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.443 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.443 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.443 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.443 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.443 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.444 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27796 2024-09-14T05:30:13.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.444 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:13.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:13.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:13.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2800000000000002e-07 2024-09-14T05:30:13.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211432+0000", 2024-09-14T05:30:13.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:30:13.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211439+0000", 2024-09-14T05:30:13.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5239999999999997e-06 2024-09-14T05:30:13.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211444+0000", 2024-09-14T05:30:13.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.95e-06 2024-09-14T05:30:13.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295602+0000", 2024-09-14T05:30:13.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084158017000000002 2024-09-14T05:30:13.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295674+0000", 2024-09-14T05:30:13.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2385999999999994e-05 2024-09-14T05:30:13.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.447 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.447 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295809+0000", 2024-09-14T05:30:13.447 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013495900000000001 2024-09-14T05:30:13.447 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.447 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.447 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.447 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27797 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 266240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.447 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:13.447 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.166502498, 2024-09-14T05:30:13.447 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084898597000000006, 2024-09-14T05:30:13.447 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.448 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.448 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.448 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.448 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.448 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.448 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27797 2024-09-14T05:30:13.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.448 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.448 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.448 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:13.448 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:13.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:13.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:30:13.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211452+0000", 2024-09-14T05:30:13.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.708e-06 2024-09-14T05:30:13.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211458+0000", 2024-09-14T05:30:13.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3489999999999997e-06 2024-09-14T05:30:13.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211464+0000", 2024-09-14T05:30:13.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3820000000000003e-06 2024-09-14T05:30:13.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295839+0000", 2024-09-14T05:30:13.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084375055000000004 2024-09-14T05:30:13.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295918+0000", 2024-09-14T05:30:13.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9253999999999997e-05 2024-09-14T05:30:13.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296349+0000", 2024-09-14T05:30:13.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043051 2024-09-14T05:30:13.451 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.451 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.451 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.452 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27798 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.452 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:13.452 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.166482782999999, 2024-09-14T05:30:13.452 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085300275999999994, 2024-09-14T05:30:13.452 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.452 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.452 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.452 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.452 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.452 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.452 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27798 2024-09-14T05:30:13.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.452 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.453 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:13.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.453 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:13.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.453 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:13.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:13.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211472+0000", 2024-09-14T05:30:13.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7579999999999999e-06 2024-09-14T05:30:13.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211478+0000", 2024-09-14T05:30:13.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0310000000000004e-06 2024-09-14T05:30:13.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211483+0000", 2024-09-14T05:30:13.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8110000000000002e-06 2024-09-14T05:30:13.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296469+0000", 2024-09-14T05:30:13.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084986575999999994 2024-09-14T05:30:13.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296545+0000", 2024-09-14T05:30:13.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5749999999999998e-05 2024-09-14T05:30:13.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296770+0000", 2024-09-14T05:30:13.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022498900000000001 2024-09-14T05:30:13.456 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.456 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.456 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.456 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27799 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 307200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.456 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:13.456 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.166463870000001, 2024-09-14T05:30:13.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085718461999999995, 2024-09-14T05:30:13.456 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.456 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.456 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.457 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.457 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.457 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.457 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27799 2024-09-14T05:30:13.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.457 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:13.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:13.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:13.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8000000000000001e-07 2024-09-14T05:30:13.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211491+0000", 2024-09-14T05:30:13.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-14T05:30:13.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211497+0000", 2024-09-14T05:30:13.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.857e-06 2024-09-14T05:30:13.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211502+0000", 2024-09-14T05:30:13.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9790000000000001e-06 2024-09-14T05:30:13.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296800+0000", 2024-09-14T05:30:13.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085298099000000002 2024-09-14T05:30:13.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296871+0000", 2024-09-14T05:30:13.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0879999999999999e-05 2024-09-14T05:30:13.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297207+0000", 2024-09-14T05:30:13.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033651699999999999 2024-09-14T05:30:13.460 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.460 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.460 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.460 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27826 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.460 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:30:13.461 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.164748749000001, 2024-09-14T05:30:13.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084306729999999996, 2024-09-14T05:30:13.461 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.461 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.461 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.461 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.461 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.461 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.461 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27826 2024-09-14T05:30:13.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.461 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.461 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:30:13.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:30:13.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213205+0000", 2024-09-14T05:30:13.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.277e-06 2024-09-14T05:30:13.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213210+0000", 2024-09-14T05:30:13.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3120000000000001e-06 2024-09-14T05:30:13.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213226+0000", 2024-09-14T05:30:13.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5279000000000001e-05 2024-09-14T05:30:13.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213234+0000", 2024-09-14T05:30:13.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7020000000000007e-06 2024-09-14T05:30:13.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297239+0000", 2024-09-14T05:30:13.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084004876000000006 2024-09-14T05:30:13.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297311+0000", 2024-09-14T05:30:13.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1336999999999998e-05 2024-09-14T05:30:13.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297511+0000", 2024-09-14T05:30:13.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019994699999999999 2024-09-14T05:30:13.464 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.464 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.464 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.465 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27827 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:13.465 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:30:13.465 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.164703562, 2024-09-14T05:30:13.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084520537000000007, 2024-09-14T05:30:13.465 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:13.465 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:13.465 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:13.465 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:13.465 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:13.465 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.465 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27827 2024-09-14T05:30:13.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.465 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:13.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:13.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:30:13.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:13.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:30:13.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:13.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:13.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213250+0000", 2024-09-14T05:30:13.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1099999999999995e-07 2024-09-14T05:30:13.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:13.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213252+0000", 2024-09-14T05:30:13.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6699999999999998e-06 2024-09-14T05:30:13.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:13.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213259+0000", 2024-09-14T05:30:13.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8469999999999998e-06 2024-09-14T05:30:13.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213265+0000", 2024-09-14T05:30:13.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4940000000000003e-06 2024-09-14T05:30:13.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:13.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297541+0000", 2024-09-14T05:30:13.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084276731999999993 2024-09-14T05:30:13.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:13.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297579+0000", 2024-09-14T05:30:13.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8108999999999997e-05 2024-09-14T05:30:13.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:13.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:13.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:13.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297770+0000", 2024-09-14T05:30:13.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019007400000000001 2024-09-14T05:30:13.469 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.469 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.469 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.469 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:13.469 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:13.469 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:13.470 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:30:13.471 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:13.470+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:13.495 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-14T05:30:13.502 INFO:teuthology.orchestra.run.smithi071.stdout:4271 read 0x2aebce2 thru 0x2aef79c (0x3abb bytes) 2024-09-14T05:30:13.504 INFO:teuthology.orchestra.run.smithi071.stdout:4273 write 0x4362dd8 thru 0x4370a91 (0xdcba bytes) 2024-09-14T05:30:13.512 INFO:teuthology.orchestra.run.smithi071.stdout:4274 writesame 0x1eff41d thru 0x1effdce data_size 0x11(0x9b2 bytes) 2024-09-14T05:30:13.518 INFO:teuthology.orchestra.run.smithi071.stdout:4275 write 0x4540c01 thru 0x4545b69 (0x4f69 bytes) 2024-09-14T05:30:13.523 INFO:teuthology.orchestra.run.smithi071.stdout:4277 read 0x2eb7982 thru 0x2ec418e (0xc80d bytes) 2024-09-14T05:30:13.523 INFO:teuthology.orchestra.run.smithi071.stdout:4278 writesame 0x34260d5 thru 0x3432e3b data_size 0x1(0xcd67 bytes) 2024-09-14T05:30:13.558 INFO:teuthology.orchestra.run.smithi071.stdout:4280 punch from 0x8eb2d22 to 0x8eb4547, (0x1825 bytes) 2024-09-14T05:30:13.564 INFO:teuthology.orchestra.run.smithi071.stdout:4281 write 0xecdcad2 thru 0xece2852 (0x5d81 bytes) 2024-09-14T05:30:13.568 INFO:teuthology.orchestra.run.smithi071.stdout:4282 write 0x44df8b8 thru 0x44e12bf (0x1a08 bytes) 2024-09-14T05:30:13.569 INFO:teuthology.orchestra.run.smithi071.stdout:4283 read 0xa4f3593 thru 0xa4fcc96 (0x9704 bytes) 2024-09-14T05:30:13.569 INFO:teuthology.orchestra.run.smithi071.stdout:4284 writesame 0x53c7f7b thru 0x53d16af data_size 0x21(0x9735 bytes) 2024-09-14T05:30:13.571 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:13.571+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:13.574 INFO:teuthology.orchestra.run.smithi071.stdout:4285 write 0xe431e33 thru 0xe435a42 (0x3c10 bytes) 2024-09-14T05:30:13.578 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:13.578 INFO:teuthology.orchestra.run.smithi071.stdout:4287 read 0xe1694e8 thru 0xe16f559 (0x6072 bytes) 2024-09-14T05:30:13.578 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:13.584 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:13.584 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:13.584 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:13.584 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:13.593 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:30:13.594 INFO:teuthology.orchestra.run.smithi071.stdout:4290 trunc from 0xece2853 to 0xf5b152 2024-09-14T05:30:13.625 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:13.625 INFO:teuthology.orchestra.run.smithi071.stdout:4292 punch from 0xba3c09 to 0xba97aa, (0x5ba1 bytes) 2024-09-14T05:30:13.646 INFO:teuthology.orchestra.run.smithi071.stdout:4293 writesame 0xc72fd17 thru 0xc735584 data_size 0xb(0x586e bytes) 2024-09-14T05:30:13.646 INFO:teuthology.orchestra.run.smithi071.stdout:4294 write 0x8adcb9b thru 0x8adda16 (0xe7c bytes) 2024-09-14T05:30:13.649 INFO:teuthology.orchestra.run.smithi071.stdout:4296 writesame 0xb85a2f2 thru 0xb86834c data_size 0x23(0xe05b bytes) 2024-09-14T05:30:13.652 INFO:teuthology.orchestra.run.smithi071.stdout:4299 writesame 0x8834821 thru 0x883acc1 data_size 0x1f(0x64a1 bytes) 2024-09-14T05:30:13.653 INFO:teuthology.orchestra.run.smithi071.stdout:4300 writesame 0x3ac6f3d thru 0x3ad45df data_size 0x1(0xd6a3 bytes) 2024-09-14T05:30:13.672 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:13.671+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:13.684 INFO:teuthology.orchestra.run.smithi071.stdout:4302 trunc from 0xc735585 to 0xbb23119 2024-09-14T05:30:13.691 INFO:teuthology.orchestra.run.smithi071.stdout:4303 write 0x39cdc3c thru 0x39dba90 (0xde55 bytes) 2024-09-14T05:30:13.693 INFO:teuthology.orchestra.run.smithi071.stdout:4305 write 0xe8fc06 thru 0xe914c8 (0x18c3 bytes) 2024-09-14T05:30:13.696 INFO:teuthology.orchestra.run.smithi071.stdout:4307 trunc from 0xbb23119 to 0xa86d4ae 2024-09-14T05:30:13.700 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:13.700 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:13.700 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:13.700 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:13.701 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:13.703 INFO:teuthology.orchestra.run.smithi071.stdout:4308 punch from 0x72c08c to 0x7398aa, (0xd81e bytes) 2024-09-14T05:30:13.703 INFO:teuthology.orchestra.run.smithi071.stdout:4309 read 0xc9c723 thru 0xca4de9 (0x86c7 bytes) 2024-09-14T05:30:13.709 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:30:13.714 INFO:teuthology.orchestra.run.smithi071.stdout:4312 trunc from 0xa86d4ae to 0x20e4bab 2024-09-14T05:30:13.745 INFO:teuthology.orchestra.run.smithi071.stdout:4313 write 0xc1fe34c thru 0xc20b868 (0xd51d bytes) 2024-09-14T05:30:13.750 INFO:teuthology.orchestra.run.smithi071.stdout:4315 writesame 0x948bc43 thru 0x9498b2b data_size 0x31(0xcee9 bytes) 2024-09-14T05:30:13.754 INFO:teuthology.orchestra.run.smithi071.stdout:4316 read 0x6101a8b thru 0x610d792 (0xbd08 bytes) 2024-09-14T05:30:13.755 INFO:teuthology.orchestra.run.smithi071.stdout:4317 writesame 0x97cc532 thru 0x97d1fe5 data_size 0x24(0x5ab4 bytes) 2024-09-14T05:30:13.762 INFO:teuthology.orchestra.run.smithi071.stdout:4318 writesame 0xc71ba5d thru 0xc72049b data_size 0x1(0x4a3f bytes) 2024-09-14T05:30:13.765 INFO:teuthology.orchestra.run.smithi071.stdout:4319 trunc from 0xc72049c to 0xa1545cc 2024-09-14T05:30:13.773 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:13.772+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:13.775 INFO:teuthology.orchestra.run.smithi071.stdout:4320 write 0x87a39f4 thru 0x87acf23 (0x9530 bytes) 2024-09-14T05:30:13.778 INFO:teuthology.orchestra.run.smithi071.stdout:4322 read 0x78671b1 thru 0x786d18c (0x5fdc bytes) 2024-09-14T05:30:13.778 INFO:teuthology.orchestra.run.smithi071.stdout:4323 read 0x48adea9 thru 0x48b9def (0xbf47 bytes) 2024-09-14T05:30:13.783 INFO:teuthology.orchestra.run.smithi071.stdout:4324 trunc from 0xa1545cc to 0x6519b0e 2024-09-14T05:30:13.807 INFO:teuthology.orchestra.run.smithi071.stdout:4325 writesame 0xdbd9d21 thru 0xdbe163e data_size 0x25(0x791e bytes) 2024-09-14T05:30:13.807 INFO:teuthology.orchestra.run.smithi071.stdout:4327 writesame 0x528c478 thru 0x529bfcd data_size 0x35(0xfb56 bytes) 2024-09-14T05:30:13.808 INFO:teuthology.orchestra.run.smithi071.stdout:4328 write 0xd010b21 thru 0xd0207f8 (0xfcd8 bytes) 2024-09-14T05:30:13.812 INFO:teuthology.orchestra.run.smithi071.stdout:4330 write 0xa0deef6 thru 0xa0e94fb (0xa606 bytes) 2024-09-14T05:30:13.814 INFO:teuthology.orchestra.run.smithi071.stdout:4331 writesame 0xd576112 thru 0xd583258 data_size 0x5(0xd147 bytes) 2024-09-14T05:30:13.816 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:13.816 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:13.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:13.816 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:13.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.816 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3523 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6062080~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.817 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:13.817 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.300321769999996, 2024-09-14T05:30:13.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096911030999999995, 2024-09-14T05:30:13.817 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.817 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.817 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.817 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.817 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.817 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.817 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3523 2024-09-14T05:30:13.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.817 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:13.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:13.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514359+0000", 2024-09-14T05:30:13.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9299999999999999e-07 2024-09-14T05:30:13.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514361+0000", 2024-09-14T05:30:13.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:13.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514366+0000", 2024-09-14T05:30:13.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1660000000000002e-06 2024-09-14T05:30:13.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514371+0000", 2024-09-14T05:30:13.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.648e-06 2024-09-14T05:30:13.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.610954+0000", 2024-09-14T05:30:13.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096582170999999994 2024-09-14T05:30:13.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611002+0000", 2024-09-14T05:30:13.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7923999999999997e-05 2024-09-14T05:30:13.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611269+0000", 2024-09-14T05:30:13.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026789499999999998 2024-09-14T05:30:13.821 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.821 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.821 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.821 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3524 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6553600~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.821 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:13.821 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.300302578, 2024-09-14T05:30:13.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097263347999999999, 2024-09-14T05:30:13.821 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.821 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.821 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.821 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.821 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.822 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3524 2024-09-14T05:30:13.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.822 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:13.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:13.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:13.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:13.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514380+0000", 2024-09-14T05:30:13.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6500000000000001e-06 2024-09-14T05:30:13.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514386+0000", 2024-09-14T05:30:13.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7360000000000004e-06 2024-09-14T05:30:13.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514391+0000", 2024-09-14T05:30:13.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2830000000000003e-06 2024-09-14T05:30:13.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611295+0000", 2024-09-14T05:30:13.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096904814000000006 2024-09-14T05:30:13.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611343+0000", 2024-09-14T05:30:13.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7506999999999998e-05 2024-09-14T05:30:13.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611641+0000", 2024-09-14T05:30:13.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029798800000000001 2024-09-14T05:30:13.826 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.826 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.826 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.826 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3525 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7045120~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.826 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:13.826 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.300280690999998, 2024-09-14T05:30:13.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097644422999999994, 2024-09-14T05:30:13.826 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.826 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.826 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.826 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.826 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.827 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.827 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3525 2024-09-14T05:30:13.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.827 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:13.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:13.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514400+0000", 2024-09-14T05:30:13.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.77e-07 2024-09-14T05:30:13.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514402+0000", 2024-09-14T05:30:13.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6530000000000001e-06 2024-09-14T05:30:13.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514411+0000", 2024-09-14T05:30:13.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7429999999999997e-06 2024-09-14T05:30:13.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514417+0000", 2024-09-14T05:30:13.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2249999999999997e-06 2024-09-14T05:30:13.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611668+0000", 2024-09-14T05:30:13.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097250681000000005 2024-09-14T05:30:13.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611716+0000", 2024-09-14T05:30:13.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7336000000000002e-05 2024-09-14T05:30:13.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612044+0000", 2024-09-14T05:30:13.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032840799999999998 2024-09-14T05:30:13.830 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.830 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.830 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.830 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3526 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7536640~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.830 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:13.830 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.300256206999997, 2024-09-14T05:30:13.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097943884999999994, 2024-09-14T05:30:13.831 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.831 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.831 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.831 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.831 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.831 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.831 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3526 2024-09-14T05:30:13.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.831 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:13.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:13.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:13.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0400000000000002e-07 2024-09-14T05:30:13.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514426+0000", 2024-09-14T05:30:13.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:13.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514432+0000", 2024-09-14T05:30:13.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.105e-06 2024-09-14T05:30:13.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514438+0000", 2024-09-14T05:30:13.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5149999999999997e-06 2024-09-14T05:30:13.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612072+0000", 2024-09-14T05:30:13.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097633947999999998 2024-09-14T05:30:13.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612120+0000", 2024-09-14T05:30:13.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8458999999999998e-05 2024-09-14T05:30:13.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612368+0000", 2024-09-14T05:30:13.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024761999999999999 2024-09-14T05:30:13.835 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.835 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.835 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.835 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3527 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 8028160~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.835 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:13.835 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.300235493999999, 2024-09-14T05:30:13.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098244955999999994, 2024-09-14T05:30:13.835 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.835 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.835 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.835 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.835 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.836 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.836 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3527 2024-09-14T05:30:13.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.836 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:13.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:13.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519842+0000", 2024-09-14T05:30:13.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053975230000000004 2024-09-14T05:30:13.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519851+0000", 2024-09-14T05:30:13.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5599999999999994e-06 2024-09-14T05:30:13.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519858+0000", 2024-09-14T05:30:13.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2409999999999998e-06 2024-09-14T05:30:13.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519863+0000", 2024-09-14T05:30:13.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:30:13.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612395+0000", 2024-09-14T05:30:13.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092532552000000004 2024-09-14T05:30:13.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612442+0000", 2024-09-14T05:30:13.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6486000000000002e-05 2024-09-14T05:30:13.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612690+0000", 2024-09-14T05:30:13.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024791599999999999 2024-09-14T05:30:13.839 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.839 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.839 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.839 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3603 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5177344~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.839 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:13.840 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.294683049, 2024-09-14T05:30:13.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096796436, 2024-09-14T05:30:13.840 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.840 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.840 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.840 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.840 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.840 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.840 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3603 2024-09-14T05:30:13.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.840 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:13.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:13.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:13.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1899999999999999e-07 2024-09-14T05:30:13.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519998+0000", 2024-09-14T05:30:13.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5000000000000001e-07 2024-09-14T05:30:13.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520002+0000", 2024-09-14T05:30:13.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7299999999999999e-06 2024-09-14T05:30:13.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520004+0000", 2024-09-14T05:30:13.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.429e-06 2024-09-14T05:30:13.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616493+0000", 2024-09-14T05:30:13.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096488345000000003 2024-09-14T05:30:13.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616544+0000", 2024-09-14T05:30:13.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1724000000000001e-05 2024-09-14T05:30:13.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616794+0000", 2024-09-14T05:30:13.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000249139 2024-09-14T05:30:13.844 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.844 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.844 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.844 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3604 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5668864~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.844 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:13.844 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.294668411000004, 2024-09-14T05:30:13.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108222999999994, 2024-09-14T05:30:13.844 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.844 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.845 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.845 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.845 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.845 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.845 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3604 2024-09-14T05:30:13.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.845 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:13.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:13.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:13.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7000000000000001e-07 2024-09-14T05:30:13.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520013+0000", 2024-09-14T05:30:13.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.389e-06 2024-09-14T05:30:13.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520018+0000", 2024-09-14T05:30:13.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3019999999999996e-06 2024-09-14T05:30:13.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520026+0000", 2024-09-14T05:30:13.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6170000000000007e-06 2024-09-14T05:30:13.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616816+0000", 2024-09-14T05:30:13.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096789766999999999 2024-09-14T05:30:13.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616871+0000", 2024-09-14T05:30:13.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4811000000000001e-05 2024-09-14T05:30:13.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617120+0000", 2024-09-14T05:30:13.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024906700000000001 2024-09-14T05:30:13.848 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.848 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.848 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.849 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3605 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6160384~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.849 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:13.849 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.294649950999997, 2024-09-14T05:30:13.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097423945999999997, 2024-09-14T05:30:13.849 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.849 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.849 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.849 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.849 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.849 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.849 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3605 2024-09-14T05:30:13.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.850 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:13.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:13.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:13.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3999999999999998e-07 2024-09-14T05:30:13.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520032+0000", 2024-09-14T05:30:13.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.083e-06 2024-09-14T05:30:13.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520036+0000", 2024-09-14T05:30:13.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.019e-06 2024-09-14T05:30:13.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520038+0000", 2024-09-14T05:30:13.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4849999999999999e-06 2024-09-14T05:30:13.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617147+0000", 2024-09-14T05:30:13.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108665999999996 2024-09-14T05:30:13.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617199+0000", 2024-09-14T05:30:13.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2558000000000001e-05 2024-09-14T05:30:13.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617454+0000", 2024-09-14T05:30:13.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025489499999999999 2024-09-14T05:30:13.853 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.853 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.853 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.853 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3606 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6651904~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.853 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:13.853 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.294639228999998, 2024-09-14T05:30:13.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097745705000000002, 2024-09-14T05:30:13.854 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.854 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.854 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.854 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.854 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.854 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.854 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3606 2024-09-14T05:30:13.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.854 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:13.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:13.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:13.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-14T05:30:13.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520042+0000", 2024-09-14T05:30:13.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4799999999999997e-07 2024-09-14T05:30:13.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520046+0000", 2024-09-14T05:30:13.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8149999999999999e-06 2024-09-14T05:30:13.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520049+0000", 2024-09-14T05:30:13.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.576e-06 2024-09-14T05:30:13.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617476+0000", 2024-09-14T05:30:13.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097427558999999997 2024-09-14T05:30:13.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617522+0000", 2024-09-14T05:30:13.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5927999999999998e-05 2024-09-14T05:30:13.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617787+0000", 2024-09-14T05:30:13.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000264663 2024-09-14T05:30:13.857 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.858 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.858 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.858 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3607 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7143424~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.858 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:13.858 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.294628701999997, 2024-09-14T05:30:13.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097871125000000003, 2024-09-14T05:30:13.858 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.858 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.858 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.858 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.858 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.858 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.858 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3607 2024-09-14T05:30:13.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.859 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:13.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:13.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520052+0000", 2024-09-14T05:30:13.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.03e-07 2024-09-14T05:30:13.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520053+0000", 2024-09-14T05:30:13.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.85e-07 2024-09-14T05:30:13.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520056+0000", 2024-09-14T05:30:13.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9140000000000001e-06 2024-09-14T05:30:13.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520059+0000", 2024-09-14T05:30:13.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4600000000000002e-06 2024-09-14T05:30:13.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617809+0000", 2024-09-14T05:30:13.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097750216000000001 2024-09-14T05:30:13.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617864+0000", 2024-09-14T05:30:13.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5211999999999999e-05 2024-09-14T05:30:13.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617923+0000", 2024-09-14T05:30:13.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8235000000000001e-05 2024-09-14T05:30:13.862 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.862 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.862 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.862 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3608 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7634944~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.862 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:13.862 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.294618393999997, 2024-09-14T05:30:13.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098199212999999994, 2024-09-14T05:30:13.863 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.863 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.863 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.863 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.863 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.863 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.863 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3608 2024-09-14T05:30:13.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.863 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:13.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:13.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525953+0000", 2024-09-14T05:30:13.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058911850000000002 2024-09-14T05:30:13.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525957+0000", 2024-09-14T05:30:13.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2379999999999999e-06 2024-09-14T05:30:13.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525967+0000", 2024-09-14T05:30:13.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5389999999999996e-06 2024-09-14T05:30:13.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525972+0000", 2024-09-14T05:30:13.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.541e-06 2024-09-14T05:30:13.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617945+0000", 2024-09-14T05:30:13.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091972597000000003 2024-09-14T05:30:13.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617992+0000", 2024-09-14T05:30:13.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7397e-05 2024-09-14T05:30:13.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.618261+0000", 2024-09-14T05:30:13.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026871600000000001 2024-09-14T05:30:13.866 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.866 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.866 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.867 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3640 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6586368~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.867 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:13.867 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.288510551999998, 2024-09-14T05:30:13.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097068580000000002, 2024-09-14T05:30:13.867 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.867 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.867 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.867 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.867 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.867 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.867 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3640 2024-09-14T05:30:13.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.868 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:13.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:13.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:13.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.23e-07 2024-09-14T05:30:13.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526171+0000", 2024-09-14T05:30:13.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:30:13.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526175+0000", 2024-09-14T05:30:13.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8369999999999999e-06 2024-09-14T05:30:13.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526177+0000", 2024-09-14T05:30:13.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4959999999999999e-06 2024-09-14T05:30:13.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622847+0000", 2024-09-14T05:30:13.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096669790000000005 2024-09-14T05:30:13.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622902+0000", 2024-09-14T05:30:13.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5157000000000001e-05 2024-09-14T05:30:13.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623238+0000", 2024-09-14T05:30:13.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033603399999999997 2024-09-14T05:30:13.871 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.871 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.871 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.871 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3641 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7077888~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.871 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:13.871 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.288500042000003, 2024-09-14T05:30:13.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097371141999999994, 2024-09-14T05:30:13.871 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.871 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.872 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.872 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.872 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.872 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.872 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3641 2024-09-14T05:30:13.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.872 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:13.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:13.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531851+0000", 2024-09-14T05:30:13.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005671265 2024-09-14T05:30:13.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531855+0000", 2024-09-14T05:30:13.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0849999999999999e-06 2024-09-14T05:30:13.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531867+0000", 2024-09-14T05:30:13.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1028e-05 2024-09-14T05:30:13.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531872+0000", 2024-09-14T05:30:13.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000003e-06 2024-09-14T05:30:13.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623251+0000", 2024-09-14T05:30:13.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091378980999999998 2024-09-14T05:30:13.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623295+0000", 2024-09-14T05:30:13.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3881999999999998e-05 2024-09-14T05:30:13.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623551+0000", 2024-09-14T05:30:13.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025636099999999998 2024-09-14T05:30:13.875 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.875 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.875 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.876 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3721 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.876 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:13.876 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.282641767000001, 2024-09-14T05:30:13.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096806472000000005, 2024-09-14T05:30:13.876 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.876 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.876 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.876 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.876 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.876 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.877 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3721 2024-09-14T05:30:13.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.877 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:13.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:13.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532039+0000", 2024-09-14T05:30:13.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.05e-07 2024-09-14T05:30:13.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532040+0000", 2024-09-14T05:30:13.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9099999999999991e-07 2024-09-14T05:30:13.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532043+0000", 2024-09-14T05:30:13.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6610000000000002e-06 2024-09-14T05:30:13.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532046+0000", 2024-09-14T05:30:13.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4080000000000001e-06 2024-09-14T05:30:13.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628545+0000", 2024-09-14T05:30:13.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096499615999999996 2024-09-14T05:30:13.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628592+0000", 2024-09-14T05:30:13.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6907000000000003e-05 2024-09-14T05:30:13.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628845+0000", 2024-09-14T05:30:13.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025268399999999998 2024-09-14T05:30:13.881 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.881 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.881 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.881 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3722 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6193152~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.881 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:13.881 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.282631686999999, 2024-09-14T05:30:13.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097146587000000006, 2024-09-14T05:30:13.881 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.881 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.881 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.881 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.882 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3722 2024-09-14T05:30:13.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.882 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:13.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:13.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538191+0000", 2024-09-14T05:30:13.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061429969999999999 2024-09-14T05:30:13.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538200+0000", 2024-09-14T05:30:13.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7069999999999998e-06 2024-09-14T05:30:13.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538212+0000", 2024-09-14T05:30:13.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1504e-05 2024-09-14T05:30:13.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538219+0000", 2024-09-14T05:30:13.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7500000000000003e-06 2024-09-14T05:30:13.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628869+0000", 2024-09-14T05:30:13.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090649224 2024-09-14T05:30:13.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628919+0000", 2024-09-14T05:30:13.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0144e-05 2024-09-14T05:30:13.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.629195+0000", 2024-09-14T05:30:13.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027626099999999998 2024-09-14T05:30:13.885 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.885 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.885 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.885 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3758 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7110656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.885 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:13.886 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.276037471000002, 2024-09-14T05:30:13.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096977544999999998, 2024-09-14T05:30:13.886 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.886 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.886 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.886 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.886 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.886 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.886 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3758 2024-09-14T05:30:13.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.886 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:13.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:13.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:13.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:13.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538645+0000", 2024-09-14T05:30:13.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.843e-06 2024-09-14T05:30:13.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538651+0000", 2024-09-14T05:30:13.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6169999999999998e-06 2024-09-14T05:30:13.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538655+0000", 2024-09-14T05:30:13.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5399999999999997e-06 2024-09-14T05:30:13.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635225+0000", 2024-09-14T05:30:13.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096570149999999993 2024-09-14T05:30:13.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635272+0000", 2024-09-14T05:30:13.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7199e-05 2024-09-14T05:30:13.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635620+0000", 2024-09-14T05:30:13.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034782600000000003 2024-09-14T05:30:13.890 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.890 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.890 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.890 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3759 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.890 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:13.890 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.276019560000002, 2024-09-14T05:30:13.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097335331999999997, 2024-09-14T05:30:13.890 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.890 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.890 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.891 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.891 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.891 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.891 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3759 2024-09-14T05:30:13.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.891 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:13.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:13.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544013+0000", 2024-09-14T05:30:13.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053520829999999997 2024-09-14T05:30:13.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544016+0000", 2024-09-14T05:30:13.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6660000000000001e-06 2024-09-14T05:30:13.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544026+0000", 2024-09-14T05:30:13.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7089999999999997e-06 2024-09-14T05:30:13.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544043+0000", 2024-09-14T05:30:13.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833999999999998e-05 2024-09-14T05:30:13.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635646+0000", 2024-09-14T05:30:13.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091603544999999995 2024-09-14T05:30:13.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635696+0000", 2024-09-14T05:30:13.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9307000000000001e-05 2024-09-14T05:30:13.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635996+0000", 2024-09-14T05:30:13.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030018800000000001 2024-09-14T05:30:13.894 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.894 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.894 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.894 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3838 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6225920~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.895 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:13.895 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.270357879999999, 2024-09-14T05:30:13.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096894235999999995, 2024-09-14T05:30:13.895 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.895 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.895 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.895 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.895 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.895 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.895 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3838 2024-09-14T05:30:13.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.895 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:13.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:13.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544323+0000", 2024-09-14T05:30:13.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-14T05:30:13.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544325+0000", 2024-09-14T05:30:13.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.75e-06 2024-09-14T05:30:13.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544331+0000", 2024-09-14T05:30:13.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5019999999999998e-06 2024-09-14T05:30:13.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544335+0000", 2024-09-14T05:30:13.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2989999999999998e-06 2024-09-14T05:30:13.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640869+0000", 2024-09-14T05:30:13.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096533351000000003 2024-09-14T05:30:13.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640919+0000", 2024-09-14T05:30:13.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999999e-05 2024-09-14T05:30:13.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641217+0000", 2024-09-14T05:30:13.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029717600000000002 2024-09-14T05:30:13.899 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.899 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.899 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.899 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3839 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.899 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:13.899 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.270338942000002, 2024-09-14T05:30:13.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097240339999999995, 2024-09-14T05:30:13.899 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.899 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.899 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.899 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.900 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.900 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.900 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3839 2024-09-14T05:30:13.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.900 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:13.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:13.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549834+0000", 2024-09-14T05:30:13.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054927819999999999 2024-09-14T05:30:13.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549838+0000", 2024-09-14T05:30:13.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8689999999999997e-06 2024-09-14T05:30:13.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549848+0000", 2024-09-14T05:30:13.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0154e-05 2024-09-14T05:30:13.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549853+0000", 2024-09-14T05:30:13.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9740000000000001e-06 2024-09-14T05:30:13.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641245+0000", 2024-09-14T05:30:13.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091392245999999996 2024-09-14T05:30:13.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641295+0000", 2024-09-14T05:30:13.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0018999999999997e-05 2024-09-14T05:30:13.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641582+0000", 2024-09-14T05:30:13.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028629600000000001 2024-09-14T05:30:13.903 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.903 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.903 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.903 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3868 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:13.903 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:13.904 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.264676364000003, 2024-09-14T05:30:13.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096859555999999999, 2024-09-14T05:30:13.904 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:13.904 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:13.904 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:13.904 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:13.904 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:13.904 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:13.904 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3868 2024-09-14T05:30:13.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.904 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:13.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:13.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:13.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:13.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:13.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:13.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:13.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556173+0000", 2024-09-14T05:30:13.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006169038 2024-09-14T05:30:13.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:13.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556177+0000", 2024-09-14T05:30:13.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0010000000000003e-06 2024-09-14T05:30:13.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:13.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556188+0000", 2024-09-14T05:30:13.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0675e-05 2024-09-14T05:30:13.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:13.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556199+0000", 2024-09-14T05:30:13.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.155e-05 2024-09-14T05:30:13.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:13.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646511+0000", 2024-09-14T05:30:13.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090312187000000002 2024-09-14T05:30:13.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:13.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646563+0000", 2024-09-14T05:30:13.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2219000000000003e-05 2024-09-14T05:30:13.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:13.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:13.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:13.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646863+0000", 2024-09-14T05:30:13.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029988600000000002 2024-09-14T05:30:13.908 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.908 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.908 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.908 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:13.908 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:13.908 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:13.910 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:30:13.910 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:13.909+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:13.911 INFO:teuthology.orchestra.run.smithi071.stdout:4332 read 0x8d05fac thru 0x8d06ed5 (0xf2a bytes) 2024-09-14T05:30:13.911 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:13.912 INFO:teuthology.orchestra.run.smithi071.stdout:4334 read 0x4b2b8c0 thru 0x4b3aeb6 (0xf5f7 bytes) 2024-09-14T05:30:13.912 INFO:teuthology.orchestra.run.smithi071.stdout:4335 trunc from 0xdbe163f to 0x8273277 2024-09-14T05:30:13.912 INFO:teuthology.orchestra.run.smithi071.stdout:4338 write 0x3760d58 thru 0x3766dfa (0x60a3 bytes) 2024-09-14T05:30:13.912 INFO:teuthology.orchestra.run.smithi071.stdout:4340 write 0x2ee37f0 thru 0x2ee9ea1 (0x66b2 bytes) 2024-09-14T05:30:13.912 INFO:teuthology.orchestra.run.smithi071.stdout:4342 trunc from 0x8273277 to 0xe41efa1 2024-09-14T05:30:13.912 INFO:teuthology.orchestra.run.smithi071.stdout:4343 read 0x3b0a1f5 thru 0x3b1687a (0xc686 bytes) 2024-09-14T05:30:13.912 INFO:teuthology.orchestra.run.smithi071.stdout:4344 read 0x79793c8 thru 0x79794d5 (0x10e bytes) 2024-09-14T05:30:13.912 INFO:teuthology.orchestra.run.smithi071.stdout:4345 writesame 0xc744f6c thru 0xc751ced data_size 0x5(0xcd82 bytes) 2024-09-14T05:30:13.912 INFO:teuthology.orchestra.run.smithi071.stdout:4347 read 0xd469a26 thru 0xd47779e (0xdd79 bytes) 2024-09-14T05:30:13.912 INFO:teuthology.orchestra.run.smithi071.stdout:4348 read 0x292d625 thru 0x2938b74 (0xb550 bytes) 2024-09-14T05:30:13.912 INFO:teuthology.orchestra.run.smithi071.stdout:4349 read 0x795eb1a thru 0x796db36 (0xf01d bytes) 2024-09-14T05:30:13.912 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:13.912 INFO:teuthology.orchestra.run.smithi071.stdout:4351 read 0x377190e thru 0x3777cbb (0x63ae bytes) 2024-09-14T05:30:13.912 INFO:teuthology.orchestra.run.smithi071.stdout:4352 write 0x754a8c5 thru 0x754cd7e (0x24ba bytes) 2024-09-14T05:30:13.913 INFO:teuthology.orchestra.run.smithi071.stdout:4354 writesame 0x5545acb thru 0x554cc69 data_size 0x1d(0x719f bytes) 2024-09-14T05:30:13.913 INFO:teuthology.orchestra.run.smithi071.stdout:4355 clone 35 order 19 su 8192 sc 2 2024-09-14T05:30:13.916 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:30:14.010 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:14.010+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:14.036 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:14.036 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:14.053 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:14.053 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:14.054 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:14.054 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:14.054 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:14.054 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:14.063 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:30:14.068 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:14.069 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:14.084 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:14.084 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:14.100 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:14.101 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:14.111 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:14.110+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:14.117 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:14.117 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:14.134 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:14.134 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:14.150 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:14.151 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:14.168 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:14.168 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:14.177 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:30:14.220 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:14.220+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:14.280 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:14.280 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:14.289 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:30:14.289 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T05:30:14.297 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-14T05:30:14.306 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T05:30:14.307 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:30:14.330 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:14.329+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:14.386 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone34 from 0xe41efa1 (overlap 0xf5b152) to 0xdd1f96 2024-09-14T05:30:14.413 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:14.412+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:14.439 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:14.439 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:14.448 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:30:14.508 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-14T05:30:14.520 INFO:teuthology.orchestra.run.smithi071.stdout:4356 writesame 0x3af7672 thru 0x3b0354d data_size 0x23(0xbedc bytes) 2024-09-14T05:30:14.532 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:14.532+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:14.569 INFO:teuthology.orchestra.run.smithi071.stdout:4357 read 0xdbb9f91 thru 0xdbc48c8 (0xa938 bytes) 2024-09-14T05:30:14.569 INFO:teuthology.orchestra.run.smithi071.stdout:4358 punch from 0x69aa5d4 to 0x69b2981, (0x83ad bytes) 2024-09-14T05:30:14.569 INFO:teuthology.orchestra.run.smithi071.stdout:4360 punch from 0xabf174c to 0xabfcf48, (0xb7fc bytes) 2024-09-14T05:30:14.569 INFO:teuthology.orchestra.run.smithi071.stdout:4361 read 0xe349dfd thru 0xe34b960 (0x1b64 bytes) 2024-09-14T05:30:14.571 INFO:teuthology.orchestra.run.smithi071.stdout:4362 trunc from 0xe41efa1 to 0xbba4239 2024-09-14T05:30:14.581 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:14.581 INFO:teuthology.orchestra.run.smithi071.stdout:4364 read 0xa65f918 thru 0xa66d5e9 (0xdcd2 bytes) 2024-09-14T05:30:14.589 INFO:teuthology.orchestra.run.smithi071.stdout:4367 write 0xd930909 thru 0xd937134 (0x682c bytes) 2024-09-14T05:30:14.594 INFO:teuthology.orchestra.run.smithi071.stdout:4368 writesame 0x1252fcd thru 0x1257ff6 data_size 0x1f(0x502a bytes) 2024-09-14T05:30:14.595 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:14.595 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:14.604 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:30:14.607 INFO:teuthology.orchestra.run.smithi071.stdout:4370 trunc from 0xd937135 to 0x69fb13a 2024-09-14T05:30:14.623 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:14.630 INFO:teuthology.orchestra.run.smithi071.stdout:4372 write 0x7793fd8 thru 0x77a21a6 (0xe1cf bytes) 2024-09-14T05:30:14.636 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:14.645 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:30:14.645 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:30:14.646 INFO:teuthology.orchestra.run.smithi071.stdout:4377 write 0xa6f0455 thru 0xa6f711d (0x6cc9 bytes) 2024-09-14T05:30:14.647 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:14.645+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:14.653 INFO:teuthology.orchestra.run.smithi071.stdout:4378 punch from 0x8fdd1d to 0x9053c7, (0x76aa bytes) 2024-09-14T05:30:14.661 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-09-14T05:30:14.446791+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1074029964,"num_objects":1904,"num_object_clones":487,"num_object_copies":3808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1904,"num_whiteouts":1007,"num_read":142225,"num_read_kb":448927,"num_write":12795,"num_write_kb":223297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11109,"ondisk_log_size":11109,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":572316,"kb_used_data":268664,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754402404,"statfs":{"total":773094113280,"available":772508061696,"internally_reserved":0,"allocated":275111936,"data_stored":280941585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43269347,"num_objects":120,"num_object_clones":35,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":42,"num_read":7870,"num_read_kb":22146,"num_write":2069,"num_write_kb":12337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000710"},"pg_stats":[{"pgid":"3.1f","version":"65'448","reported_seq":9587,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.289555+0000","last_change":"2024-09-14T05:29:26.019843+0000","last_active":"2024-09-14T05:30:11.289555+0000","last_peered":"2024-09-14T05:30:11.289555+0000","last_clean":"2024-09-14T05:30:11.289555+0000","last_became_active":"2024-09-14T05:29:24.018892+0000","last_became_peered":"2024-09-14T05:29:24.018892+0000","last_unstale":"2024-09-14T05:30:11.289555+0000","last_undegraded":"2024-09-14T05:30:11.289555+0000","last_fullsized":"2024-09-14T05:30:11.289555+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:18:05.949158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011130109999999999,"stat_sum":{"num_bytes":42568241,"num_objects":62,"num_object_clones":15,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":32,"num_read":10616,"num_read_kb":33599,"num_write":552,"num_write_kb":28618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"66'233","reported_seq":1704,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.345545+0000","last_change":"2024-09-14T05:29:26.019068+0000","last_active":"2024-09-14T05:30:11.345545+0000","last_peered":"2024-09-14T05:30:11.345545+0000","last_clean":"2024-09-14T05:30:11.345545+0000","last_became_active":"2024-09-14T05:29:24.014468+0000","last_became_peered":"2024-09-14T05:29:24.014468+0000","last_unstale":"2024-09-14T05:30:11.345545+0000","last_undegraded":"2024-09-14T05:30:11.345545+0000","last_fullsized":"2024-09-14T05:30:11.345545+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:31:09.297175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000399031,"stat_sum":{"num_bytes":53702062,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":27,"num_read":1627,"num_read_kb":22111,"num_write":246,"num_write_kb":19416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"65'189","reported_seq":8573,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.257472+0000","last_change":"2024-09-14T05:29:26.165031+0000","last_active":"2024-09-14T05:30:11.257472+0000","last_peered":"2024-09-14T05:30:11.257472+0000","last_clean":"2024-09-14T05:30:11.257472+0000","last_became_active":"2024-09-14T05:29:24.014251+0000","last_became_peered":"2024-09-14T05:29:24.014251+0000","last_unstale":"2024-09-14T05:30:11.257472+0000","last_undegraded":"2024-09-14T05:30:11.257472+0000","last_fullsized":"2024-09-14T05:30:11.257472+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:12:24.549903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010261580000000001,"stat_sum":{"num_bytes":53832863,"num_objects":60,"num_object_clones":20,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":31,"num_read":8335,"num_read_kb":19767,"num_write":174,"num_write_kb":4655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"66'339","reported_seq":9343,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.393833+0000","last_change":"2024-09-14T05:29:26.014243+0000","last_active":"2024-09-14T05:30:11.393833+0000","last_peered":"2024-09-14T05:30:11.393833+0000","last_clean":"2024-09-14T05:30:11.393833+0000","last_became_active":"2024-09-14T05:29:24.012427+0000","last_became_peered":"2024-09-14T05:29:24.012427+0000","last_unstale":"2024-09-14T05:30:11.393833+0000","last_undegraded":"2024-09-14T05:30:11.393833+0000","last_fullsized":"2024-09-14T05:30:11.393833+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:24:31.534521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045797999999999998,"stat_sum":{"num_bytes":62236263,"num_objects":73,"num_object_clones":20,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":38,"num_read":9953,"num_read_kb":33335,"num_write":330,"num_write_kb":4156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"65'366","reported_seq":2772,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256276+0000","last_change":"2024-09-14T05:29:26.014462+0000","last_active":"2024-09-14T05:30:11.256276+0000","last_peered":"2024-09-14T05:30:11.256276+0000","last_clean":"2024-09-14T05:30:11.256276+0000","last_became_active":"2024-09-14T05:29:24.012079+0000","last_became_peered":"2024-09-14T05:29:24.012079+0000","last_unstale":"2024-09-14T05:30:11.256276+0000","last_undegraded":"2024-09-14T05:30:11.256276+0000","last_fullsized":"2024-09-14T05:30:11.256276+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:27:44.817359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070589999999999997,"stat_sum":{"num_bytes":37003506,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":34,"num_read":3716,"num_read_kb":24505,"num_write":409,"num_write_kb":21331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"66'667","reported_seq":3882,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.453187+0000","last_change":"2024-09-14T05:29:26.014453+0000","last_active":"2024-09-14T05:30:11.453187+0000","last_peered":"2024-09-14T05:30:11.453187+0000","last_clean":"2024-09-14T05:30:11.453187+0000","last_became_active":"2024-09-14T05:29:24.012461+0000","last_became_peered":"2024-09-14T05:29:24.012461+0000","last_unstale":"2024-09-14T05:30:11.453187+0000","last_undegraded":"2024-09-14T05:30:11.453187+0000","last_fullsized":"2024-09-14T05:30:11.453187+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:30:20.541523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061290100000000001,"stat_sum":{"num_bytes":26977201,"num_objects":71,"num_object_clones":18,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":36,"num_read":6434,"num_read_kb":29871,"num_write":955,"num_write_kb":5976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"65'232","reported_seq":1544,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256274+0000","last_change":"2024-09-14T05:29:26.014685+0000","last_active":"2024-09-14T05:30:11.256274+0000","last_peered":"2024-09-14T05:30:11.256274+0000","last_clean":"2024-09-14T05:30:11.256274+0000","last_became_active":"2024-09-14T05:29:24.018753+0000","last_became_peered":"2024-09-14T05:29:24.018753+0000","last_unstale":"2024-09-14T05:30:11.256274+0000","last_undegraded":"2024-09-14T05:30:11.256274+0000","last_fullsized":"2024-09-14T05:30:11.256274+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:21:54.507503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089943399999999995,"stat_sum":{"num_bytes":40471085,"num_objects":70,"num_object_clones":19,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":31,"num_read":1370,"num_read_kb":12605,"num_write":218,"num_write_kb":8710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"64'430","reported_seq":2073,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.289593+0000","last_change":"2024-09-14T05:29:26.019928+0000","last_active":"2024-09-14T05:30:11.289593+0000","last_peered":"2024-09-14T05:30:11.289593+0000","last_clean":"2024-09-14T05:30:11.289593+0000","last_became_active":"2024-09-14T05:29:24.011176+0000","last_became_peered":"2024-09-14T05:29:24.011176+0000","last_unstale":"2024-09-14T05:30:11.289593+0000","last_undegraded":"2024-09-14T05:30:11.289593+0000","last_fullsized":"2024-09-14T05:30:11.289593+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:43:15.679468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011097850000000001,"stat_sum":{"num_bytes":12323793,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":25,"num_read":3993,"num_read_kb":6803,"num_write":480,"num_write_kb":2570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"64'235","reported_seq":1771,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.289583+0000","last_change":"2024-09-14T05:29:26.019132+0000","last_active":"2024-09-14T05:30:11.289583+0000","last_peered":"2024-09-14T05:30:11.289583+0000","last_clean":"2024-09-14T05:30:11.289583+0000","last_became_active":"2024-09-14T05:29:24.014005+0000","last_became_peered":"2024-09-14T05:29:24.014005+0000","last_unstale":"2024-09-14T05:30:11.289583+0000","last_undegraded":"2024-09-14T05:30:11.289583+0000","last_fullsized":"2024-09-14T05:30:11.289583+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:41.814918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045225300000000001,"stat_sum":{"num_bytes":7394881,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":22,"num_read":2050,"num_read_kb":4565,"num_write":226,"num_write_kb":3006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"63'232","reported_seq":9335,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.257399+0000","last_change":"2024-09-14T05:29:26.028571+0000","last_active":"2024-09-14T05:30:11.257399+0000","last_peered":"2024-09-14T05:30:11.257399+0000","last_clean":"2024-09-14T05:30:11.257399+0000","last_became_active":"2024-09-14T05:29:24.013434+0000","last_became_peered":"2024-09-14T05:29:24.013434+0000","last_unstale":"2024-09-14T05:30:11.257399+0000","last_undegraded":"2024-09-14T05:30:11.257399+0000","last_fullsized":"2024-09-14T05:30:11.257399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:35:46.978990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038301200000000001,"stat_sum":{"num_bytes":53492260,"num_objects":63,"num_object_clones":18,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":37,"num_read":9055,"num_read_kb":20637,"num_write":224,"num_write_kb":6734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"67'272","reported_seq":2152,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T05:30:12.557046+0000","last_change":"2024-09-14T05:29:26.020364+0000","last_active":"2024-09-14T05:30:12.557046+0000","last_peered":"2024-09-14T05:30:12.557046+0000","last_clean":"2024-09-14T05:30:12.557046+0000","last_became_active":"2024-09-14T05:29:24.014457+0000","last_became_peered":"2024-09-14T05:29:24.014457+0000","last_unstale":"2024-09-14T05:30:12.557046+0000","last_undegraded":"2024-09-14T05:30:12.557046+0000","last_fullsized":"2024-09-14T05:30:12.557046+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:00:48.968877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010888930000000001,"stat_sum":{"num_bytes":74500397,"num_objects":63,"num_object_clones":19,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":29,"num_read":2367,"num_read_kb":8189,"num_write":296,"num_write_kb":11575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"66'314","reported_seq":1497,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.343505+0000","last_change":"2024-09-14T05:29:26.164291+0000","last_active":"2024-09-14T05:30:11.343505+0000","last_peered":"2024-09-14T05:30:11.343505+0000","last_clean":"2024-09-14T05:30:11.343505+0000","last_became_active":"2024-09-14T05:29:24.013266+0000","last_became_peered":"2024-09-14T05:29:24.013266+0000","last_unstale":"2024-09-14T05:30:11.343505+0000","last_undegraded":"2024-09-14T05:30:11.343505+0000","last_fullsized":"2024-09-14T05:30:11.343505+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:26:41.280756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060129699999999999,"stat_sum":{"num_bytes":15017115,"num_objects":53,"num_object_clones":9,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":34,"num_read":2194,"num_read_kb":2850,"num_write":321,"num_write_kb":2896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":108,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.257324+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:30:11.257324+0000","last_peered":"2024-09-14T05:30:11.257324+0000","last_clean":"2024-09-14T05:30:11.257324+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:30:11.257324+0000","last_undegraded":"2024-09-14T05:30:11.257324+0000","last_fullsized":"2024-09-14T05:30:11.257324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"66'197","reported_seq":1466,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.374550+0000","last_change":"2024-09-14T05:29:26.014725+0000","last_active":"2024-09-14T05:30:11.374550+0000","last_peered":"2024-09-14T05:30:11.374550+0000","last_clean":"2024-09-14T05:30:11.374550+0000","last_became_active":"2024-09-14T05:29:24.012099+0000","last_became_peered":"2024-09-14T05:29:24.012099+0000","last_unstale":"2024-09-14T05:30:11.374550+0000","last_undegraded":"2024-09-14T05:30:11.374550+0000","last_fullsized":"2024-09-14T05:30:11.374550+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:53:28.191127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000794599,"stat_sum":{"num_bytes":9244050,"num_objects":48,"num_object_clones":14,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":27,"num_read":1629,"num_read_kb":2120,"num_write":185,"num_write_kb":2285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":106,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256453+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:30:11.256453+0000","last_peered":"2024-09-14T05:30:11.256453+0000","last_clean":"2024-09-14T05:30:11.256453+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:30:11.256453+0000","last_undegraded":"2024-09-14T05:30:11.256453+0000","last_fullsized":"2024-09-14T05:30:11.256453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"65'609","reported_seq":2225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256362+0000","last_change":"2024-09-14T05:29:26.015082+0000","last_active":"2024-09-14T05:30:11.256362+0000","last_peered":"2024-09-14T05:30:11.256362+0000","last_clean":"2024-09-14T05:30:11.256362+0000","last_became_active":"2024-09-14T05:29:24.011481+0000","last_became_peered":"2024-09-14T05:29:24.011481+0000","last_unstale":"2024-09-14T05:30:11.256362+0000","last_undegraded":"2024-09-14T05:30:11.256362+0000","last_fullsized":"2024-09-14T05:30:11.256362+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:40:43.877931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011470110000000001,"stat_sum":{"num_bytes":44366438,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":28,"num_read":4439,"num_read_kb":4313,"num_write":806,"num_write_kb":2205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":108,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.257636+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:30:11.257636+0000","last_peered":"2024-09-14T05:30:11.257636+0000","last_clean":"2024-09-14T05:30:11.257636+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:30:11.257636+0000","last_undegraded":"2024-09-14T05:30:11.257636+0000","last_fullsized":"2024-09-14T05:30:11.257636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"66'402","reported_seq":1568,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.415217+0000","last_change":"2024-09-14T05:29:40.095950+0000","last_active":"2024-09-14T05:30:11.415217+0000","last_peered":"2024-09-14T05:30:11.415217+0000","last_clean":"2024-09-14T05:30:11.415217+0000","last_became_active":"2024-09-14T05:29:40.095441+0000","last_became_peered":"2024-09-14T05:29:40.095441+0000","last_unstale":"2024-09-14T05:30:11.415217+0000","last_undegraded":"2024-09-14T05:30:11.415217+0000","last_fullsized":"2024-09-14T05:30:11.415217+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:21:10.196955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053582499999999997,"stat_sum":{"num_bytes":20088036,"num_objects":61,"num_object_clones":15,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":36,"num_read":2300,"num_read_kb":9494,"num_write":487,"num_write_kb":8836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.289493+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:30:11.289493+0000","last_peered":"2024-09-14T05:30:11.289493+0000","last_clean":"2024-09-14T05:30:11.289493+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:30:11.289493+0000","last_undegraded":"2024-09-14T05:30:11.289493+0000","last_fullsized":"2024-09-14T05:30:11.289493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"64'224","reported_seq":1481,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.257029+0000","last_change":"2024-09-14T05:29:26.028547+0000","last_active":"2024-09-14T05:30:11.257029+0000","last_peered":"2024-09-14T05:30:11.257029+0000","last_clean":"2024-09-14T05:30:11.257029+0000","last_became_active":"2024-09-14T05:29:24.016940+0000","last_became_peered":"2024-09-14T05:29:24.016940+0000","last_unstale":"2024-09-14T05:30:11.257029+0000","last_undegraded":"2024-09-14T05:30:11.257029+0000","last_fullsized":"2024-09-14T05:30:11.257029+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:19:55.576781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042396399999999998,"stat_sum":{"num_bytes":11629336,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":27,"num_read":1511,"num_read_kb":4295,"num_write":235,"num_write_kb":2921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"67'295","reported_seq":1569,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T05:30:12.539706+0000","last_change":"2024-09-14T05:29:26.019829+0000","last_active":"2024-09-14T05:30:12.539706+0000","last_peered":"2024-09-14T05:30:12.539706+0000","last_clean":"2024-09-14T05:30:12.539706+0000","last_became_active":"2024-09-14T05:29:24.015905+0000","last_became_peered":"2024-09-14T05:29:24.015905+0000","last_unstale":"2024-09-14T05:30:12.539706+0000","last_undegraded":"2024-09-14T05:30:12.539706+0000","last_fullsized":"2024-09-14T05:30:12.539706+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:23:40.712802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049652099999999998,"stat_sum":{"num_bytes":14384859,"num_objects":73,"num_object_clones":21,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":42,"num_read":1602,"num_read_kb":5831,"num_write":345,"num_write_kb":4754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":110,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.258000+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:30:11.258000+0000","last_peered":"2024-09-14T05:30:11.258000+0000","last_clean":"2024-09-14T05:30:11.258000+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:30:11.258000+0000","last_undegraded":"2024-09-14T05:30:11.258000+0000","last_fullsized":"2024-09-14T05:30:11.258000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"66'308","reported_seq":10227,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.341709+0000","last_change":"2024-09-14T05:29:26.019261+0000","last_active":"2024-09-14T05:30:11.341709+0000","last_peered":"2024-09-14T05:30:11.341709+0000","last_clean":"2024-09-14T05:30:11.341709+0000","last_became_active":"2024-09-14T05:29:24.011764+0000","last_became_peered":"2024-09-14T05:29:24.011764+0000","last_unstale":"2024-09-14T05:30:11.341709+0000","last_undegraded":"2024-09-14T05:30:11.341709+0000","last_fullsized":"2024-09-14T05:30:11.341709+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:03:40.768055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055502299999999997,"stat_sum":{"num_bytes":42046776,"num_objects":60,"num_object_clones":14,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":31,"num_read":11201,"num_read_kb":37709,"num_write":280,"num_write_kb":4114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":108,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.303213+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:30:11.303213+0000","last_peered":"2024-09-14T05:30:11.303213+0000","last_clean":"2024-09-14T05:30:11.303213+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:30:11.303213+0000","last_undegraded":"2024-09-14T05:30:11.303213+0000","last_fullsized":"2024-09-14T05:30:11.303213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"66'565","reported_seq":1462,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256988+0000","last_change":"2024-09-14T05:29:40.095721+0000","last_active":"2024-09-14T05:30:11.256988+0000","last_peered":"2024-09-14T05:30:11.256988+0000","last_clean":"2024-09-14T05:30:11.256988+0000","last_became_active":"2024-09-14T05:29:40.095255+0000","last_became_peered":"2024-09-14T05:29:40.095255+0000","last_unstale":"2024-09-14T05:30:11.256988+0000","last_undegraded":"2024-09-14T05:30:11.256988+0000","last_fullsized":"2024-09-14T05:30:11.256988+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:06:37.866414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077857800000000004,"stat_sum":{"num_bytes":49326784,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":25,"num_read":2877,"num_read_kb":5289,"num_write":813,"num_write_kb":8865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":108,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256588+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:30:11.256588+0000","last_peered":"2024-09-14T05:30:11.256588+0000","last_clean":"2024-09-14T05:30:11.256588+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:30:11.256588+0000","last_undegraded":"2024-09-14T05:30:11.256588+0000","last_fullsized":"2024-09-14T05:30:11.256588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"66'433","reported_seq":2119,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.417016+0000","last_change":"2024-09-14T05:29:26.015124+0000","last_active":"2024-09-14T05:30:11.417016+0000","last_peered":"2024-09-14T05:30:11.417016+0000","last_clean":"2024-09-14T05:30:11.417016+0000","last_became_active":"2024-09-14T05:29:24.015792+0000","last_became_peered":"2024-09-14T05:29:24.015792+0000","last_unstale":"2024-09-14T05:30:11.417016+0000","last_undegraded":"2024-09-14T05:30:11.417016+0000","last_fullsized":"2024-09-14T05:30:11.417016+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:56:13.066139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001049964,"stat_sum":{"num_bytes":16760193,"num_objects":67,"num_object_clones":16,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":40,"num_read":3198,"num_read_kb":7568,"num_write":508,"num_write_kb":3535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":109,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256992+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:30:11.256992+0000","last_peered":"2024-09-14T05:30:11.256992+0000","last_clean":"2024-09-14T05:30:11.256992+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:30:11.256992+0000","last_undegraded":"2024-09-14T05:30:11.256992+0000","last_fullsized":"2024-09-14T05:30:11.256992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":199,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.261017+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:30:11.261017+0000","last_peered":"2024-09-14T05:30:11.261017+0000","last_clean":"2024-09-14T05:30:11.261017+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:30:11.261017+0000","last_undegraded":"2024-09-14T05:30:11.261017+0000","last_fullsized":"2024-09-14T05:30:11.261017+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"66'189","reported_seq":1060,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256394+0000","last_change":"2024-09-14T05:29:26.014430+0000","last_active":"2024-09-14T05:30:11.256394+0000","last_peered":"2024-09-14T05:30:11.256394+0000","last_clean":"2024-09-14T05:30:11.256394+0000","last_became_active":"2024-09-14T05:29:24.012222+0000","last_became_peered":"2024-09-14T05:29:24.012222+0000","last_unstale":"2024-09-14T05:30:11.256394+0000","last_undegraded":"2024-09-14T05:30:11.256394+0000","last_fullsized":"2024-09-14T05:30:11.256394+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:22:30.510692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076580099999999998,"stat_sum":{"num_bytes":26728033,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":27,"num_read":834,"num_read_kb":4295,"num_write":161,"num_write_kb":3445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"66'270","reported_seq":1092,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.398480+0000","last_change":"2024-09-14T05:29:26.164542+0000","last_active":"2024-09-14T05:30:11.398480+0000","last_peered":"2024-09-14T05:30:11.398480+0000","last_clean":"2024-09-14T05:30:11.398480+0000","last_became_active":"2024-09-14T05:29:24.012041+0000","last_became_peered":"2024-09-14T05:29:24.012041+0000","last_unstale":"2024-09-14T05:30:11.398480+0000","last_undegraded":"2024-09-14T05:30:11.398480+0000","last_fullsized":"2024-09-14T05:30:11.398480+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:40:03.452031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078334300000000004,"stat_sum":{"num_bytes":12716752,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":25,"num_read":1105,"num_read_kb":1839,"num_write":286,"num_write_kb":2491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"66'444","reported_seq":1845,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.383567+0000","last_change":"2024-09-14T05:29:26.165164+0000","last_active":"2024-09-14T05:30:11.383567+0000","last_peered":"2024-09-14T05:30:11.383567+0000","last_clean":"2024-09-14T05:30:11.383567+0000","last_became_active":"2024-09-14T05:29:24.011403+0000","last_became_peered":"2024-09-14T05:29:24.011403+0000","last_unstale":"2024-09-14T05:30:11.383567+0000","last_undegraded":"2024-09-14T05:30:11.383567+0000","last_fullsized":"2024-09-14T05:30:11.383567+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:55.865120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083114500000000004,"stat_sum":{"num_bytes":18545837,"num_objects":64,"num_object_clones":18,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":30,"num_read":2984,"num_read_kb":7171,"num_write":548,"num_write_kb":4514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"66'436","reported_seq":9409,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.398029+0000","last_change":"2024-09-14T05:29:26.163783+0000","last_active":"2024-09-14T05:30:11.398029+0000","last_peered":"2024-09-14T05:30:11.398029+0000","last_clean":"2024-09-14T05:30:11.398029+0000","last_became_active":"2024-09-14T05:29:24.006836+0000","last_became_peered":"2024-09-14T05:29:24.006836+0000","last_unstale":"2024-09-14T05:30:11.398029+0000","last_undegraded":"2024-09-14T05:30:11.398029+0000","last_fullsized":"2024-09-14T05:30:11.398029+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:04:42.666646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031188400000000001,"stat_sum":{"num_bytes":37332916,"num_objects":64,"num_object_clones":10,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":35,"num_read":11206,"num_read_kb":29064,"num_write":535,"num_write_kb":12685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"66'198","reported_seq":2421,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T05:30:12.320668+0000","last_change":"2024-09-14T05:29:26.020113+0000","last_active":"2024-09-14T05:30:12.320668+0000","last_peered":"2024-09-14T05:30:12.320668+0000","last_clean":"2024-09-14T05:30:12.320668+0000","last_became_active":"2024-09-14T05:29:24.012574+0000","last_became_peered":"2024-09-14T05:29:24.012574+0000","last_unstale":"2024-09-14T05:30:12.320668+0000","last_undegraded":"2024-09-14T05:30:12.320668+0000","last_fullsized":"2024-09-14T05:30:12.320668+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:15:13.596278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072680599999999998,"stat_sum":{"num_bytes":39953671,"num_objects":60,"num_object_clones":20,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":32,"num_read":2199,"num_read_kb":8085,"num_write":166,"num_write_kb":5126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"66'774","reported_seq":9609,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.428007+0000","last_change":"2024-09-14T05:29:26.164736+0000","last_active":"2024-09-14T05:30:11.428007+0000","last_peered":"2024-09-14T05:30:11.428007+0000","last_clean":"2024-09-14T05:30:11.428007+0000","last_became_active":"2024-09-14T05:29:24.012401+0000","last_became_peered":"2024-09-14T05:29:24.012401+0000","last_unstale":"2024-09-14T05:30:11.428007+0000","last_undegraded":"2024-09-14T05:30:11.428007+0000","last_fullsized":"2024-09-14T05:30:11.428007+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:25:12.169350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085260200000000005,"stat_sum":{"num_bytes":43455562,"num_objects":72,"num_object_clones":18,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":40,"num_read":11810,"num_read_kb":19585,"num_write":1063,"num_write_kb":4975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"66'194","reported_seq":1587,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.447905+0000","last_change":"2024-09-14T05:29:26.028811+0000","last_active":"2024-09-14T05:30:11.447905+0000","last_peered":"2024-09-14T05:30:11.447905+0000","last_clean":"2024-09-14T05:30:11.447905+0000","last_became_active":"2024-09-14T05:29:24.015315+0000","last_became_peered":"2024-09-14T05:29:24.015315+0000","last_unstale":"2024-09-14T05:30:11.447905+0000","last_undegraded":"2024-09-14T05:30:11.447905+0000","last_fullsized":"2024-09-14T05:30:11.447905+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:55:30.171267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046485199999999997,"stat_sum":{"num_bytes":25287542,"num_objects":65,"num_object_clones":20,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":39,"num_read":1331,"num_read_kb":2486,"num_write":169,"num_write_kb":3014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"67'309","reported_seq":10735,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T05:30:12.481411+0000","last_change":"2024-09-14T05:29:26.020027+0000","last_active":"2024-09-14T05:30:12.481411+0000","last_peered":"2024-09-14T05:30:12.481411+0000","last_clean":"2024-09-14T05:30:12.481411+0000","last_became_active":"2024-09-14T05:29:24.018854+0000","last_became_peered":"2024-09-14T05:29:24.018854+0000","last_unstale":"2024-09-14T05:30:12.481411+0000","last_undegraded":"2024-09-14T05:30:12.481411+0000","last_fullsized":"2024-09-14T05:30:12.481411+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:56.376950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063672000000000002,"stat_sum":{"num_bytes":34972261,"num_objects":83,"num_object_clones":22,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":45,"num_read":11220,"num_read_kb":29115,"num_write":284,"num_write_kb":10826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"65'229","reported_seq":1234,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.402693+0000","last_change":"2024-09-14T05:29:26.014200+0000","last_active":"2024-09-14T05:30:11.402693+0000","last_peered":"2024-09-14T05:30:11.402693+0000","last_clean":"2024-09-14T05:30:11.402693+0000","last_became_active":"2024-09-14T05:29:24.010897+0000","last_became_peered":"2024-09-14T05:29:24.010897+0000","last_unstale":"2024-09-14T05:30:11.402693+0000","last_undegraded":"2024-09-14T05:30:11.402693+0000","last_fullsized":"2024-09-14T05:30:11.402693+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:37:30.543147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051531799999999996,"stat_sum":{"num_bytes":33936373,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":26,"num_read":1014,"num_read_kb":8865,"num_write":202,"num_write_kb":6050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"66'344","reported_seq":2324,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.431151+0000","last_change":"2024-09-14T05:29:26.164788+0000","last_active":"2024-09-14T05:30:11.431151+0000","last_peered":"2024-09-14T05:30:11.431151+0000","last_clean":"2024-09-14T05:30:11.431151+0000","last_became_active":"2024-09-14T05:29:24.013792+0000","last_became_peered":"2024-09-14T05:29:24.013792+0000","last_unstale":"2024-09-14T05:30:11.431151+0000","last_undegraded":"2024-09-14T05:30:11.431151+0000","last_fullsized":"2024-09-14T05:30:11.431151+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:08:01.065225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090671299999999996,"stat_sum":{"num_bytes":29225047,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":3686,"num_read_kb":15807,"num_write":343,"num_write_kb":2409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"65'211","reported_seq":1246,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.417214+0000","last_change":"2024-09-14T05:29:40.095745+0000","last_active":"2024-09-14T05:30:11.417214+0000","last_peered":"2024-09-14T05:30:11.417214+0000","last_clean":"2024-09-14T05:30:11.417214+0000","last_became_active":"2024-09-14T05:29:40.095331+0000","last_became_peered":"2024-09-14T05:29:40.095331+0000","last_unstale":"2024-09-14T05:30:11.417214+0000","last_undegraded":"2024-09-14T05:30:11.417214+0000","last_fullsized":"2024-09-14T05:30:11.417214+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:29:19.959417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000774981,"stat_sum":{"num_bytes":27414058,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":1534,"num_read_kb":8788,"num_write":192,"num_write_kb":4039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"66'446","reported_seq":1759,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256917+0000","last_change":"2024-09-14T05:29:26.164295+0000","last_active":"2024-09-14T05:30:11.256917+0000","last_peered":"2024-09-14T05:30:11.256917+0000","last_clean":"2024-09-14T05:30:11.256917+0000","last_became_active":"2024-09-14T05:29:24.014371+0000","last_became_peered":"2024-09-14T05:29:24.014371+0000","last_unstale":"2024-09-14T05:30:11.256917+0000","last_undegraded":"2024-09-14T05:30:11.256917+0000","last_fullsized":"2024-09-14T05:30:11.256917+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:32:06.372111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050570599999999997,"stat_sum":{"num_bytes":56505386,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":26,"num_read":2769,"num_read_kb":18316,"num_write":653,"num_write_kb":4935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1073439577,"num_objects":1900,"num_object_clones":487,"num_object_copies":3800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1900,"num_whiteouts":1007,"num_read":142159,"num_read_kb":448872,"num_write":12692,"num_write_kb":221667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":269176832,"data_stored":278252433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11034,"ondisk_log_size":11034,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476753,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88248,"kb_used_data":48740,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94283592,"statfs":{"total":96636764160,"available":96546398208,"internally_reserved":0,"allocated":49909760,"data_stored":50231088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53520,"kb_used_data":22328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94318320,"statfs":{"total":96636764160,"available":96581959680,"internally_reserved":0,"allocated":22863872,"data_stored":22825640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66656,"kb_used_data":27140,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94305184,"statfs":{"total":96636764160,"available":96568508416,"internally_reserved":0,"allocated":27791360,"data_stored":29005176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83860,"kb_used_data":44352,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94287980,"statfs":{"total":96636764160,"available":96550891520,"internally_reserved":0,"allocated":45416448,"data_stored":46429833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67008,"kb_used_data":31664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94304832,"statfs":{"total":96636764160,"available":96568147968,"internally_reserved":0,"allocated":32423936,"data_stored":32787746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63384,"kb_used_data":23844,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94308456,"statfs":{"total":96636764160,"available":96571858944,"internally_reserved":0,"allocated":24416256,"data_stored":25416164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64212,"kb_used_data":24708,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94307628,"statfs":{"total":96636764160,"available":96571011072,"internally_reserved":0,"allocated":25300992,"data_stored":26172013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85428,"kb_used_data":45888,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94286412,"statfs":{"total":96636764160,"available":96549285888,"internally_reserved":0,"allocated":46989312,"data_stored":48073925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49963008,"data_stored":50678949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45649920,"data_stored":47147496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24694784,"data_stored":25965649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23789568,"data_stored":25191018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":31768576,"data_stored":32529631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":44085248,"data_stored":45509553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":27037696,"data_stored":28669034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":22188032,"data_stored":22561103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:30:14.663 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:30:14.664 INFO:teuthology.orchestra.run.smithi071.stdout:4379 writesame 0xc62efb9 thru 0xc6303c2 data_size 0x13(0x140a bytes) 2024-09-14T05:30:14.675 INFO:teuthology.orchestra.run.smithi071.stdout:4380 trunc from 0xc6303c3 to 0x2bfb87 2024-09-14T05:30:14.706 INFO:teuthology.orchestra.run.smithi071.stdout:4381 read 0xf7918 thru 0x101233 (0x991c bytes) 2024-09-14T05:30:14.716 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:14.715+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:14.720 INFO:teuthology.orchestra.run.smithi071.stdout:4382 writesame 0x6516d8a thru 0x65203ee data_size 0x1(0x9665 bytes) 2024-09-14T05:30:14.720 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:14.720 INFO:teuthology.orchestra.run.smithi071.stdout:4384 read 0x366fb74 thru 0x366ffad (0x43a bytes) 2024-09-14T05:30:14.727 INFO:teuthology.orchestra.run.smithi071.stdout:4386 write 0x8991aca thru 0x89a14f9 (0xfa30 bytes) 2024-09-14T05:30:14.742 INFO:teuthology.orchestra.run.smithi071.stdout:4387 writesame 0xd3f4c93 thru 0xd3fe023 data_size 0x1(0x9391 bytes) 2024-09-14T05:30:14.742 INFO:teuthology.orchestra.run.smithi071.stdout:4388 write 0xbb3e574 thru 0xbb4b710 (0xd19d bytes) 2024-09-14T05:30:14.758 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:14.758 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:14.760 INFO:teuthology.orchestra.run.smithi071.stdout:4389 trunc from 0xd3fe024 to 0x111ac51 2024-09-14T05:30:14.767 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:30:14.783 INFO:teuthology.orchestra.run.smithi071.stdout:4391 write 0x781eb7d thru 0x78257b3 (0x6c37 bytes) 2024-09-14T05:30:14.791 INFO:teuthology.orchestra.run.smithi071.stdout:4392 write 0xe3c43 thru 0xea797 (0x6b55 bytes) 2024-09-14T05:30:14.830 INFO:teuthology.orchestra.run.smithi071.stdout:4393 punch from 0x4884737 to 0x48928f8, (0xe1c1 bytes) 2024-09-14T05:30:14.831 INFO:teuthology.orchestra.run.smithi071.stdout:4396 read 0x5b8e6aa thru 0x5b90d96 (0x26ed bytes) 2024-09-14T05:30:14.831 INFO:teuthology.orchestra.run.smithi071.stdout:4397 read 0x13ba045 thru 0x13c1cea (0x7ca6 bytes) 2024-09-14T05:30:14.831 INFO:teuthology.orchestra.run.smithi071.stdout:4401 writesame 0x15502fa thru 0x155f70e data_size 0x1(0xf415 bytes) 2024-09-14T05:30:14.851 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:14.851+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:14.854 INFO:teuthology.orchestra.run.smithi071.stdout:4402 trunc from 0x78257b4 to 0xb57ecbd 2024-09-14T05:30:14.859 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:14.866 INFO:teuthology.orchestra.run.smithi071.stdout:4405 write 0xca9a7ba thru 0xcaa962b (0xee72 bytes) 2024-09-14T05:30:14.872 INFO:teuthology.orchestra.run.smithi071.stdout:4406 punch from 0x77b763a to 0x77b77ab, (0x171 bytes) 2024-09-14T05:30:14.872 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:14.872 INFO:teuthology.orchestra.run.smithi071.stdout:4409 write 0x1fc3703 thru 0x1fce5e4 (0xaee2 bytes) 2024-09-14T05:30:14.879 INFO:teuthology.orchestra.run.smithi071.stdout:4410 write 0x7284825 thru 0x728b507 (0x6ce3 bytes) 2024-09-14T05:30:14.885 INFO:teuthology.orchestra.run.smithi071.stdout:4411 punch from 0x9560f3b to 0x9564e31, (0x3ef6 bytes) 2024-09-14T05:30:14.885 INFO:teuthology.orchestra.run.smithi071.stdout:4414 write 0x7fa779a thru 0x7fa7afc (0x363 bytes) 2024-09-14T05:30:14.888 INFO:teuthology.orchestra.run.smithi071.stdout:4415 read 0x74ceb0b thru 0x74d0071 (0x1567 bytes) 2024-09-14T05:30:14.888 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:14.888 INFO:teuthology.orchestra.run.smithi071.stdout:4418 read 0xa53c74b thru 0xa53d150 (0xa06 bytes) 2024-09-14T05:30:14.888 INFO:teuthology.orchestra.run.smithi071.stdout:4419 punch from 0x947c51a to 0x9488192, (0xbc78 bytes) 2024-09-14T05:30:14.888 INFO:teuthology.orchestra.run.smithi071.stdout:4420 write 0x2ce67cc thru 0x2cec5fa (0x5e2f bytes) 2024-09-14T05:30:14.893 INFO:teuthology.orchestra.run.smithi071.stdout:4421 punch from 0xaa1e1c1 to 0xaa286e1, (0xa520 bytes) 2024-09-14T05:30:14.894 INFO:teuthology.orchestra.run.smithi071.stdout:4422 punch from 0xacc356d to 0xacc945b, (0x5eee bytes) 2024-09-14T05:30:14.894 INFO:teuthology.orchestra.run.smithi071.stdout:4424 read 0xa20668b thru 0xa212c1f (0xc595 bytes) 2024-09-14T05:30:14.894 INFO:teuthology.orchestra.run.smithi071.stdout:4425 read 0x92ea969 thru 0x92fa004 (0xf69c bytes) 2024-09-14T05:30:14.894 INFO:teuthology.orchestra.run.smithi071.stdout:4426 write 0x2a9fa18 thru 0x2aabec0 (0xc4a9 bytes) 2024-09-14T05:30:14.900 INFO:teuthology.orchestra.run.smithi071.stdout:4427 writesame 0xafa7db3 thru 0xafab588 data_size 0x7(0x37d6 bytes) 2024-09-14T05:30:14.906 INFO:teuthology.orchestra.run.smithi071.stdout:4430 punch from 0x25c7bfa to 0x25cd3b1, (0x57b7 bytes) 2024-09-14T05:30:14.906 INFO:teuthology.orchestra.run.smithi071.stdout:4433 writesame 0x63c9a39 thru 0x63cbdf9 data_size 0x1b(0x23c1 bytes) 2024-09-14T05:30:14.914 INFO:teuthology.orchestra.run.smithi071.stdout:4434 read 0x355d33 thru 0x361bf9 (0xbec7 bytes) 2024-09-14T05:30:14.914 INFO:teuthology.orchestra.run.smithi071.stdout:4435 read 0x56d3254 thru 0x56db998 (0x8745 bytes) 2024-09-14T05:30:14.914 INFO:teuthology.orchestra.run.smithi071.stdout:4438 read 0xbb96195 thru 0xbb98ba7 (0x2a13 bytes) 2024-09-14T05:30:14.914 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:14.914 INFO:teuthology.orchestra.run.smithi071.stdout:4445 read 0x18390b thru 0x18ffb3 (0xc6a9 bytes) 2024-09-14T05:30:14.916 INFO:teuthology.orchestra.run.smithi071.stdout:4446 read 0x7a2f5fc thru 0x7a37c6e (0x8673 bytes) 2024-09-14T05:30:14.916 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:14.916 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:14.916 INFO:teuthology.orchestra.run.smithi071.stdout:4451 writesame 0x392193 thru 0x3a0864 data_size 0x5f(0xe6d2 bytes) 2024-09-14T05:30:14.917 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:14.917 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:14.917 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:14.916+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:14.920 INFO:teuthology.orchestra.run.smithi071.stdout:4453 read 0x9843009 thru 0x9850650 (0xd648 bytes) 2024-09-14T05:30:14.920 INFO:teuthology.orchestra.run.smithi071.stdout:4454 punch from 0x5777b54 to 0x5778ef9, (0x13a5 bytes) 2024-09-14T05:30:14.920 INFO:teuthology.orchestra.run.smithi071.stdout:4456 read 0x5a8d111 thru 0x5a941ad (0x709d bytes) 2024-09-14T05:30:14.920 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:14.920 INFO:teuthology.orchestra.run.smithi071.stdout:4458 punch from 0x6e81c0 to 0x6f1203, (0x9043 bytes) 2024-09-14T05:30:14.925 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:30:14.932 INFO:teuthology.orchestra.run.smithi071.stdout:4459 trunc from 0xcaa962c to 0x262a397 2024-09-14T05:30:14.944 INFO:teuthology.orchestra.run.smithi071.stdout:4460 read 0x1892b61 thru 0x189bbe9 (0x9089 bytes) 2024-09-14T05:30:14.945 INFO:teuthology.orchestra.run.smithi071.stdout:4461 trunc from 0x262a397 to 0x13bac82 2024-09-14T05:30:14.955 INFO:teuthology.orchestra.run.smithi071.stdout:4463 read 0xe46f68 thru 0xe47ab2 (0xb4b bytes) 2024-09-14T05:30:14.962 INFO:teuthology.orchestra.run.smithi071.stdout:4464 write 0x2df28e5 thru 0x2df9f7d (0x7699 bytes) 2024-09-14T05:30:14.969 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:14.975 INFO:teuthology.orchestra.run.smithi071.stdout:4467 write 0x38a0352 thru 0x38af6da (0xf389 bytes) 2024-09-14T05:30:14.982 INFO:teuthology.orchestra.run.smithi071.stdout:4469 punch from 0x1e501f2 to 0x1e5c1c3, (0xbfd1 bytes) 2024-09-14T05:30:14.983 INFO:teuthology.orchestra.run.smithi071.stdout:4470 punch from 0x78bb0a to 0x78d9c5, (0x1ebb bytes) 2024-09-14T05:30:14.991 INFO:teuthology.orchestra.run.smithi071.stdout:4471 write 0x64b5912 thru 0x64bf77f (0x9e6e bytes) 2024-09-14T05:30:15.005 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:30:15.005 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:30:15.011 INFO:teuthology.orchestra.run.smithi071.stdout:4472 writesame 0xe492e2e thru 0xe49ab2f data_size 0x1(0x7d02 bytes) 2024-09-14T05:30:15.011 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.012 INFO:teuthology.orchestra.run.smithi071.stdout:4475 write 0x4b223a5 thru 0x4b2a970 (0x85cc bytes) 2024-09-14T05:30:15.015 INFO:teuthology.orchestra.run.smithi071.stdout:4476 write 0xd6ea2d6 thru 0xd6ea452 (0x17d bytes) 2024-09-14T05:30:15.016 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-09-14T05:30:14.446791+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1074029964,"num_objects":1904,"num_object_clones":487,"num_object_copies":3808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1904,"num_whiteouts":1007,"num_read":142225,"num_read_kb":448927,"num_write":12795,"num_write_kb":223297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11109,"ondisk_log_size":11109,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":572316,"kb_used_data":268664,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754402404,"statfs":{"total":773094113280,"available":772508061696,"internally_reserved":0,"allocated":275111936,"data_stored":280941585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43269347,"num_objects":120,"num_object_clones":35,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":42,"num_read":7870,"num_read_kb":22146,"num_write":2069,"num_write_kb":12337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000710"},"pg_stats":[{"pgid":"3.1f","version":"65'448","reported_seq":9587,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.289555+0000","last_change":"2024-09-14T05:29:26.019843+0000","last_active":"2024-09-14T05:30:11.289555+0000","last_peered":"2024-09-14T05:30:11.289555+0000","last_clean":"2024-09-14T05:30:11.289555+0000","last_became_active":"2024-09-14T05:29:24.018892+0000","last_became_peered":"2024-09-14T05:29:24.018892+0000","last_unstale":"2024-09-14T05:30:11.289555+0000","last_undegraded":"2024-09-14T05:30:11.289555+0000","last_fullsized":"2024-09-14T05:30:11.289555+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:18:05.949158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011130109999999999,"stat_sum":{"num_bytes":42568241,"num_objects":62,"num_object_clones":15,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":32,"num_read":10616,"num_read_kb":33599,"num_write":552,"num_write_kb":28618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"66'233","reported_seq":1704,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.345545+0000","last_change":"2024-09-14T05:29:26.019068+0000","last_active":"2024-09-14T05:30:11.345545+0000","last_peered":"2024-09-14T05:30:11.345545+0000","last_clean":"2024-09-14T05:30:11.345545+0000","last_became_active":"2024-09-14T05:29:24.014468+0000","last_became_peered":"2024-09-14T05:29:24.014468+0000","last_unstale":"2024-09-14T05:30:11.345545+0000","last_undegraded":"2024-09-14T05:30:11.345545+0000","last_fullsized":"2024-09-14T05:30:11.345545+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:31:09.297175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000399031,"stat_sum":{"num_bytes":53702062,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":27,"num_read":1627,"num_read_kb":22111,"num_write":246,"num_write_kb":19416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"65'189","reported_seq":8573,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.257472+0000","last_change":"2024-09-14T05:29:26.165031+0000","last_active":"2024-09-14T05:30:11.257472+0000","last_peered":"2024-09-14T05:30:11.257472+0000","last_clean":"2024-09-14T05:30:11.257472+0000","last_became_active":"2024-09-14T05:29:24.014251+0000","last_became_peered":"2024-09-14T05:29:24.014251+0000","last_unstale":"2024-09-14T05:30:11.257472+0000","last_undegraded":"2024-09-14T05:30:11.257472+0000","last_fullsized":"2024-09-14T05:30:11.257472+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:12:24.549903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010261580000000001,"stat_sum":{"num_bytes":53832863,"num_objects":60,"num_object_clones":20,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":31,"num_read":8335,"num_read_kb":19767,"num_write":174,"num_write_kb":4655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"66'339","reported_seq":9343,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.393833+0000","last_change":"2024-09-14T05:29:26.014243+0000","last_active":"2024-09-14T05:30:11.393833+0000","last_peered":"2024-09-14T05:30:11.393833+0000","last_clean":"2024-09-14T05:30:11.393833+0000","last_became_active":"2024-09-14T05:29:24.012427+0000","last_became_peered":"2024-09-14T05:29:24.012427+0000","last_unstale":"2024-09-14T05:30:11.393833+0000","last_undegraded":"2024-09-14T05:30:11.393833+0000","last_fullsized":"2024-09-14T05:30:11.393833+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:24:31.534521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045797999999999998,"stat_sum":{"num_bytes":62236263,"num_objects":73,"num_object_clones":20,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":38,"num_read":9953,"num_read_kb":33335,"num_write":330,"num_write_kb":4156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"65'366","reported_seq":2772,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256276+0000","last_change":"2024-09-14T05:29:26.014462+0000","last_active":"2024-09-14T05:30:11.256276+0000","last_peered":"2024-09-14T05:30:11.256276+0000","last_clean":"2024-09-14T05:30:11.256276+0000","last_became_active":"2024-09-14T05:29:24.012079+0000","last_became_peered":"2024-09-14T05:29:24.012079+0000","last_unstale":"2024-09-14T05:30:11.256276+0000","last_undegraded":"2024-09-14T05:30:11.256276+0000","last_fullsized":"2024-09-14T05:30:11.256276+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:27:44.817359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070589999999999997,"stat_sum":{"num_bytes":37003506,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":34,"num_read":3716,"num_read_kb":24505,"num_write":409,"num_write_kb":21331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"66'667","reported_seq":3882,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.453187+0000","last_change":"2024-09-14T05:29:26.014453+0000","last_active":"2024-09-14T05:30:11.453187+0000","last_peered":"2024-09-14T05:30:11.453187+0000","last_clean":"2024-09-14T05:30:11.453187+0000","last_became_active":"2024-09-14T05:29:24.012461+0000","last_became_peered":"2024-09-14T05:29:24.012461+0000","last_unstale":"2024-09-14T05:30:11.453187+0000","last_undegraded":"2024-09-14T05:30:11.453187+0000","last_fullsized":"2024-09-14T05:30:11.453187+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:30:20.541523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061290100000000001,"stat_sum":{"num_bytes":26977201,"num_objects":71,"num_object_clones":18,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":36,"num_read":6434,"num_read_kb":29871,"num_write":955,"num_write_kb":5976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"65'232","reported_seq":1544,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256274+0000","last_change":"2024-09-14T05:29:26.014685+0000","last_active":"2024-09-14T05:30:11.256274+0000","last_peered":"2024-09-14T05:30:11.256274+0000","last_clean":"2024-09-14T05:30:11.256274+0000","last_became_active":"2024-09-14T05:29:24.018753+0000","last_became_peered":"2024-09-14T05:29:24.018753+0000","last_unstale":"2024-09-14T05:30:11.256274+0000","last_undegraded":"2024-09-14T05:30:11.256274+0000","last_fullsized":"2024-09-14T05:30:11.256274+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:21:54.507503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089943399999999995,"stat_sum":{"num_bytes":40471085,"num_objects":70,"num_object_clones":19,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":31,"num_read":1370,"num_read_kb":12605,"num_write":218,"num_write_kb":8710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"64'430","reported_seq":2073,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.289593+0000","last_change":"2024-09-14T05:29:26.019928+0000","last_active":"2024-09-14T05:30:11.289593+0000","last_peered":"2024-09-14T05:30:11.289593+0000","last_clean":"2024-09-14T05:30:11.289593+0000","last_became_active":"2024-09-14T05:29:24.011176+0000","last_became_peered":"2024-09-14T05:29:24.011176+0000","last_unstale":"2024-09-14T05:30:11.289593+0000","last_undegraded":"2024-09-14T05:30:11.289593+0000","last_fullsized":"2024-09-14T05:30:11.289593+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:43:15.679468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011097850000000001,"stat_sum":{"num_bytes":12323793,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":25,"num_read":3993,"num_read_kb":6803,"num_write":480,"num_write_kb":2570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"64'235","reported_seq":1771,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.289583+0000","last_change":"2024-09-14T05:29:26.019132+0000","last_active":"2024-09-14T05:30:11.289583+0000","last_peered":"2024-09-14T05:30:11.289583+0000","last_clean":"2024-09-14T05:30:11.289583+0000","last_became_active":"2024-09-14T05:29:24.014005+0000","last_became_peered":"2024-09-14T05:29:24.014005+0000","last_unstale":"2024-09-14T05:30:11.289583+0000","last_undegraded":"2024-09-14T05:30:11.289583+0000","last_fullsized":"2024-09-14T05:30:11.289583+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:41.814918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045225300000000001,"stat_sum":{"num_bytes":7394881,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":22,"num_read":2050,"num_read_kb":4565,"num_write":226,"num_write_kb":3006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"63'232","reported_seq":9335,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.257399+0000","last_change":"2024-09-14T05:29:26.028571+0000","last_active":"2024-09-14T05:30:11.257399+0000","last_peered":"2024-09-14T05:30:11.257399+0000","last_clean":"2024-09-14T05:30:11.257399+0000","last_became_active":"2024-09-14T05:29:24.013434+0000","last_became_peered":"2024-09-14T05:29:24.013434+0000","last_unstale":"2024-09-14T05:30:11.257399+0000","last_undegraded":"2024-09-14T05:30:11.257399+0000","last_fullsized":"2024-09-14T05:30:11.257399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:35:46.978990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038301200000000001,"stat_sum":{"num_bytes":53492260,"num_objects":63,"num_object_clones":18,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":37,"num_read":9055,"num_read_kb":20637,"num_write":224,"num_write_kb":6734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"67'272","reported_seq":2152,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T05:30:12.557046+0000","last_change":"2024-09-14T05:29:26.020364+0000","last_active":"2024-09-14T05:30:12.557046+0000","last_peered":"2024-09-14T05:30:12.557046+0000","last_clean":"2024-09-14T05:30:12.557046+0000","last_became_active":"2024-09-14T05:29:24.014457+0000","last_became_peered":"2024-09-14T05:29:24.014457+0000","last_unstale":"2024-09-14T05:30:12.557046+0000","last_undegraded":"2024-09-14T05:30:12.557046+0000","last_fullsized":"2024-09-14T05:30:12.557046+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:00:48.968877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010888930000000001,"stat_sum":{"num_bytes":74500397,"num_objects":63,"num_object_clones":19,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":29,"num_read":2367,"num_read_kb":8189,"num_write":296,"num_write_kb":11575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"66'314","reported_seq":1497,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.343505+0000","last_change":"2024-09-14T05:29:26.164291+0000","last_active":"2024-09-14T05:30:11.343505+0000","last_peered":"2024-09-14T05:30:11.343505+0000","last_clean":"2024-09-14T05:30:11.343505+0000","last_became_active":"2024-09-14T05:29:24.013266+0000","last_became_peered":"2024-09-14T05:29:24.013266+0000","last_unstale":"2024-09-14T05:30:11.343505+0000","last_undegraded":"2024-09-14T05:30:11.343505+0000","last_fullsized":"2024-09-14T05:30:11.343505+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:26:41.280756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060129699999999999,"stat_sum":{"num_bytes":15017115,"num_objects":53,"num_object_clones":9,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":34,"num_read":2194,"num_read_kb":2850,"num_write":321,"num_write_kb":2896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":108,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.257324+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:30:11.257324+0000","last_peered":"2024-09-14T05:30:11.257324+0000","last_clean":"2024-09-14T05:30:11.257324+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:30:11.257324+0000","last_undegraded":"2024-09-14T05:30:11.257324+0000","last_fullsized":"2024-09-14T05:30:11.257324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"66'197","reported_seq":1466,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.374550+0000","last_change":"2024-09-14T05:29:26.014725+0000","last_active":"2024-09-14T05:30:11.374550+0000","last_peered":"2024-09-14T05:30:11.374550+0000","last_clean":"2024-09-14T05:30:11.374550+0000","last_became_active":"2024-09-14T05:29:24.012099+0000","last_became_peered":"2024-09-14T05:29:24.012099+0000","last_unstale":"2024-09-14T05:30:11.374550+0000","last_undegraded":"2024-09-14T05:30:11.374550+0000","last_fullsized":"2024-09-14T05:30:11.374550+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:53:28.191127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000794599,"stat_sum":{"num_bytes":9244050,"num_objects":48,"num_object_clones":14,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":27,"num_read":1629,"num_read_kb":2120,"num_write":185,"num_write_kb":2285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":106,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256453+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:30:11.256453+0000","last_peered":"2024-09-14T05:30:11.256453+0000","last_clean":"2024-09-14T05:30:11.256453+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:30:11.256453+0000","last_undegraded":"2024-09-14T05:30:11.256453+0000","last_fullsized":"2024-09-14T05:30:11.256453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"65'609","reported_seq":2225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256362+0000","last_change":"2024-09-14T05:29:26.015082+0000","last_active":"2024-09-14T05:30:11.256362+0000","last_peered":"2024-09-14T05:30:11.256362+0000","last_clean":"2024-09-14T05:30:11.256362+0000","last_became_active":"2024-09-14T05:29:24.011481+0000","last_became_peered":"2024-09-14T05:29:24.011481+0000","last_unstale":"2024-09-14T05:30:11.256362+0000","last_undegraded":"2024-09-14T05:30:11.256362+0000","last_fullsized":"2024-09-14T05:30:11.256362+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:40:43.877931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011470110000000001,"stat_sum":{"num_bytes":44366438,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":28,"num_read":4439,"num_read_kb":4313,"num_write":806,"num_write_kb":2205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":108,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.257636+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:30:11.257636+0000","last_peered":"2024-09-14T05:30:11.257636+0000","last_clean":"2024-09-14T05:30:11.257636+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:30:11.257636+0000","last_undegraded":"2024-09-14T05:30:11.257636+0000","last_fullsized":"2024-09-14T05:30:11.257636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"66'402","reported_seq":1568,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.415217+0000","last_change":"2024-09-14T05:29:40.095950+0000","last_active":"2024-09-14T05:30:11.415217+0000","last_peered":"2024-09-14T05:30:11.415217+0000","last_clean":"2024-09-14T05:30:11.415217+0000","last_became_active":"2024-09-14T05:29:40.095441+0000","last_became_peered":"2024-09-14T05:29:40.095441+0000","last_unstale":"2024-09-14T05:30:11.415217+0000","last_undegraded":"2024-09-14T05:30:11.415217+0000","last_fullsized":"2024-09-14T05:30:11.415217+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:21:10.196955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053582499999999997,"stat_sum":{"num_bytes":20088036,"num_objects":61,"num_object_clones":15,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":36,"num_read":2300,"num_read_kb":9494,"num_write":487,"num_write_kb":8836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.289493+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:30:11.289493+0000","last_peered":"2024-09-14T05:30:11.289493+0000","last_clean":"2024-09-14T05:30:11.289493+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:30:11.289493+0000","last_undegraded":"2024-09-14T05:30:11.289493+0000","last_fullsized":"2024-09-14T05:30:11.289493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"64'224","reported_seq":1481,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.257029+0000","last_change":"2024-09-14T05:29:26.028547+0000","last_active":"2024-09-14T05:30:11.257029+0000","last_peered":"2024-09-14T05:30:11.257029+0000","last_clean":"2024-09-14T05:30:11.257029+0000","last_became_active":"2024-09-14T05:29:24.016940+0000","last_became_peered":"2024-09-14T05:29:24.016940+0000","last_unstale":"2024-09-14T05:30:11.257029+0000","last_undegraded":"2024-09-14T05:30:11.257029+0000","last_fullsized":"2024-09-14T05:30:11.257029+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:19:55.576781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042396399999999998,"stat_sum":{"num_bytes":11629336,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":27,"num_read":1511,"num_read_kb":4295,"num_write":235,"num_write_kb":2921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"67'295","reported_seq":1569,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T05:30:12.539706+0000","last_change":"2024-09-14T05:29:26.019829+0000","last_active":"2024-09-14T05:30:12.539706+0000","last_peered":"2024-09-14T05:30:12.539706+0000","last_clean":"2024-09-14T05:30:12.539706+0000","last_became_active":"2024-09-14T05:29:24.015905+0000","last_became_peered":"2024-09-14T05:29:24.015905+0000","last_unstale":"2024-09-14T05:30:12.539706+0000","last_undegraded":"2024-09-14T05:30:12.539706+0000","last_fullsized":"2024-09-14T05:30:12.539706+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:23:40.712802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049652099999999998,"stat_sum":{"num_bytes":14384859,"num_objects":73,"num_object_clones":21,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":42,"num_read":1602,"num_read_kb":5831,"num_write":345,"num_write_kb":4754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":110,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.258000+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:30:11.258000+0000","last_peered":"2024-09-14T05:30:11.258000+0000","last_clean":"2024-09-14T05:30:11.258000+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:30:11.258000+0000","last_undegraded":"2024-09-14T05:30:11.258000+0000","last_fullsized":"2024-09-14T05:30:11.258000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"66'308","reported_seq":10227,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.341709+0000","last_change":"2024-09-14T05:29:26.019261+0000","last_active":"2024-09-14T05:30:11.341709+0000","last_peered":"2024-09-14T05:30:11.341709+0000","last_clean":"2024-09-14T05:30:11.341709+0000","last_became_active":"2024-09-14T05:29:24.011764+0000","last_became_peered":"2024-09-14T05:29:24.011764+0000","last_unstale":"2024-09-14T05:30:11.341709+0000","last_undegraded":"2024-09-14T05:30:11.341709+0000","last_fullsized":"2024-09-14T05:30:11.341709+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:03:40.768055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055502299999999997,"stat_sum":{"num_bytes":42046776,"num_objects":60,"num_object_clones":14,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":31,"num_read":11201,"num_read_kb":37709,"num_write":280,"num_write_kb":4114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":108,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.303213+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:30:11.303213+0000","last_peered":"2024-09-14T05:30:11.303213+0000","last_clean":"2024-09-14T05:30:11.303213+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:30:11.303213+0000","last_undegraded":"2024-09-14T05:30:11.303213+0000","last_fullsized":"2024-09-14T05:30:11.303213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"66'565","reported_seq":1462,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256988+0000","last_change":"2024-09-14T05:29:40.095721+0000","last_active":"2024-09-14T05:30:11.256988+0000","last_peered":"2024-09-14T05:30:11.256988+0000","last_clean":"2024-09-14T05:30:11.256988+0000","last_became_active":"2024-09-14T05:29:40.095255+0000","last_became_peered":"2024-09-14T05:29:40.095255+0000","last_unstale":"2024-09-14T05:30:11.256988+0000","last_undegraded":"2024-09-14T05:30:11.256988+0000","last_fullsized":"2024-09-14T05:30:11.256988+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:06:37.866414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077857800000000004,"stat_sum":{"num_bytes":49326784,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":25,"num_read":2877,"num_read_kb":5289,"num_write":813,"num_write_kb":8865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":108,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256588+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:30:11.256588+0000","last_peered":"2024-09-14T05:30:11.256588+0000","last_clean":"2024-09-14T05:30:11.256588+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:30:11.256588+0000","last_undegraded":"2024-09-14T05:30:11.256588+0000","last_fullsized":"2024-09-14T05:30:11.256588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"66'433","reported_seq":2119,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.417016+0000","last_change":"2024-09-14T05:29:26.015124+0000","last_active":"2024-09-14T05:30:11.417016+0000","last_peered":"2024-09-14T05:30:11.417016+0000","last_clean":"2024-09-14T05:30:11.417016+0000","last_became_active":"2024-09-14T05:29:24.015792+0000","last_became_peered":"2024-09-14T05:29:24.015792+0000","last_unstale":"2024-09-14T05:30:11.417016+0000","last_undegraded":"2024-09-14T05:30:11.417016+0000","last_fullsized":"2024-09-14T05:30:11.417016+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:56:13.066139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001049964,"stat_sum":{"num_bytes":16760193,"num_objects":67,"num_object_clones":16,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":40,"num_read":3198,"num_read_kb":7568,"num_write":508,"num_write_kb":3535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":109,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256992+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:30:11.256992+0000","last_peered":"2024-09-14T05:30:11.256992+0000","last_clean":"2024-09-14T05:30:11.256992+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:30:11.256992+0000","last_undegraded":"2024-09-14T05:30:11.256992+0000","last_fullsized":"2024-09-14T05:30:11.256992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":199,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.261017+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:30:11.261017+0000","last_peered":"2024-09-14T05:30:11.261017+0000","last_clean":"2024-09-14T05:30:11.261017+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:30:11.261017+0000","last_undegraded":"2024-09-14T05:30:11.261017+0000","last_fullsized":"2024-09-14T05:30:11.261017+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"66'189","reported_seq":1060,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256394+0000","last_change":"2024-09-14T05:29:26.014430+0000","last_active":"2024-09-14T05:30:11.256394+0000","last_peered":"2024-09-14T05:30:11.256394+0000","last_clean":"2024-09-14T05:30:11.256394+0000","last_became_active":"2024-09-14T05:29:24.012222+0000","last_became_peered":"2024-09-14T05:29:24.012222+0000","last_unstale":"2024-09-14T05:30:11.256394+0000","last_undegraded":"2024-09-14T05:30:11.256394+0000","last_fullsized":"2024-09-14T05:30:11.256394+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:22:30.510692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076580099999999998,"stat_sum":{"num_bytes":26728033,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":27,"num_read":834,"num_read_kb":4295,"num_write":161,"num_write_kb":3445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"66'270","reported_seq":1092,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.398480+0000","last_change":"2024-09-14T05:29:26.164542+0000","last_active":"2024-09-14T05:30:11.398480+0000","last_peered":"2024-09-14T05:30:11.398480+0000","last_clean":"2024-09-14T05:30:11.398480+0000","last_became_active":"2024-09-14T05:29:24.012041+0000","last_became_peered":"2024-09-14T05:29:24.012041+0000","last_unstale":"2024-09-14T05:30:11.398480+0000","last_undegraded":"2024-09-14T05:30:11.398480+0000","last_fullsized":"2024-09-14T05:30:11.398480+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:40:03.452031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078334300000000004,"stat_sum":{"num_bytes":12716752,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":25,"num_read":1105,"num_read_kb":1839,"num_write":286,"num_write_kb":2491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"66'444","reported_seq":1845,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.383567+0000","last_change":"2024-09-14T05:29:26.165164+0000","last_active":"2024-09-14T05:30:11.383567+0000","last_peered":"2024-09-14T05:30:11.383567+0000","last_clean":"2024-09-14T05:30:11.383567+0000","last_became_active":"2024-09-14T05:29:24.011403+0000","last_became_peered":"2024-09-14T05:29:24.011403+0000","last_unstale":"2024-09-14T05:30:11.383567+0000","last_undegraded":"2024-09-14T05:30:11.383567+0000","last_fullsized":"2024-09-14T05:30:11.383567+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:55.865120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083114500000000004,"stat_sum":{"num_bytes":18545837,"num_objects":64,"num_object_clones":18,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":30,"num_read":2984,"num_read_kb":7171,"num_write":548,"num_write_kb":4514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"66'436","reported_seq":9409,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.398029+0000","last_change":"2024-09-14T05:29:26.163783+0000","last_active":"2024-09-14T05:30:11.398029+0000","last_peered":"2024-09-14T05:30:11.398029+0000","last_clean":"2024-09-14T05:30:11.398029+0000","last_became_active":"2024-09-14T05:29:24.006836+0000","last_became_peered":"2024-09-14T05:29:24.006836+0000","last_unstale":"2024-09-14T05:30:11.398029+0000","last_undegraded":"2024-09-14T05:30:11.398029+0000","last_fullsized":"2024-09-14T05:30:11.398029+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:04:42.666646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031188400000000001,"stat_sum":{"num_bytes":37332916,"num_objects":64,"num_object_clones":10,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":35,"num_read":11206,"num_read_kb":29064,"num_write":535,"num_write_kb":12685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"66'198","reported_seq":2421,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T05:30:12.320668+0000","last_change":"2024-09-14T05:29:26.020113+0000","last_active":"2024-09-14T05:30:12.320668+0000","last_peered":"2024-09-14T05:30:12.320668+0000","last_clean":"2024-09-14T05:30:12.320668+0000","last_became_active":"2024-09-14T05:29:24.012574+0000","last_became_peered":"2024-09-14T05:29:24.012574+0000","last_unstale":"2024-09-14T05:30:12.320668+0000","last_undegraded":"2024-09-14T05:30:12.320668+0000","last_fullsized":"2024-09-14T05:30:12.320668+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:15:13.596278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072680599999999998,"stat_sum":{"num_bytes":39953671,"num_objects":60,"num_object_clones":20,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":32,"num_read":2199,"num_read_kb":8085,"num_write":166,"num_write_kb":5126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"66'774","reported_seq":9609,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.428007+0000","last_change":"2024-09-14T05:29:26.164736+0000","last_active":"2024-09-14T05:30:11.428007+0000","last_peered":"2024-09-14T05:30:11.428007+0000","last_clean":"2024-09-14T05:30:11.428007+0000","last_became_active":"2024-09-14T05:29:24.012401+0000","last_became_peered":"2024-09-14T05:29:24.012401+0000","last_unstale":"2024-09-14T05:30:11.428007+0000","last_undegraded":"2024-09-14T05:30:11.428007+0000","last_fullsized":"2024-09-14T05:30:11.428007+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:25:12.169350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085260200000000005,"stat_sum":{"num_bytes":43455562,"num_objects":72,"num_object_clones":18,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":40,"num_read":11810,"num_read_kb":19585,"num_write":1063,"num_write_kb":4975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"66'194","reported_seq":1587,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.447905+0000","last_change":"2024-09-14T05:29:26.028811+0000","last_active":"2024-09-14T05:30:11.447905+0000","last_peered":"2024-09-14T05:30:11.447905+0000","last_clean":"2024-09-14T05:30:11.447905+0000","last_became_active":"2024-09-14T05:29:24.015315+0000","last_became_peered":"2024-09-14T05:29:24.015315+0000","last_unstale":"2024-09-14T05:30:11.447905+0000","last_undegraded":"2024-09-14T05:30:11.447905+0000","last_fullsized":"2024-09-14T05:30:11.447905+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:55:30.171267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046485199999999997,"stat_sum":{"num_bytes":25287542,"num_objects":65,"num_object_clones":20,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":39,"num_read":1331,"num_read_kb":2486,"num_write":169,"num_write_kb":3014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"67'309","reported_seq":10735,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T05:30:12.481411+0000","last_change":"2024-09-14T05:29:26.020027+0000","last_active":"2024-09-14T05:30:12.481411+0000","last_peered":"2024-09-14T05:30:12.481411+0000","last_clean":"2024-09-14T05:30:12.481411+0000","last_became_active":"2024-09-14T05:29:24.018854+0000","last_became_peered":"2024-09-14T05:29:24.018854+0000","last_unstale":"2024-09-14T05:30:12.481411+0000","last_undegraded":"2024-09-14T05:30:12.481411+0000","last_fullsized":"2024-09-14T05:30:12.481411+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:56.376950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063672000000000002,"stat_sum":{"num_bytes":34972261,"num_objects":83,"num_object_clones":22,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":45,"num_read":11220,"num_read_kb":29115,"num_write":284,"num_write_kb":10826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"65'229","reported_seq":1234,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.402693+0000","last_change":"2024-09-14T05:29:26.014200+0000","last_active":"2024-09-14T05:30:11.402693+0000","last_peered":"2024-09-14T05:30:11.402693+0000","last_clean":"2024-09-14T05:30:11.402693+0000","last_became_active":"2024-09-14T05:29:24.010897+0000","last_became_peered":"2024-09-14T05:29:24.010897+0000","last_unstale":"2024-09-14T05:30:11.402693+0000","last_undegraded":"2024-09-14T05:30:11.402693+0000","last_fullsized":"2024-09-14T05:30:11.402693+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:37:30.543147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051531799999999996,"stat_sum":{"num_bytes":33936373,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":26,"num_read":1014,"num_read_kb":8865,"num_write":202,"num_write_kb":6050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"66'344","reported_seq":2324,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.431151+0000","last_change":"2024-09-14T05:29:26.164788+0000","last_active":"2024-09-14T05:30:11.431151+0000","last_peered":"2024-09-14T05:30:11.431151+0000","last_clean":"2024-09-14T05:30:11.431151+0000","last_became_active":"2024-09-14T05:29:24.013792+0000","last_became_peered":"2024-09-14T05:29:24.013792+0000","last_unstale":"2024-09-14T05:30:11.431151+0000","last_undegraded":"2024-09-14T05:30:11.431151+0000","last_fullsized":"2024-09-14T05:30:11.431151+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:08:01.065225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090671299999999996,"stat_sum":{"num_bytes":29225047,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":3686,"num_read_kb":15807,"num_write":343,"num_write_kb":2409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"65'211","reported_seq":1246,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.417214+0000","last_change":"2024-09-14T05:29:40.095745+0000","last_active":"2024-09-14T05:30:11.417214+0000","last_peered":"2024-09-14T05:30:11.417214+0000","last_clean":"2024-09-14T05:30:11.417214+0000","last_became_active":"2024-09-14T05:29:40.095331+0000","last_became_peered":"2024-09-14T05:29:40.095331+0000","last_unstale":"2024-09-14T05:30:11.417214+0000","last_undegraded":"2024-09-14T05:30:11.417214+0000","last_fullsized":"2024-09-14T05:30:11.417214+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:29:19.959417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000774981,"stat_sum":{"num_bytes":27414058,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":1534,"num_read_kb":8788,"num_write":192,"num_write_kb":4039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"66'446","reported_seq":1759,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256917+0000","last_change":"2024-09-14T05:29:26.164295+0000","last_active":"2024-09-14T05:30:11.256917+0000","last_peered":"2024-09-14T05:30:11.256917+0000","last_clean":"2024-09-14T05:30:11.256917+0000","last_became_active":"2024-09-14T05:29:24.014371+0000","last_became_peered":"2024-09-14T05:29:24.014371+0000","last_unstale":"2024-09-14T05:30:11.256917+0000","last_undegraded":"2024-09-14T05:30:11.256917+0000","last_fullsized":"2024-09-14T05:30:11.256917+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:32:06.372111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050570599999999997,"stat_sum":{"num_bytes":56505386,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":26,"num_read":2769,"num_read_kb":18316,"num_write":653,"num_write_kb":4935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1073439577,"num_objects":1900,"num_object_clones":487,"num_object_copies":3800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1900,"num_whiteouts":1007,"num_read":142159,"num_read_kb":448872,"num_write":12692,"num_write_kb":221667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":269176832,"data_stored":278252433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11034,"ondisk_log_size":11034,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476753,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88248,"kb_used_data":48740,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94283592,"statfs":{"total":96636764160,"available":96546398208,"internally_reserved":0,"allocated":49909760,"data_stored":50231088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53520,"kb_used_data":22328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94318320,"statfs":{"total":96636764160,"available":96581959680,"internally_reserved":0,"allocated":22863872,"data_stored":22825640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66656,"kb_used_data":27140,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94305184,"statfs":{"total":96636764160,"available":96568508416,"internally_reserved":0,"allocated":27791360,"data_stored":29005176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83860,"kb_used_data":44352,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94287980,"statfs":{"total":96636764160,"available":96550891520,"internally_reserved":0,"allocated":45416448,"data_stored":46429833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67008,"kb_used_data":31664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94304832,"statfs":{"total":96636764160,"available":96568147968,"internally_reserved":0,"allocated":32423936,"data_stored":32787746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63384,"kb_used_data":23844,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94308456,"statfs":{"total":96636764160,"available":96571858944,"internally_reserved":0,"allocated":24416256,"data_stored":25416164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64212,"kb_used_data":24708,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94307628,"statfs":{"total":96636764160,"available":96571011072,"internally_reserved":0,"allocated":25300992,"data_stored":26172013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85428,"kb_used_data":45888,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94286412,"statfs":{"total":96636764160,"available":96549285888,"internally_reserved":0,"allocated":46989312,"data_stored":48073925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49963008,"data_stored":50678949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45649920,"data_stored":47147496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24694784,"data_stored":25965649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23789568,"data_stored":25191018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":31768576,"data_stored":32529631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":44085248,"data_stored":45509553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":27037696,"data_stored":28669034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":22188032,"data_stored":22561103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:30:15.018 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T05:30:15.019 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T05:30:15.019 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:30:15.035 INFO:teuthology.orchestra.run.smithi071.stdout:4478 trunc from 0xe49ab30 to 0x740f64 2024-09-14T05:30:15.035 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:15.034+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:15.060 INFO:teuthology.orchestra.run.smithi071.stdout:4481 write 0x8aac9db thru 0x8aae49e (0x1ac4 bytes) 2024-09-14T05:30:15.064 INFO:teuthology.orchestra.run.smithi071.stdout:4482 write 0x63f465 thru 0x647017 (0x7bb3 bytes) 2024-09-14T05:30:15.068 INFO:teuthology.orchestra.run.smithi071.stdout:4483 punch from 0x77ee12 to 0x788163, (0x9351 bytes) 2024-09-14T05:30:15.068 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.073 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:15.073 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:15.074 INFO:teuthology.orchestra.run.smithi071.stdout:4485 writesame 0x9d4974b thru 0x9d4dc34 data_size 0x1(0x44ea bytes) 2024-09-14T05:30:15.074 INFO:teuthology.orchestra.run.smithi071.stdout:4486 read 0x6203fdb thru 0x62113f4 (0xd41a bytes) 2024-09-14T05:30:15.075 INFO:teuthology.orchestra.run.smithi071.stdout:4488 read 0x36c3de0 thru 0x36c970f (0x5930 bytes) 2024-09-14T05:30:15.075 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.075 INFO:teuthology.orchestra.run.smithi071.stdout:4490 read 0x3445def thru 0x3453043 (0xd255 bytes) 2024-09-14T05:30:15.075 INFO:teuthology.orchestra.run.smithi071.stdout:4492 read 0x43d158 thru 0x4477fd (0xa6a6 bytes) 2024-09-14T05:30:15.081 INFO:teuthology.orchestra.run.smithi071.stdout:4493 write 0xb3b3b38 thru 0xb3b7fdf (0x44a8 bytes) 2024-09-14T05:30:15.082 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:30:15.094 INFO:teuthology.orchestra.run.smithi071.stdout:4494 trunc from 0xb3b7fe0 to 0x3b29ab2 2024-09-14T05:30:15.116 INFO:teuthology.orchestra.run.smithi071.stdout:4495 write 0x9a22adb thru 0x9a2a55c (0x7a82 bytes) 2024-09-14T05:30:15.120 INFO:teuthology.orchestra.run.smithi071.stdout:4496 write 0x6e4f44c thru 0x6e5df8e (0xeb43 bytes) 2024-09-14T05:30:15.124 INFO:teuthology.orchestra.run.smithi071.stdout:4499 read 0x48f3c30 thru 0x4900b95 (0xcf66 bytes) 2024-09-14T05:30:15.125 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:15.125+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:15.129 INFO:teuthology.orchestra.run.smithi071.stdout:4500 trunc from 0x9a2a55d to 0xde71091 2024-09-14T05:30:15.134 INFO:teuthology.orchestra.run.smithi071.stdout:4501 read 0x3dcac83 thru 0x3dd41bd (0x953b bytes) 2024-09-14T05:30:15.140 INFO:teuthology.orchestra.run.smithi071.stdout:4505 write 0xe2a878f thru 0xe2b6564 (0xddd6 bytes) 2024-09-14T05:30:15.144 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.144 INFO:teuthology.orchestra.run.smithi071.stdout:4508 punch from 0xd4b767b to 0xd4c3f18, (0xc89d bytes) 2024-09-14T05:30:15.144 INFO:teuthology.orchestra.run.smithi071.stdout:4510 write 0x9d31882 thru 0x9d3b779 (0x9ef8 bytes) 2024-09-14T05:30:15.149 INFO:teuthology.orchestra.run.smithi071.stdout:4513 punch from 0x4b4baa4 to 0x4b5967a, (0xdbd6 bytes) 2024-09-14T05:30:15.149 INFO:teuthology.orchestra.run.smithi071.stdout:4514 punch from 0x6e4a702 to 0x6e59662, (0xef60 bytes) 2024-09-14T05:30:15.149 INFO:teuthology.orchestra.run.smithi071.stdout:4515 writesame 0x1ed1214 thru 0x1ed3d7a data_size 0x1(0x2b67 bytes) 2024-09-14T05:30:15.156 INFO:teuthology.orchestra.run.smithi071.stdout:4517 write 0x9f8357c thru 0x9f8bd97 (0x881c bytes) 2024-09-14T05:30:15.162 INFO:teuthology.orchestra.run.smithi071.stdout:4518 read 0x707389c thru 0x707db8d (0xa2f2 bytes) 2024-09-14T05:30:15.162 INFO:teuthology.orchestra.run.smithi071.stdout:4519 read 0x941cad8 thru 0x94217c1 (0x4cea bytes) 2024-09-14T05:30:15.162 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.162 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.162 INFO:teuthology.orchestra.run.smithi071.stdout:4526 read 0x3ffef76 thru 0x4006978 (0x7a03 bytes) 2024-09-14T05:30:15.163 INFO:teuthology.orchestra.run.smithi071.stdout:4527 read 0x3f416bf thru 0x3f4c93e (0xb280 bytes) 2024-09-14T05:30:15.163 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.163 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.163 INFO:teuthology.orchestra.run.smithi071.stdout:4530 write 0x8001255 thru 0x800d4e5 (0xc291 bytes) 2024-09-14T05:30:15.167 INFO:teuthology.orchestra.run.smithi071.stdout:4531 write 0x32d2690 thru 0x32dd0f0 (0xaa61 bytes) 2024-09-14T05:30:15.173 INFO:teuthology.orchestra.run.smithi071.stdout:4532 read 0xd4760f0 thru 0xd478387 (0x2298 bytes) 2024-09-14T05:30:15.173 INFO:teuthology.orchestra.run.smithi071.stdout:4533 read 0xcd18b21 thru 0xcd243ab (0xb88b bytes) 2024-09-14T05:30:15.173 INFO:teuthology.orchestra.run.smithi071.stdout:4536 read 0x17e336f thru 0x17e86f5 (0x5387 bytes) 2024-09-14T05:30:15.173 INFO:teuthology.orchestra.run.smithi071.stdout:4537 write 0xc45ac7d thru 0xc467ad4 (0xce58 bytes) 2024-09-14T05:30:15.177 INFO:teuthology.orchestra.run.smithi071.stdout:4538 write 0x2dc68e6 thru 0x2dd073e (0x9e59 bytes) 2024-09-14T05:30:15.184 INFO:teuthology.orchestra.run.smithi071.stdout:4539 writesame 0xcff0cb6 thru 0xcff5203 data_size 0x2(0x454e bytes) 2024-09-14T05:30:15.190 INFO:teuthology.orchestra.run.smithi071.stdout:4540 writesame 0x6d2b8a1 thru 0x6d3b7a3 data_size 0x2f(0xff03 bytes) 2024-09-14T05:30:15.193 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:15.193 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:15.200 INFO:teuthology.orchestra.run.smithi071.stdout:4541 write 0xe79a5c7 thru 0xe7a60fe (0xbb38 bytes) 2024-09-14T05:30:15.204 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:30:15.204 INFO:teuthology.orchestra.run.smithi071.stdout:4542 trunc from 0xe7a60ff to 0xe956c1b 2024-09-14T05:30:15.209 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.209 INFO:teuthology.orchestra.run.smithi071.stdout:4544 write 0x425c23c thru 0x425d600 (0x13c5 bytes) 2024-09-14T05:30:15.212 INFO:teuthology.orchestra.run.smithi071.stdout:4545 write 0x36198fe thru 0x362480c (0xaf0f bytes) 2024-09-14T05:30:15.228 INFO:teuthology.orchestra.run.smithi071.stdout:4546 trunc from 0xe956c1b to 0x6ec7c1d 2024-09-14T05:30:15.243 INFO:teuthology.orchestra.run.smithi071.stdout:4547 write 0x235798e thru 0x2359ad0 (0x2143 bytes) 2024-09-14T05:30:15.244 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:15.244+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:15.253 INFO:teuthology.orchestra.run.smithi071.stdout:4549 trunc from 0x6ec7c1d to 0xbb854d6 2024-09-14T05:30:15.258 INFO:teuthology.orchestra.run.smithi071.stdout:4550 write 0x77b4b8f thru 0x77bbd01 (0x7173 bytes) 2024-09-14T05:30:15.262 INFO:teuthology.orchestra.run.smithi071.stdout:4553 write 0xb0befb6 thru 0xb0c8150 (0x919b bytes) 2024-09-14T05:30:15.266 INFO:teuthology.orchestra.run.smithi071.stdout:4554 punch from 0x1e2bb82 to 0x1e3a60e, (0xea8c bytes) 2024-09-14T05:30:15.274 INFO:teuthology.orchestra.run.smithi071.stdout:4555 writesame 0xdc6623a thru 0xdc69f24 data_size 0x1(0x3ceb bytes) 2024-09-14T05:30:15.274 INFO:teuthology.orchestra.run.smithi071.stdout:4556 write 0xbb4032c thru 0xbb4b059 (0xad2e bytes) 2024-09-14T05:30:15.279 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.279 INFO:teuthology.orchestra.run.smithi071.stdout:4558 write 0x6fdbe7a thru 0x6fdf4ae (0x3635 bytes) 2024-09-14T05:30:15.283 INFO:teuthology.orchestra.run.smithi071.stdout:4559 punch from 0x28aaa78 to 0x28abe40, (0x13c8 bytes) 2024-09-14T05:30:15.283 INFO:teuthology.orchestra.run.smithi071.stdout:4560 read 0x3c5a565 thru 0x3c6819c (0xdc38 bytes) 2024-09-14T05:30:15.283 INFO:teuthology.orchestra.run.smithi071.stdout:4562 read 0x4ad2532 thru 0x4ad8464 (0x5f33 bytes) 2024-09-14T05:30:15.283 INFO:teuthology.orchestra.run.smithi071.stdout:4563 read 0x9e29d06 thru 0x9e366da (0xc9d5 bytes) 2024-09-14T05:30:15.284 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.284 INFO:teuthology.orchestra.run.smithi071.stdout:4565 punch from 0xa6b8e8e to 0xa6c2bb4, (0x9d26 bytes) 2024-09-14T05:30:15.284 INFO:teuthology.orchestra.run.smithi071.stdout:4566 read 0x4a7aa7 thru 0x4b77f4 (0xfd4e bytes) 2024-09-14T05:30:15.284 INFO:teuthology.orchestra.run.smithi071.stdout:4567 write 0x62c7e00 thru 0x62d5116 (0xd317 bytes) 2024-09-14T05:30:15.287 INFO:teuthology.orchestra.run.smithi071.stdout:4569 write 0x5f57a72 thru 0x5f5ec55 (0x71e4 bytes) 2024-09-14T05:30:15.292 INFO:teuthology.orchestra.run.smithi071.stdout:4571 write 0x15dca54 thru 0x15eac2b (0xe1d8 bytes) 2024-09-14T05:30:15.297 INFO:teuthology.orchestra.run.smithi071.stdout:4572 writesame 0x17d2374 thru 0x17df715 data_size 0x2(0xd3a2 bytes) 2024-09-14T05:30:15.316 INFO:teuthology.orchestra.run.smithi071.stdout:4573 trunc from 0xdc69f25 to 0x738a7be 2024-09-14T05:30:15.322 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:15.322+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:15.328 INFO:teuthology.orchestra.run.smithi071.stdout:4574 read 0x13ddeeb thru 0x13e2f65 (0x507b bytes) 2024-09-14T05:30:15.328 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.329 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:15.329 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:15.336 INFO:teuthology.orchestra.run.smithi071.stdout:4576 write 0x95cda28 thru 0x95d2adb (0x50b4 bytes) 2024-09-14T05:30:15.339 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:30:15.343 INFO:teuthology.orchestra.run.smithi071.stdout:4579 write 0x738de51 thru 0x73909dd (0x2b8d bytes) 2024-09-14T05:30:15.354 INFO:teuthology.orchestra.run.smithi071.stdout:4580 trunc from 0x95d2adc to 0xcb1869e 2024-09-14T05:30:15.359 INFO:teuthology.orchestra.run.smithi071.stdout:4582 read 0x4986179 thru 0x498a8c5 (0x474d bytes) 2024-09-14T05:30:15.360 INFO:teuthology.orchestra.run.smithi071.stdout:4583 writesame 0x9661098 thru 0x966aa71 data_size 0x2f(0x99da bytes) 2024-09-14T05:30:15.365 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.366 INFO:teuthology.orchestra.run.smithi071.stdout:4587 writesame 0x74d408f thru 0x74d845e data_size 0x50(0x43d0 bytes) 2024-09-14T05:30:15.370 INFO:teuthology.orchestra.run.smithi071.stdout:4588 read 0x24eefd5 thru 0x24f5443 (0x646f bytes) 2024-09-14T05:30:15.382 INFO:teuthology.orchestra.run.smithi071.stdout:4589 trunc from 0xcb1869e to 0x2a7b77a 2024-09-14T05:30:15.383 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:30:15.384 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:30:15.400 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-09-14T05:30:14.446791+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1074029964,"num_objects":1904,"num_object_clones":487,"num_object_copies":3808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1904,"num_whiteouts":1007,"num_read":142225,"num_read_kb":448927,"num_write":12795,"num_write_kb":223297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11109,"ondisk_log_size":11109,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":572316,"kb_used_data":268664,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754402404,"statfs":{"total":773094113280,"available":772508061696,"internally_reserved":0,"allocated":275111936,"data_stored":280941585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43269347,"num_objects":120,"num_object_clones":35,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":42,"num_read":7870,"num_read_kb":22146,"num_write":2069,"num_write_kb":12337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000710"},"pg_stats":[{"pgid":"3.1f","version":"65'448","reported_seq":9587,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.289555+0000","last_change":"2024-09-14T05:29:26.019843+0000","last_active":"2024-09-14T05:30:11.289555+0000","last_peered":"2024-09-14T05:30:11.289555+0000","last_clean":"2024-09-14T05:30:11.289555+0000","last_became_active":"2024-09-14T05:29:24.018892+0000","last_became_peered":"2024-09-14T05:29:24.018892+0000","last_unstale":"2024-09-14T05:30:11.289555+0000","last_undegraded":"2024-09-14T05:30:11.289555+0000","last_fullsized":"2024-09-14T05:30:11.289555+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:18:05.949158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011130109999999999,"stat_sum":{"num_bytes":42568241,"num_objects":62,"num_object_clones":15,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":32,"num_read":10616,"num_read_kb":33599,"num_write":552,"num_write_kb":28618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"66'233","reported_seq":1704,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.345545+0000","last_change":"2024-09-14T05:29:26.019068+0000","last_active":"2024-09-14T05:30:11.345545+0000","last_peered":"2024-09-14T05:30:11.345545+0000","last_clean":"2024-09-14T05:30:11.345545+0000","last_became_active":"2024-09-14T05:29:24.014468+0000","last_became_peered":"2024-09-14T05:29:24.014468+0000","last_unstale":"2024-09-14T05:30:11.345545+0000","last_undegraded":"2024-09-14T05:30:11.345545+0000","last_fullsized":"2024-09-14T05:30:11.345545+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:31:09.297175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000399031,"stat_sum":{"num_bytes":53702062,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":27,"num_read":1627,"num_read_kb":22111,"num_write":246,"num_write_kb":19416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"65'189","reported_seq":8573,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.257472+0000","last_change":"2024-09-14T05:29:26.165031+0000","last_active":"2024-09-14T05:30:11.257472+0000","last_peered":"2024-09-14T05:30:11.257472+0000","last_clean":"2024-09-14T05:30:11.257472+0000","last_became_active":"2024-09-14T05:29:24.014251+0000","last_became_peered":"2024-09-14T05:29:24.014251+0000","last_unstale":"2024-09-14T05:30:11.257472+0000","last_undegraded":"2024-09-14T05:30:11.257472+0000","last_fullsized":"2024-09-14T05:30:11.257472+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:12:24.549903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010261580000000001,"stat_sum":{"num_bytes":53832863,"num_objects":60,"num_object_clones":20,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":31,"num_read":8335,"num_read_kb":19767,"num_write":174,"num_write_kb":4655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"66'339","reported_seq":9343,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.393833+0000","last_change":"2024-09-14T05:29:26.014243+0000","last_active":"2024-09-14T05:30:11.393833+0000","last_peered":"2024-09-14T05:30:11.393833+0000","last_clean":"2024-09-14T05:30:11.393833+0000","last_became_active":"2024-09-14T05:29:24.012427+0000","last_became_peered":"2024-09-14T05:29:24.012427+0000","last_unstale":"2024-09-14T05:30:11.393833+0000","last_undegraded":"2024-09-14T05:30:11.393833+0000","last_fullsized":"2024-09-14T05:30:11.393833+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:24:31.534521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045797999999999998,"stat_sum":{"num_bytes":62236263,"num_objects":73,"num_object_clones":20,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":38,"num_read":9953,"num_read_kb":33335,"num_write":330,"num_write_kb":4156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"65'366","reported_seq":2772,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256276+0000","last_change":"2024-09-14T05:29:26.014462+0000","last_active":"2024-09-14T05:30:11.256276+0000","last_peered":"2024-09-14T05:30:11.256276+0000","last_clean":"2024-09-14T05:30:11.256276+0000","last_became_active":"2024-09-14T05:29:24.012079+0000","last_became_peered":"2024-09-14T05:29:24.012079+0000","last_unstale":"2024-09-14T05:30:11.256276+0000","last_undegraded":"2024-09-14T05:30:11.256276+0000","last_fullsized":"2024-09-14T05:30:11.256276+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:27:44.817359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070589999999999997,"stat_sum":{"num_bytes":37003506,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":34,"num_read":3716,"num_read_kb":24505,"num_write":409,"num_write_kb":21331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"66'667","reported_seq":3882,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.453187+0000","last_change":"2024-09-14T05:29:26.014453+0000","last_active":"2024-09-14T05:30:11.453187+0000","last_peered":"2024-09-14T05:30:11.453187+0000","last_clean":"2024-09-14T05:30:11.453187+0000","last_became_active":"2024-09-14T05:29:24.012461+0000","last_became_peered":"2024-09-14T05:29:24.012461+0000","last_unstale":"2024-09-14T05:30:11.453187+0000","last_undegraded":"2024-09-14T05:30:11.453187+0000","last_fullsized":"2024-09-14T05:30:11.453187+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:30:20.541523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061290100000000001,"stat_sum":{"num_bytes":26977201,"num_objects":71,"num_object_clones":18,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":36,"num_read":6434,"num_read_kb":29871,"num_write":955,"num_write_kb":5976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"65'232","reported_seq":1544,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256274+0000","last_change":"2024-09-14T05:29:26.014685+0000","last_active":"2024-09-14T05:30:11.256274+0000","last_peered":"2024-09-14T05:30:11.256274+0000","last_clean":"2024-09-14T05:30:11.256274+0000","last_became_active":"2024-09-14T05:29:24.018753+0000","last_became_peered":"2024-09-14T05:29:24.018753+0000","last_unstale":"2024-09-14T05:30:11.256274+0000","last_undegraded":"2024-09-14T05:30:11.256274+0000","last_fullsized":"2024-09-14T05:30:11.256274+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:21:54.507503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089943399999999995,"stat_sum":{"num_bytes":40471085,"num_objects":70,"num_object_clones":19,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":31,"num_read":1370,"num_read_kb":12605,"num_write":218,"num_write_kb":8710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"64'430","reported_seq":2073,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.289593+0000","last_change":"2024-09-14T05:29:26.019928+0000","last_active":"2024-09-14T05:30:11.289593+0000","last_peered":"2024-09-14T05:30:11.289593+0000","last_clean":"2024-09-14T05:30:11.289593+0000","last_became_active":"2024-09-14T05:29:24.011176+0000","last_became_peered":"2024-09-14T05:29:24.011176+0000","last_unstale":"2024-09-14T05:30:11.289593+0000","last_undegraded":"2024-09-14T05:30:11.289593+0000","last_fullsized":"2024-09-14T05:30:11.289593+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:43:15.679468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011097850000000001,"stat_sum":{"num_bytes":12323793,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":25,"num_read":3993,"num_read_kb":6803,"num_write":480,"num_write_kb":2570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"64'235","reported_seq":1771,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.289583+0000","last_change":"2024-09-14T05:29:26.019132+0000","last_active":"2024-09-14T05:30:11.289583+0000","last_peered":"2024-09-14T05:30:11.289583+0000","last_clean":"2024-09-14T05:30:11.289583+0000","last_became_active":"2024-09-14T05:29:24.014005+0000","last_became_peered":"2024-09-14T05:29:24.014005+0000","last_unstale":"2024-09-14T05:30:11.289583+0000","last_undegraded":"2024-09-14T05:30:11.289583+0000","last_fullsized":"2024-09-14T05:30:11.289583+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:41.814918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045225300000000001,"stat_sum":{"num_bytes":7394881,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":22,"num_read":2050,"num_read_kb":4565,"num_write":226,"num_write_kb":3006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"63'232","reported_seq":9335,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.257399+0000","last_change":"2024-09-14T05:29:26.028571+0000","last_active":"2024-09-14T05:30:11.257399+0000","last_peered":"2024-09-14T05:30:11.257399+0000","last_clean":"2024-09-14T05:30:11.257399+0000","last_became_active":"2024-09-14T05:29:24.013434+0000","last_became_peered":"2024-09-14T05:29:24.013434+0000","last_unstale":"2024-09-14T05:30:11.257399+0000","last_undegraded":"2024-09-14T05:30:11.257399+0000","last_fullsized":"2024-09-14T05:30:11.257399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:35:46.978990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038301200000000001,"stat_sum":{"num_bytes":53492260,"num_objects":63,"num_object_clones":18,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":37,"num_read":9055,"num_read_kb":20637,"num_write":224,"num_write_kb":6734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"67'272","reported_seq":2152,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T05:30:12.557046+0000","last_change":"2024-09-14T05:29:26.020364+0000","last_active":"2024-09-14T05:30:12.557046+0000","last_peered":"2024-09-14T05:30:12.557046+0000","last_clean":"2024-09-14T05:30:12.557046+0000","last_became_active":"2024-09-14T05:29:24.014457+0000","last_became_peered":"2024-09-14T05:29:24.014457+0000","last_unstale":"2024-09-14T05:30:12.557046+0000","last_undegraded":"2024-09-14T05:30:12.557046+0000","last_fullsized":"2024-09-14T05:30:12.557046+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:00:48.968877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010888930000000001,"stat_sum":{"num_bytes":74500397,"num_objects":63,"num_object_clones":19,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":29,"num_read":2367,"num_read_kb":8189,"num_write":296,"num_write_kb":11575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"66'314","reported_seq":1497,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.343505+0000","last_change":"2024-09-14T05:29:26.164291+0000","last_active":"2024-09-14T05:30:11.343505+0000","last_peered":"2024-09-14T05:30:11.343505+0000","last_clean":"2024-09-14T05:30:11.343505+0000","last_became_active":"2024-09-14T05:29:24.013266+0000","last_became_peered":"2024-09-14T05:29:24.013266+0000","last_unstale":"2024-09-14T05:30:11.343505+0000","last_undegraded":"2024-09-14T05:30:11.343505+0000","last_fullsized":"2024-09-14T05:30:11.343505+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:26:41.280756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060129699999999999,"stat_sum":{"num_bytes":15017115,"num_objects":53,"num_object_clones":9,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":34,"num_read":2194,"num_read_kb":2850,"num_write":321,"num_write_kb":2896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":108,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.257324+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:30:11.257324+0000","last_peered":"2024-09-14T05:30:11.257324+0000","last_clean":"2024-09-14T05:30:11.257324+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:30:11.257324+0000","last_undegraded":"2024-09-14T05:30:11.257324+0000","last_fullsized":"2024-09-14T05:30:11.257324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"66'197","reported_seq":1466,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.374550+0000","last_change":"2024-09-14T05:29:26.014725+0000","last_active":"2024-09-14T05:30:11.374550+0000","last_peered":"2024-09-14T05:30:11.374550+0000","last_clean":"2024-09-14T05:30:11.374550+0000","last_became_active":"2024-09-14T05:29:24.012099+0000","last_became_peered":"2024-09-14T05:29:24.012099+0000","last_unstale":"2024-09-14T05:30:11.374550+0000","last_undegraded":"2024-09-14T05:30:11.374550+0000","last_fullsized":"2024-09-14T05:30:11.374550+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:53:28.191127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000794599,"stat_sum":{"num_bytes":9244050,"num_objects":48,"num_object_clones":14,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":27,"num_read":1629,"num_read_kb":2120,"num_write":185,"num_write_kb":2285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":106,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256453+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:30:11.256453+0000","last_peered":"2024-09-14T05:30:11.256453+0000","last_clean":"2024-09-14T05:30:11.256453+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:30:11.256453+0000","last_undegraded":"2024-09-14T05:30:11.256453+0000","last_fullsized":"2024-09-14T05:30:11.256453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"65'609","reported_seq":2225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256362+0000","last_change":"2024-09-14T05:29:26.015082+0000","last_active":"2024-09-14T05:30:11.256362+0000","last_peered":"2024-09-14T05:30:11.256362+0000","last_clean":"2024-09-14T05:30:11.256362+0000","last_became_active":"2024-09-14T05:29:24.011481+0000","last_became_peered":"2024-09-14T05:29:24.011481+0000","last_unstale":"2024-09-14T05:30:11.256362+0000","last_undegraded":"2024-09-14T05:30:11.256362+0000","last_fullsized":"2024-09-14T05:30:11.256362+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:40:43.877931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011470110000000001,"stat_sum":{"num_bytes":44366438,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":28,"num_read":4439,"num_read_kb":4313,"num_write":806,"num_write_kb":2205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":108,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.257636+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:30:11.257636+0000","last_peered":"2024-09-14T05:30:11.257636+0000","last_clean":"2024-09-14T05:30:11.257636+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:30:11.257636+0000","last_undegraded":"2024-09-14T05:30:11.257636+0000","last_fullsized":"2024-09-14T05:30:11.257636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"66'402","reported_seq":1568,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.415217+0000","last_change":"2024-09-14T05:29:40.095950+0000","last_active":"2024-09-14T05:30:11.415217+0000","last_peered":"2024-09-14T05:30:11.415217+0000","last_clean":"2024-09-14T05:30:11.415217+0000","last_became_active":"2024-09-14T05:29:40.095441+0000","last_became_peered":"2024-09-14T05:29:40.095441+0000","last_unstale":"2024-09-14T05:30:11.415217+0000","last_undegraded":"2024-09-14T05:30:11.415217+0000","last_fullsized":"2024-09-14T05:30:11.415217+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:21:10.196955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053582499999999997,"stat_sum":{"num_bytes":20088036,"num_objects":61,"num_object_clones":15,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":36,"num_read":2300,"num_read_kb":9494,"num_write":487,"num_write_kb":8836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.289493+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:30:11.289493+0000","last_peered":"2024-09-14T05:30:11.289493+0000","last_clean":"2024-09-14T05:30:11.289493+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:30:11.289493+0000","last_undegraded":"2024-09-14T05:30:11.289493+0000","last_fullsized":"2024-09-14T05:30:11.289493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"64'224","reported_seq":1481,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.257029+0000","last_change":"2024-09-14T05:29:26.028547+0000","last_active":"2024-09-14T05:30:11.257029+0000","last_peered":"2024-09-14T05:30:11.257029+0000","last_clean":"2024-09-14T05:30:11.257029+0000","last_became_active":"2024-09-14T05:29:24.016940+0000","last_became_peered":"2024-09-14T05:29:24.016940+0000","last_unstale":"2024-09-14T05:30:11.257029+0000","last_undegraded":"2024-09-14T05:30:11.257029+0000","last_fullsized":"2024-09-14T05:30:11.257029+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:19:55.576781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042396399999999998,"stat_sum":{"num_bytes":11629336,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":27,"num_read":1511,"num_read_kb":4295,"num_write":235,"num_write_kb":2921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"67'295","reported_seq":1569,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T05:30:12.539706+0000","last_change":"2024-09-14T05:29:26.019829+0000","last_active":"2024-09-14T05:30:12.539706+0000","last_peered":"2024-09-14T05:30:12.539706+0000","last_clean":"2024-09-14T05:30:12.539706+0000","last_became_active":"2024-09-14T05:29:24.015905+0000","last_became_peered":"2024-09-14T05:29:24.015905+0000","last_unstale":"2024-09-14T05:30:12.539706+0000","last_undegraded":"2024-09-14T05:30:12.539706+0000","last_fullsized":"2024-09-14T05:30:12.539706+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:23:40.712802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049652099999999998,"stat_sum":{"num_bytes":14384859,"num_objects":73,"num_object_clones":21,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":42,"num_read":1602,"num_read_kb":5831,"num_write":345,"num_write_kb":4754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":110,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.258000+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:30:11.258000+0000","last_peered":"2024-09-14T05:30:11.258000+0000","last_clean":"2024-09-14T05:30:11.258000+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:30:11.258000+0000","last_undegraded":"2024-09-14T05:30:11.258000+0000","last_fullsized":"2024-09-14T05:30:11.258000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"66'308","reported_seq":10227,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.341709+0000","last_change":"2024-09-14T05:29:26.019261+0000","last_active":"2024-09-14T05:30:11.341709+0000","last_peered":"2024-09-14T05:30:11.341709+0000","last_clean":"2024-09-14T05:30:11.341709+0000","last_became_active":"2024-09-14T05:29:24.011764+0000","last_became_peered":"2024-09-14T05:29:24.011764+0000","last_unstale":"2024-09-14T05:30:11.341709+0000","last_undegraded":"2024-09-14T05:30:11.341709+0000","last_fullsized":"2024-09-14T05:30:11.341709+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:03:40.768055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055502299999999997,"stat_sum":{"num_bytes":42046776,"num_objects":60,"num_object_clones":14,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":31,"num_read":11201,"num_read_kb":37709,"num_write":280,"num_write_kb":4114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":108,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.303213+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:30:11.303213+0000","last_peered":"2024-09-14T05:30:11.303213+0000","last_clean":"2024-09-14T05:30:11.303213+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:30:11.303213+0000","last_undegraded":"2024-09-14T05:30:11.303213+0000","last_fullsized":"2024-09-14T05:30:11.303213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"66'565","reported_seq":1462,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256988+0000","last_change":"2024-09-14T05:29:40.095721+0000","last_active":"2024-09-14T05:30:11.256988+0000","last_peered":"2024-09-14T05:30:11.256988+0000","last_clean":"2024-09-14T05:30:11.256988+0000","last_became_active":"2024-09-14T05:29:40.095255+0000","last_became_peered":"2024-09-14T05:29:40.095255+0000","last_unstale":"2024-09-14T05:30:11.256988+0000","last_undegraded":"2024-09-14T05:30:11.256988+0000","last_fullsized":"2024-09-14T05:30:11.256988+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:06:37.866414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077857800000000004,"stat_sum":{"num_bytes":49326784,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":25,"num_read":2877,"num_read_kb":5289,"num_write":813,"num_write_kb":8865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":108,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256588+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:30:11.256588+0000","last_peered":"2024-09-14T05:30:11.256588+0000","last_clean":"2024-09-14T05:30:11.256588+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:30:11.256588+0000","last_undegraded":"2024-09-14T05:30:11.256588+0000","last_fullsized":"2024-09-14T05:30:11.256588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"66'433","reported_seq":2119,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.417016+0000","last_change":"2024-09-14T05:29:26.015124+0000","last_active":"2024-09-14T05:30:11.417016+0000","last_peered":"2024-09-14T05:30:11.417016+0000","last_clean":"2024-09-14T05:30:11.417016+0000","last_became_active":"2024-09-14T05:29:24.015792+0000","last_became_peered":"2024-09-14T05:29:24.015792+0000","last_unstale":"2024-09-14T05:30:11.417016+0000","last_undegraded":"2024-09-14T05:30:11.417016+0000","last_fullsized":"2024-09-14T05:30:11.417016+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:56:13.066139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001049964,"stat_sum":{"num_bytes":16760193,"num_objects":67,"num_object_clones":16,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":40,"num_read":3198,"num_read_kb":7568,"num_write":508,"num_write_kb":3535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":109,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256992+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:30:11.256992+0000","last_peered":"2024-09-14T05:30:11.256992+0000","last_clean":"2024-09-14T05:30:11.256992+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:30:11.256992+0000","last_undegraded":"2024-09-14T05:30:11.256992+0000","last_fullsized":"2024-09-14T05:30:11.256992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":199,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.261017+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:30:11.261017+0000","last_peered":"2024-09-14T05:30:11.261017+0000","last_clean":"2024-09-14T05:30:11.261017+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:30:11.261017+0000","last_undegraded":"2024-09-14T05:30:11.261017+0000","last_fullsized":"2024-09-14T05:30:11.261017+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"66'189","reported_seq":1060,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256394+0000","last_change":"2024-09-14T05:29:26.014430+0000","last_active":"2024-09-14T05:30:11.256394+0000","last_peered":"2024-09-14T05:30:11.256394+0000","last_clean":"2024-09-14T05:30:11.256394+0000","last_became_active":"2024-09-14T05:29:24.012222+0000","last_became_peered":"2024-09-14T05:29:24.012222+0000","last_unstale":"2024-09-14T05:30:11.256394+0000","last_undegraded":"2024-09-14T05:30:11.256394+0000","last_fullsized":"2024-09-14T05:30:11.256394+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:22:30.510692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076580099999999998,"stat_sum":{"num_bytes":26728033,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":27,"num_read":834,"num_read_kb":4295,"num_write":161,"num_write_kb":3445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"66'270","reported_seq":1092,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.398480+0000","last_change":"2024-09-14T05:29:26.164542+0000","last_active":"2024-09-14T05:30:11.398480+0000","last_peered":"2024-09-14T05:30:11.398480+0000","last_clean":"2024-09-14T05:30:11.398480+0000","last_became_active":"2024-09-14T05:29:24.012041+0000","last_became_peered":"2024-09-14T05:29:24.012041+0000","last_unstale":"2024-09-14T05:30:11.398480+0000","last_undegraded":"2024-09-14T05:30:11.398480+0000","last_fullsized":"2024-09-14T05:30:11.398480+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:40:03.452031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078334300000000004,"stat_sum":{"num_bytes":12716752,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":25,"num_read":1105,"num_read_kb":1839,"num_write":286,"num_write_kb":2491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"66'444","reported_seq":1845,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.383567+0000","last_change":"2024-09-14T05:29:26.165164+0000","last_active":"2024-09-14T05:30:11.383567+0000","last_peered":"2024-09-14T05:30:11.383567+0000","last_clean":"2024-09-14T05:30:11.383567+0000","last_became_active":"2024-09-14T05:29:24.011403+0000","last_became_peered":"2024-09-14T05:29:24.011403+0000","last_unstale":"2024-09-14T05:30:11.383567+0000","last_undegraded":"2024-09-14T05:30:11.383567+0000","last_fullsized":"2024-09-14T05:30:11.383567+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:55.865120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083114500000000004,"stat_sum":{"num_bytes":18545837,"num_objects":64,"num_object_clones":18,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":30,"num_read":2984,"num_read_kb":7171,"num_write":548,"num_write_kb":4514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"66'436","reported_seq":9409,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.398029+0000","last_change":"2024-09-14T05:29:26.163783+0000","last_active":"2024-09-14T05:30:11.398029+0000","last_peered":"2024-09-14T05:30:11.398029+0000","last_clean":"2024-09-14T05:30:11.398029+0000","last_became_active":"2024-09-14T05:29:24.006836+0000","last_became_peered":"2024-09-14T05:29:24.006836+0000","last_unstale":"2024-09-14T05:30:11.398029+0000","last_undegraded":"2024-09-14T05:30:11.398029+0000","last_fullsized":"2024-09-14T05:30:11.398029+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:04:42.666646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031188400000000001,"stat_sum":{"num_bytes":37332916,"num_objects":64,"num_object_clones":10,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":35,"num_read":11206,"num_read_kb":29064,"num_write":535,"num_write_kb":12685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"66'198","reported_seq":2421,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T05:30:12.320668+0000","last_change":"2024-09-14T05:29:26.020113+0000","last_active":"2024-09-14T05:30:12.320668+0000","last_peered":"2024-09-14T05:30:12.320668+0000","last_clean":"2024-09-14T05:30:12.320668+0000","last_became_active":"2024-09-14T05:29:24.012574+0000","last_became_peered":"2024-09-14T05:29:24.012574+0000","last_unstale":"2024-09-14T05:30:12.320668+0000","last_undegraded":"2024-09-14T05:30:12.320668+0000","last_fullsized":"2024-09-14T05:30:12.320668+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:15:13.596278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072680599999999998,"stat_sum":{"num_bytes":39953671,"num_objects":60,"num_object_clones":20,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":32,"num_read":2199,"num_read_kb":8085,"num_write":166,"num_write_kb":5126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"66'774","reported_seq":9609,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.428007+0000","last_change":"2024-09-14T05:29:26.164736+0000","last_active":"2024-09-14T05:30:11.428007+0000","last_peered":"2024-09-14T05:30:11.428007+0000","last_clean":"2024-09-14T05:30:11.428007+0000","last_became_active":"2024-09-14T05:29:24.012401+0000","last_became_peered":"2024-09-14T05:29:24.012401+0000","last_unstale":"2024-09-14T05:30:11.428007+0000","last_undegraded":"2024-09-14T05:30:11.428007+0000","last_fullsized":"2024-09-14T05:30:11.428007+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:25:12.169350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085260200000000005,"stat_sum":{"num_bytes":43455562,"num_objects":72,"num_object_clones":18,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":40,"num_read":11810,"num_read_kb":19585,"num_write":1063,"num_write_kb":4975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"66'194","reported_seq":1587,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.447905+0000","last_change":"2024-09-14T05:29:26.028811+0000","last_active":"2024-09-14T05:30:11.447905+0000","last_peered":"2024-09-14T05:30:11.447905+0000","last_clean":"2024-09-14T05:30:11.447905+0000","last_became_active":"2024-09-14T05:29:24.015315+0000","last_became_peered":"2024-09-14T05:29:24.015315+0000","last_unstale":"2024-09-14T05:30:11.447905+0000","last_undegraded":"2024-09-14T05:30:11.447905+0000","last_fullsized":"2024-09-14T05:30:11.447905+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:55:30.171267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046485199999999997,"stat_sum":{"num_bytes":25287542,"num_objects":65,"num_object_clones":20,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":39,"num_read":1331,"num_read_kb":2486,"num_write":169,"num_write_kb":3014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"67'309","reported_seq":10735,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T05:30:12.481411+0000","last_change":"2024-09-14T05:29:26.020027+0000","last_active":"2024-09-14T05:30:12.481411+0000","last_peered":"2024-09-14T05:30:12.481411+0000","last_clean":"2024-09-14T05:30:12.481411+0000","last_became_active":"2024-09-14T05:29:24.018854+0000","last_became_peered":"2024-09-14T05:29:24.018854+0000","last_unstale":"2024-09-14T05:30:12.481411+0000","last_undegraded":"2024-09-14T05:30:12.481411+0000","last_fullsized":"2024-09-14T05:30:12.481411+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:56.376950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063672000000000002,"stat_sum":{"num_bytes":34972261,"num_objects":83,"num_object_clones":22,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":45,"num_read":11220,"num_read_kb":29115,"num_write":284,"num_write_kb":10826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"65'229","reported_seq":1234,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.402693+0000","last_change":"2024-09-14T05:29:26.014200+0000","last_active":"2024-09-14T05:30:11.402693+0000","last_peered":"2024-09-14T05:30:11.402693+0000","last_clean":"2024-09-14T05:30:11.402693+0000","last_became_active":"2024-09-14T05:29:24.010897+0000","last_became_peered":"2024-09-14T05:29:24.010897+0000","last_unstale":"2024-09-14T05:30:11.402693+0000","last_undegraded":"2024-09-14T05:30:11.402693+0000","last_fullsized":"2024-09-14T05:30:11.402693+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:37:30.543147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051531799999999996,"stat_sum":{"num_bytes":33936373,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":26,"num_read":1014,"num_read_kb":8865,"num_write":202,"num_write_kb":6050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"66'344","reported_seq":2324,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.431151+0000","last_change":"2024-09-14T05:29:26.164788+0000","last_active":"2024-09-14T05:30:11.431151+0000","last_peered":"2024-09-14T05:30:11.431151+0000","last_clean":"2024-09-14T05:30:11.431151+0000","last_became_active":"2024-09-14T05:29:24.013792+0000","last_became_peered":"2024-09-14T05:29:24.013792+0000","last_unstale":"2024-09-14T05:30:11.431151+0000","last_undegraded":"2024-09-14T05:30:11.431151+0000","last_fullsized":"2024-09-14T05:30:11.431151+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:08:01.065225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090671299999999996,"stat_sum":{"num_bytes":29225047,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":3686,"num_read_kb":15807,"num_write":343,"num_write_kb":2409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"65'211","reported_seq":1246,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.417214+0000","last_change":"2024-09-14T05:29:40.095745+0000","last_active":"2024-09-14T05:30:11.417214+0000","last_peered":"2024-09-14T05:30:11.417214+0000","last_clean":"2024-09-14T05:30:11.417214+0000","last_became_active":"2024-09-14T05:29:40.095331+0000","last_became_peered":"2024-09-14T05:29:40.095331+0000","last_unstale":"2024-09-14T05:30:11.417214+0000","last_undegraded":"2024-09-14T05:30:11.417214+0000","last_fullsized":"2024-09-14T05:30:11.417214+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:29:19.959417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000774981,"stat_sum":{"num_bytes":27414058,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":1534,"num_read_kb":8788,"num_write":192,"num_write_kb":4039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"66'446","reported_seq":1759,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-14T05:30:11.256917+0000","last_change":"2024-09-14T05:29:26.164295+0000","last_active":"2024-09-14T05:30:11.256917+0000","last_peered":"2024-09-14T05:30:11.256917+0000","last_clean":"2024-09-14T05:30:11.256917+0000","last_became_active":"2024-09-14T05:29:24.014371+0000","last_became_peered":"2024-09-14T05:29:24.014371+0000","last_unstale":"2024-09-14T05:30:11.256917+0000","last_undegraded":"2024-09-14T05:30:11.256917+0000","last_fullsized":"2024-09-14T05:30:11.256917+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:32:06.372111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050570599999999997,"stat_sum":{"num_bytes":56505386,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":26,"num_read":2769,"num_read_kb":18316,"num_write":653,"num_write_kb":4935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1073439577,"num_objects":1900,"num_object_clones":487,"num_object_copies":3800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1900,"num_whiteouts":1007,"num_read":142159,"num_read_kb":448872,"num_write":12692,"num_write_kb":221667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":269176832,"data_stored":278252433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11034,"ondisk_log_size":11034,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476753,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88248,"kb_used_data":48740,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94283592,"statfs":{"total":96636764160,"available":96546398208,"internally_reserved":0,"allocated":49909760,"data_stored":50231088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53520,"kb_used_data":22328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94318320,"statfs":{"total":96636764160,"available":96581959680,"internally_reserved":0,"allocated":22863872,"data_stored":22825640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66656,"kb_used_data":27140,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94305184,"statfs":{"total":96636764160,"available":96568508416,"internally_reserved":0,"allocated":27791360,"data_stored":29005176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83860,"kb_used_data":44352,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94287980,"statfs":{"total":96636764160,"available":96550891520,"internally_reserved":0,"allocated":45416448,"data_stored":46429833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67008,"kb_used_data":31664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94304832,"statfs":{"total":96636764160,"available":96568147968,"internally_reserved":0,"allocated":32423936,"data_stored":32787746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63384,"kb_used_data":23844,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94308456,"statfs":{"total":96636764160,"available":96571858944,"internally_reserved":0,"allocated":24416256,"data_stored":25416164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64212,"kb_used_data":24708,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94307628,"statfs":{"total":96636764160,"available":96571011072,"internally_reserved":0,"allocated":25300992,"data_stored":26172013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85428,"kb_used_data":45888,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94286412,"statfs":{"total":96636764160,"available":96549285888,"internally_reserved":0,"allocated":46989312,"data_stored":48073925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49963008,"data_stored":50678949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45649920,"data_stored":47147496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24694784,"data_stored":25965649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23789568,"data_stored":25191018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":31768576,"data_stored":32529631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":44085248,"data_stored":45509553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":27037696,"data_stored":28669034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":22188032,"data_stored":22561103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:30:15.416 INFO:teuthology.orchestra.run.smithi071.stdout:4592 write 0xaeb8830 thru 0xaec2b7d (0xa34e bytes) 2024-09-14T05:30:15.424 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:15.423+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:15.429 INFO:teuthology.orchestra.run.smithi071.stdout:4593 writesame 0xd403383 thru 0xd4129a0 data_size 0x3(0xf61e bytes) 2024-09-14T05:30:15.429 INFO:teuthology.orchestra.run.smithi071.stdout:4594 read 0xcf31cd8 thru 0xcf3215b (0x484 bytes) 2024-09-14T05:30:15.430 INFO:teuthology.orchestra.run.smithi071.stdout:4596 read 0x3c137d3 thru 0x3c15686 (0x1eb4 bytes) 2024-09-14T05:30:15.430 INFO:teuthology.orchestra.run.smithi071.stdout:4597 write 0x85159ca thru 0x851b6f5 (0x5d2c bytes) 2024-09-14T05:30:15.436 INFO:teuthology.orchestra.run.smithi071.stdout:4598 writesame 0xffc176 thru 0x100b0df data_size 0x87(0xef6a bytes) 2024-09-14T05:30:15.442 INFO:teuthology.orchestra.run.smithi071.stdout:4599 read 0xb5659bc thru 0xb56a113 (0x4758 bytes) 2024-09-14T05:30:15.442 INFO:teuthology.orchestra.run.smithi071.stdout:4600 write 0x9058757 thru 0x905bc2f (0x34d9 bytes) 2024-09-14T05:30:15.446 INFO:teuthology.orchestra.run.smithi071.stdout:4601 read 0x543a443 thru 0x5441352 (0x6f10 bytes) 2024-09-14T05:30:15.446 INFO:teuthology.orchestra.run.smithi071.stdout:4602 read 0xa373a86 thru 0xa374c82 (0x11fd bytes) 2024-09-14T05:30:15.446 INFO:teuthology.orchestra.run.smithi071.stdout:4603 read 0x79618cf thru 0x7966d74 (0x54a6 bytes) 2024-09-14T05:30:15.446 INFO:teuthology.orchestra.run.smithi071.stdout:4604 write 0xb0003a3 thru 0xb0064af (0x610d bytes) 2024-09-14T05:30:15.450 INFO:teuthology.orchestra.run.smithi071.stdout:4605 punch from 0x585da37 to 0x5869851, (0xbe1a bytes) 2024-09-14T05:30:15.450 INFO:teuthology.orchestra.run.smithi071.stdout:4607 writesame 0x51bac40 thru 0x51c54ea data_size 0x25(0xa8ab bytes) 2024-09-14T05:30:15.454 INFO:teuthology.orchestra.run.smithi071.stdout:4609 read 0x2b1930c thru 0x2b2457c (0xb271 bytes) 2024-09-14T05:30:15.454 INFO:teuthology.orchestra.run.smithi071.stdout:4610 write 0xb54e2cc thru 0xb554e26 (0x6b5b bytes) 2024-09-14T05:30:15.458 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.458 INFO:teuthology.orchestra.run.smithi071.stdout:4612 write 0xc3f212e thru 0xc3fff0e (0xdde1 bytes) 2024-09-14T05:30:15.468 INFO:teuthology.orchestra.run.smithi071.stdout:4613 write 0xdf19118 thru 0xdf20983 (0x786c bytes) 2024-09-14T05:30:15.472 INFO:teuthology.orchestra.run.smithi071.stdout:4614 write 0xe9f194 thru 0xe9f593 (0x400 bytes) 2024-09-14T05:30:15.474 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:15.475 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:15.481 INFO:teuthology.orchestra.run.smithi071.stdout:4616 trunc from 0xdf20984 to 0x9604410 2024-09-14T05:30:15.488 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:30:15.494 INFO:teuthology.orchestra.run.smithi071.stdout:4617 trunc from 0x9604410 to 0xbb5cfaa 2024-09-14T05:30:15.506 INFO:teuthology.orchestra.run.smithi071.stdout:4618 write 0xd6600d8 thru 0xd66af90 (0xaeb9 bytes) 2024-09-14T05:30:15.510 INFO:teuthology.orchestra.run.smithi071.stdout:4619 writesame 0x3d25ad thru 0x3da03e data_size 0x1d(0x7a92 bytes) 2024-09-14T05:30:15.512 INFO:teuthology.orchestra.run.smithi071.stdout:4620 read 0x105b264 thru 0x10688c6 (0xd663 bytes) 2024-09-14T05:30:15.512 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.512 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.512 INFO:teuthology.orchestra.run.smithi071.stdout:4623 punch from 0x4edaaf4 to 0x4ee68e3, (0xbdef bytes) 2024-09-14T05:30:15.513 INFO:teuthology.orchestra.run.smithi071.stdout:4624 read 0xc111e39 thru 0xc11cc02 (0xadca bytes) 2024-09-14T05:30:15.513 INFO:teuthology.orchestra.run.smithi071.stdout:4625 writesame 0x162017c thru 0x1628b90 data_size 0x1(0x8a15 bytes) 2024-09-14T05:30:15.524 INFO:teuthology.orchestra.run.smithi071.stdout:4626 read 0xcee9004 thru 0xcef7c47 (0xec44 bytes) 2024-09-14T05:30:15.524 INFO:teuthology.orchestra.run.smithi071.stdout:4627 write 0xaff61f8 thru 0xb0030e5 (0xceee bytes) 2024-09-14T05:30:15.524 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:15.523+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:15.530 INFO:teuthology.orchestra.run.smithi071.stdout:4628 writesame 0x3455f50 thru 0x3465bda data_size 0x1(0xfc8b bytes) 2024-09-14T05:30:15.557 INFO:teuthology.orchestra.run.smithi071.stdout:4629 write 0x3e47de4 thru 0x3e55b6f (0xdd8c bytes) 2024-09-14T05:30:15.561 INFO:teuthology.orchestra.run.smithi071.stdout:4631 punch from 0x60b170 to 0x6101cc, (0x505c bytes) 2024-09-14T05:30:15.561 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.561 INFO:teuthology.orchestra.run.smithi071.stdout:4633 punch from 0x3d42eda to 0x3d4af00, (0x8026 bytes) 2024-09-14T05:30:15.561 INFO:teuthology.orchestra.run.smithi071.stdout:4634 read 0xc94d5f0 thru 0xc95d2c9 (0xfcda bytes) 2024-09-14T05:30:15.561 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.561 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.561 INFO:teuthology.orchestra.run.smithi071.stdout:4640 read 0x2b5fc thru 0x30691 (0x5096 bytes) 2024-09-14T05:30:15.562 INFO:teuthology.orchestra.run.smithi071.stdout:4641 read 0x142a33a thru 0x142cea6 (0x2b6d bytes) 2024-09-14T05:30:15.562 INFO:teuthology.orchestra.run.smithi071.stdout:4642 read 0x4ecf66a thru 0x4ed59bd (0x6354 bytes) 2024-09-14T05:30:15.562 INFO:teuthology.orchestra.run.smithi071.stdout:4644 punch from 0x4945477 to 0x4950754, (0xb2dd bytes) 2024-09-14T05:30:15.562 INFO:teuthology.orchestra.run.smithi071.stdout:4646 punch from 0xcd36678 to 0xcd4203e, (0xb9c6 bytes) 2024-09-14T05:30:15.576 INFO:teuthology.orchestra.run.smithi071.stdout:4650 trunc from 0xd66af91 to 0x116ed36 2024-09-14T05:30:15.592 INFO:teuthology.orchestra.run.smithi071.stdout:4651 punch from 0x4a493 to 0x4f884, (0x53f1 bytes) 2024-09-14T05:30:15.611 INFO:teuthology.orchestra.run.smithi071.stdout:4652 write 0xc8893c3 thru 0xc898eca (0xfb08 bytes) 2024-09-14T05:30:15.617 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:15.617 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:15.625 INFO:teuthology.orchestra.run.smithi071.stdout:4654 trunc from 0xc898ecb to 0x59a858e 2024-09-14T05:30:15.626 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:30:15.626 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:15.624+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:15.636 INFO:teuthology.orchestra.run.smithi071.stdout:4655 trunc from 0x59a858e to 0x45dc454 2024-09-14T05:30:15.642 INFO:teuthology.orchestra.run.smithi071.stdout:4656 write 0x88af8d thru 0x893a87 (0x8afb bytes) 2024-09-14T05:30:15.664 INFO:teuthology.orchestra.run.smithi071.stdout:4658 writesame 0xe85aec2 thru 0xe86925c data_size 0x1(0xe39b bytes) 2024-09-14T05:30:15.664 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.669 INFO:teuthology.orchestra.run.smithi071.stdout:4660 write 0xeacd7bc thru 0xead5adc (0x8321 bytes) 2024-09-14T05:30:15.675 INFO:teuthology.orchestra.run.smithi071.stdout:4661 writesame 0x64a04ee thru 0x64a538f data_size 0x42(0x4ea2 bytes) 2024-09-14T05:30:15.680 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.681 INFO:teuthology.orchestra.run.smithi071.stdout:4666 punch from 0x6ad5f6d to 0x6ae1475, (0xb508 bytes) 2024-09-14T05:30:15.681 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.681 INFO:teuthology.orchestra.run.smithi071.stdout:4672 writesame 0x2e0647 thru 0x2e3bcf data_size 0x1(0x3589 bytes) 2024-09-14T05:30:15.686 INFO:teuthology.orchestra.run.smithi071.stdout:4673 read 0x8548541 thru 0x8550d07 (0x87c7 bytes) 2024-09-14T05:30:15.686 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.686 INFO:teuthology.orchestra.run.smithi071.stdout:4676 write 0xe560cb7 thru 0xe566ad6 (0x5e20 bytes) 2024-09-14T05:30:15.690 INFO:teuthology.orchestra.run.smithi071.stdout:4677 read 0x4cf688c thru 0x4d029b2 (0xc127 bytes) 2024-09-14T05:30:15.690 INFO:teuthology.orchestra.run.smithi071.stdout:4678 writesame 0x512d750 thru 0x51378ae data_size 0x1(0xa15f bytes) 2024-09-14T05:30:15.706 INFO:teuthology.orchestra.run.smithi071.stdout:4679 read 0x980a902 thru 0x98169e0 (0xc0df bytes) 2024-09-14T05:30:15.706 INFO:teuthology.orchestra.run.smithi071.stdout:4680 write 0x1aaf5a6 thru 0x1ab0ec0 (0x191b bytes) 2024-09-14T05:30:15.711 INFO:teuthology.orchestra.run.smithi071.stdout:4682 trunc from 0xead5add to 0xdb9740c 2024-09-14T05:30:15.722 INFO:teuthology.orchestra.run.smithi071.stdout:4683 write 0xacb8cf5 thru 0xacbe57f (0x588b bytes) 2024-09-14T05:30:15.726 INFO:teuthology.orchestra.run.smithi071.stdout:4686 punch from 0x2a8e9ce to 0x2a9e40b, (0xfa3d bytes) 2024-09-14T05:30:15.726 INFO:teuthology.orchestra.run.smithi071.stdout:4688 punch from 0xb02ae03 to 0xb0370db, (0xc2d8 bytes) 2024-09-14T05:30:15.726 INFO:teuthology.orchestra.run.smithi071.stdout:4689 read 0x3b4a43e thru 0x3b4adf5 (0x9b8 bytes) 2024-09-14T05:30:15.726 INFO:teuthology.orchestra.run.smithi071.stdout:4690 write 0x3961cc8 thru 0x3970644 (0xe97d bytes) 2024-09-14T05:30:15.727 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:15.725+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:15.729 INFO:teuthology.orchestra.run.smithi071.stdout:4691 read 0x4892922 thru 0x48a0624 (0xdd03 bytes) 2024-09-14T05:30:15.730 INFO:teuthology.orchestra.run.smithi071.stdout:4693 write 0xd11f56c thru 0xd1292e7 (0x9d7c bytes) 2024-09-14T05:30:15.734 INFO:teuthology.orchestra.run.smithi071.stdout:4694 writesame 0xb2d67d9 thru 0xb2df8aa data_size 0x6f(0x90d2 bytes) 2024-09-14T05:30:15.737 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.737 INFO:teuthology.orchestra.run.smithi071.stdout:4696 writesame 0x4e1040b thru 0x4e1fcee data_size 0x22(0xf8e4 bytes) 2024-09-14T05:30:15.753 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:15.753 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:15.756 INFO:teuthology.orchestra.run.smithi071.stdout:4697 trunc from 0xdb9740c to 0x1993cc3 2024-09-14T05:30:15.766 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:30:15.771 INFO:teuthology.orchestra.run.smithi071.stdout:4698 trunc from 0x1993cc3 to 0x43e5aee 2024-09-14T05:30:15.794 INFO:teuthology.orchestra.run.smithi071.stdout:4699 write 0xde0ab87 thru 0xde15fb7 (0xb431 bytes) 2024-09-14T05:30:15.798 INFO:teuthology.orchestra.run.smithi071.stdout:4701 write 0xd8252 thru 0xe08fd (0x86ac bytes) 2024-09-14T05:30:15.800 INFO:teuthology.orchestra.run.smithi071.stdout:4702 read 0x3a413ca thru 0x3a4503e (0x3c75 bytes) 2024-09-14T05:30:15.809 INFO:teuthology.orchestra.run.smithi071.stdout:4703 trunc from 0xde15fb8 to 0x659a193 2024-09-14T05:30:15.819 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.819 INFO:teuthology.orchestra.run.smithi071.stdout:4707 read 0x61431f3 thru 0x614f5ee (0xc3fc bytes) 2024-09-14T05:30:15.819 INFO:teuthology.orchestra.run.smithi071.stdout:4708 read 0x4de7237 thru 0x4debcbe (0x4a88 bytes) 2024-09-14T05:30:15.824 INFO:teuthology.orchestra.run.smithi071.stdout:4709 trunc from 0x659a193 to 0xa3e6f12 2024-09-14T05:30:15.827 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:15.826+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:4711 punch from 0x81dcf9f to 0x81e3fb9, (0x701a bytes) 2024-09-14T05:30:15.836 INFO:teuthology.orchestra.run.smithi071.stdout:4712 write 0xaa33e1d thru 0xaa3f689 (0xb86d bytes) 2024-09-14T05:30:15.839 INFO:teuthology.orchestra.run.smithi071.stdout:4714 read 0x402271a thru 0x40311c8 (0xeaaf bytes) 2024-09-14T05:30:15.841 INFO:teuthology.orchestra.run.smithi071.stdout:4715 trunc from 0xaa3f68a to 0x99a86f3 2024-09-14T05:30:15.850 INFO:teuthology.orchestra.run.smithi071.stdout:4716 writesame 0xb5c3dc thru 0xb5db7b data_size 0x4(0x17a0 bytes) 2024-09-14T05:30:15.853 INFO:teuthology.orchestra.run.smithi071.stdout:4717 write 0x5f6cf0a thru 0x5f7ca5a (0xfb51 bytes) 2024-09-14T05:30:15.857 INFO:teuthology.orchestra.run.smithi071.stdout:4718 punch from 0x26181f8 to 0x2623be8, (0xb9f0 bytes) 2024-09-14T05:30:15.866 INFO:teuthology.orchestra.run.smithi071.stdout:4719 write 0xc34a921 thru 0xc34f624 (0x4d04 bytes) 2024-09-14T05:30:15.870 INFO:teuthology.orchestra.run.smithi071.stdout:4720 flatten 2024-09-14T05:30:15.894 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:15.894 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:15.905 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:30:15.923 INFO:teuthology.orchestra.run.smithi071.stdout:4721 trunc from 0xc34f625 to 0x34daf82 2024-09-14T05:30:15.928 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:15.927+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:15.947 INFO:teuthology.orchestra.run.smithi071.stdout:4722 trunc from 0x34daf82 to 0xdd2b3a1 2024-09-14T05:30:15.952 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:15.952 INFO:teuthology.orchestra.run.smithi071.stdout:4725 read 0x36a84d3 thru 0x36b783d (0xf36b bytes) 2024-09-14T05:30:15.963 INFO:teuthology.orchestra.run.smithi071.stdout:4726 trunc from 0xdd2b3a1 to 0x47ad13f 2024-09-14T05:30:15.972 INFO:teuthology.orchestra.run.smithi071.stdout:4729 trunc from 0x47ad13f to 0x63ffbd7 2024-09-14T05:30:15.986 INFO:teuthology.orchestra.run.smithi071.stdout:4730 write 0xa3e6bb7 thru 0xa3f0b73 (0x9fbd bytes) 2024-09-14T05:30:15.993 INFO:teuthology.orchestra.run.smithi071.stdout:4731 punch from 0x3441b6f to 0x3443757, (0x1be8 bytes) 2024-09-14T05:30:16.005 INFO:teuthology.orchestra.run.smithi071.stdout:4732 writesame 0xea962a6 thru 0xea9cf11 data_size 0x24(0x6c6c bytes) 2024-09-14T05:30:16.005 INFO:teuthology.orchestra.run.smithi071.stdout:4733 read 0xdd963da thru 0xdd9cae5 (0x670c bytes) 2024-09-14T05:30:16.016 INFO:teuthology.orchestra.run.smithi071.stdout:4734 trunc from 0xea9cf12 to 0x4f0c047 2024-09-14T05:30:16.028 INFO:teuthology.orchestra.run.smithi071.stdout:4735 trunc from 0x4f0c047 to 0x3983f25 2024-09-14T05:30:16.029 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:16.028+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:16.041 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:16.041 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:16.048 INFO:teuthology.orchestra.run.smithi071.stdout:4737 write 0xadc283a thru 0xadd25ad (0xfd74 bytes) 2024-09-14T05:30:16.052 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:30:16.060 INFO:teuthology.orchestra.run.smithi071.stdout:4738 write 0xb64bcfb thru 0xb65932e (0xd634 bytes) 2024-09-14T05:30:16.065 INFO:teuthology.orchestra.run.smithi071.stdout:4739 writesame 0x98c1370 thru 0x98d0b16 data_size 0x7(0xf7a7 bytes) 2024-09-14T05:30:16.073 INFO:teuthology.orchestra.run.smithi071.stdout:4740 write 0x5765a7c thru 0x576a61a (0x4b9f bytes) 2024-09-14T05:30:16.077 INFO:teuthology.orchestra.run.smithi071.stdout:4741 writesame 0xa1ede62 thru 0xa1eea97 data_size 0x3(0xc36 bytes) 2024-09-14T05:30:16.082 INFO:teuthology.orchestra.run.smithi071.stdout:4744 punch from 0x27a9ad8 to 0x27b93c2, (0xf8ea bytes) 2024-09-14T05:30:16.082 INFO:teuthology.orchestra.run.smithi071.stdout:4745 punch from 0x4cca6 to 0x5bc32, (0xef8c bytes) 2024-09-14T05:30:16.082 INFO:teuthology.orchestra.run.smithi071.stdout:4746 punch from 0x9b77947 to 0x9b77c7f, (0x338 bytes) 2024-09-14T05:30:16.082 INFO:teuthology.orchestra.run.smithi071.stdout:4747 punch from 0xa5671a2 to 0xa5740c4, (0xcf22 bytes) 2024-09-14T05:30:16.082 INFO:teuthology.orchestra.run.smithi071.stdout:4748 writesame 0x5f76df0 thru 0x5f7b72d data_size 0x1e(0x493e bytes) 2024-09-14T05:30:16.086 INFO:teuthology.orchestra.run.smithi071.stdout:4752 read 0xb5f4f0b thru 0xb600b26 (0xbc1c bytes) 2024-09-14T05:30:16.086 INFO:teuthology.orchestra.run.smithi071.stdout:4753 punch from 0x64a814e to 0x64abff1, (0x3ea3 bytes) 2024-09-14T05:30:16.098 INFO:teuthology.orchestra.run.smithi071.stdout:4754 trunc from 0xb65932f to 0x11fd101 2024-09-14T05:30:16.109 INFO:teuthology.orchestra.run.smithi071.stdout:4755 read 0xf7fe82 thru 0xf8bed1 (0xc050 bytes) 2024-09-14T05:30:16.110 INFO:teuthology.orchestra.run.smithi071.stdout:4757 punch from 0x96e6c0 to 0x974f59, (0x6899 bytes) 2024-09-14T05:30:16.127 INFO:teuthology.orchestra.run.smithi071.stdout:4758 writesame 0xac4447a thru 0xac517f6 data_size 0x1(0xd37d bytes) 2024-09-14T05:30:16.129 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:16.128+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:16.132 INFO:teuthology.orchestra.run.smithi071.stdout:4761 trunc from 0xac517f7 to 0xeac84f7 2024-09-14T05:30:16.137 INFO:teuthology.orchestra.run.smithi071.stdout:4762 read 0x97e1e6e thru 0x97e2e08 (0xf9b bytes) 2024-09-14T05:30:16.138 INFO:teuthology.orchestra.run.smithi071.stdout:4764 punch from 0x9ef905f to 0x9efc9d3, (0x3974 bytes) 2024-09-14T05:30:16.138 INFO:teuthology.orchestra.run.smithi071.stdout:4765 write 0x2a6e2a3 thru 0x2a70299 (0x1ff7 bytes) 2024-09-14T05:30:16.147 INFO:teuthology.orchestra.run.smithi071.stdout:4766 read 0x46a4405 thru 0x46a64a6 (0x20a2 bytes) 2024-09-14T05:30:16.147 INFO:teuthology.orchestra.run.smithi071.stdout:4767 punch from 0xee1b6b to 0xeece83, (0xb318 bytes) 2024-09-14T05:30:16.147 INFO:teuthology.orchestra.run.smithi071.stdout:4768 writesame 0x9f15ba0 thru 0x9f22154 data_size 0x1(0xc5b5 bytes) 2024-09-14T05:30:16.162 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:30:16.177 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:16.177 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:16.186 INFO:teuthology.orchestra.run.smithi071.stdout:4769 trunc from 0xeac84f7 to 0x88a47c 2024-09-14T05:30:16.192 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:30:16.199 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:16.200 INFO:teuthology.orchestra.run.smithi071.stdout:4772 read 0x6b19c0 thru 0x6b3c55 (0x2296 bytes) 2024-09-14T05:30:16.200 INFO:teuthology.orchestra.run.smithi071.stdout:4773 read 0x5110a6 thru 0x51e3ef (0xd34a bytes) 2024-09-14T05:30:16.211 INFO:teuthology.orchestra.run.smithi071.stdout:4777 write 0x50063cc thru 0x5008a2f (0x2664 bytes) 2024-09-14T05:30:16.215 INFO:teuthology.orchestra.run.smithi071.stdout:4779 read 0x3fff045 thru 0x4008b88 (0x9b44 bytes) 2024-09-14T05:30:16.215 INFO:teuthology.orchestra.run.smithi071.stdout:4780 write 0x2082e57 thru 0x20834ab (0x655 bytes) 2024-09-14T05:30:16.218 INFO:teuthology.orchestra.run.smithi071.stdout:4781 read 0x25665ad thru 0x2568cc5 (0x2719 bytes) 2024-09-14T05:30:16.218 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:16.218 INFO:teuthology.orchestra.run.smithi071.stdout:4785 write 0x31eb810 thru 0x31efdf4 (0x45e5 bytes) 2024-09-14T05:30:16.229 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:16.229+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:16.236 INFO:teuthology.orchestra.run.smithi071.stdout:4786 write 0xcf82a1a thru 0xcf87846 (0x4e2d bytes) 2024-09-14T05:30:16.244 INFO:teuthology.orchestra.run.smithi071.stdout:4787 read 0x550000a thru 0x550a59c (0xa593 bytes) 2024-09-14T05:30:16.244 INFO:teuthology.orchestra.run.smithi071.stdout:4788 read 0x47e6201 thru 0x47f2bb1 (0xc9b1 bytes) 2024-09-14T05:30:16.244 INFO:teuthology.orchestra.run.smithi071.stdout:4790 writesame 0x6f1ee37 thru 0x6f28e8e data_size 0x1c(0xa058 bytes) 2024-09-14T05:30:16.248 INFO:teuthology.orchestra.run.smithi071.stdout:4791 write 0xa70e8d4 thru 0xa71c8c8 (0xdff5 bytes) 2024-09-14T05:30:16.254 INFO:teuthology.orchestra.run.smithi071.stdout:4793 read 0x51e9d11 thru 0x51f221f (0x850f bytes) 2024-09-14T05:30:16.254 INFO:teuthology.orchestra.run.smithi071.stdout:4794 write 0x6e58a8e thru 0x6e5f2de (0x6851 bytes) 2024-09-14T05:30:16.259 INFO:teuthology.orchestra.run.smithi071.stdout:4795 write 0x6381353 thru 0x638b7f0 (0xa49e bytes) 2024-09-14T05:30:16.272 INFO:teuthology.orchestra.run.smithi071.stdout:4796 writesame 0xe79d3ec thru 0xe7ab7d0 data_size 0x1(0xe3e5 bytes) 2024-09-14T05:30:16.272 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:16.272 INFO:teuthology.orchestra.run.smithi071.stdout:4801 read 0xecf0c thru 0xfce80 (0xff75 bytes) 2024-09-14T05:30:16.272 INFO:teuthology.orchestra.run.smithi071.stdout:4803 read 0x28118e thru 0x28a473 (0x92e6 bytes) 2024-09-14T05:30:16.273 INFO:teuthology.orchestra.run.smithi071.stdout:4804 clone 36 order 24 su 1048576 sc 5 2024-09-14T05:30:16.306 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:16.306 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:16.314 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T05:30:16.325 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:16.326 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:16.326 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:16.326 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:16.330 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:16.330+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:16.332 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:30:16.349 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:16.349 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:16.366 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:16.366 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:16.384 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:16.385 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:16.401 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:16.402 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:16.418 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:16.418 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:16.432 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:16.431+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:16.437 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:16.437 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:16.460 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:16.461 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:16.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:16.461 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:16.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.461 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73757 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2816000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.461 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:30:16.461 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.898878781000001, 2024-09-14T05:30:16.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044614540000000001, 2024-09-14T05:30:16.461 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.461 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.461 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.461 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.461 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.462 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.462 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73757 2024-09-14T05:30:16.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.462 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:30:16.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:30:16.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559679+0000", 2024-09-14T05:30:16.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4000000000000002e-07 2024-09-14T05:30:16.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559681+0000", 2024-09-14T05:30:16.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1780000000000002e-06 2024-09-14T05:30:16.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559687+0000", 2024-09-14T05:30:16.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0270000000000001e-06 2024-09-14T05:30:16.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559692+0000", 2024-09-14T05:30:16.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1030000000000001e-06 2024-09-14T05:30:16.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604180+0000", 2024-09-14T05:30:16.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044487480000000003 2024-09-14T05:30:16.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604224+0000", 2024-09-14T05:30:16.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4357999999999998e-05 2024-09-14T05:30:16.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.465 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604293+0000", 2024-09-14T05:30:16.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8954000000000004e-05 2024-09-14T05:30:16.465 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.465 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.465 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.465 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73760 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2848768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.465 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:16.465 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.898858676, 2024-09-14T05:30:16.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.04469911, 2024-09-14T05:30:16.466 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.466 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.466 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.466 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.466 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.466 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.466 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73760 2024-09-14T05:30:16.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.466 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:16.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:16.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:16.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:16.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559701+0000", 2024-09-14T05:30:16.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1909999999999999e-06 2024-09-14T05:30:16.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559708+0000", 2024-09-14T05:30:16.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7279999999999998e-06 2024-09-14T05:30:16.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559713+0000", 2024-09-14T05:30:16.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8389999999999997e-06 2024-09-14T05:30:16.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604310+0000", 2024-09-14T05:30:16.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044597664000000002 2024-09-14T05:30:16.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604352+0000", 2024-09-14T05:30:16.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1236000000000003e-05 2024-09-14T05:30:16.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604398+0000", 2024-09-14T05:30:16.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6091000000000003e-05 2024-09-14T05:30:16.469 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.469 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.469 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.470 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73773 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2801664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.470 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:16.470 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.898838507000001, 2024-09-14T05:30:16.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941026000000002, 2024-09-14T05:30:16.470 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.470 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.470 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.470 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.470 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.470 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.470 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73773 2024-09-14T05:30:16.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.470 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:16.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:16.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:16.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8599999999999999e-07 2024-09-14T05:30:16.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559721+0000", 2024-09-14T05:30:16.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0540000000000002e-06 2024-09-14T05:30:16.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559729+0000", 2024-09-14T05:30:16.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1359999999999997e-06 2024-09-14T05:30:16.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559734+0000", 2024-09-14T05:30:16.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2140000000000004e-06 2024-09-14T05:30:16.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604441+0000", 2024-09-14T05:30:16.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044706828999999997 2024-09-14T05:30:16.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604522+0000", 2024-09-14T05:30:16.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0603000000000001e-05 2024-09-14T05:30:16.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604660+0000", 2024-09-14T05:30:16.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000137804 2024-09-14T05:30:16.474 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.474 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.474 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.474 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73777 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2834432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.474 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:16.474 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.898816105999998, 2024-09-14T05:30:16.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045116270999999999, 2024-09-14T05:30:16.474 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.474 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.474 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.475 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.475 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.475 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.475 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73777 2024-09-14T05:30:16.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.475 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.475 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:16.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.475 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:16.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559742+0000", 2024-09-14T05:30:16.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4099999999999999e-07 2024-09-14T05:30:16.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559744+0000", 2024-09-14T05:30:16.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0040000000000002e-06 2024-09-14T05:30:16.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559749+0000", 2024-09-14T05:30:16.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9739999999999999e-06 2024-09-14T05:30:16.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559754+0000", 2024-09-14T05:30:16.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9540000000000003e-06 2024-09-14T05:30:16.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604695+0000", 2024-09-14T05:30:16.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941003 2024-09-14T05:30:16.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604766+0000", 2024-09-14T05:30:16.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0296999999999995e-05 2024-09-14T05:30:16.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604857+0000", 2024-09-14T05:30:16.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1598e-05 2024-09-14T05:30:16.478 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.478 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.478 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.478 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73790 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.479 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:16.479 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.898796171000001, 2024-09-14T05:30:16.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316437000000001, 2024-09-14T05:30:16.479 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.479 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.479 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.479 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.479 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.479 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.479 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73790 2024-09-14T05:30:16.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.479 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:16.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:16.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:16.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7599999999999998e-07 2024-09-14T05:30:16.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559763+0000", 2024-09-14T05:30:16.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0880000000000002e-06 2024-09-14T05:30:16.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559769+0000", 2024-09-14T05:30:16.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6740000000000002e-06 2024-09-14T05:30:16.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559774+0000", 2024-09-14T05:30:16.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6110000000000004e-06 2024-09-14T05:30:16.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604905+0000", 2024-09-14T05:30:16.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045130827999999998 2024-09-14T05:30:16.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604982+0000", 2024-09-14T05:30:16.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7410999999999995e-05 2024-09-14T05:30:16.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.483 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605077+0000", 2024-09-14T05:30:16.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5449e-05 2024-09-14T05:30:16.483 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.483 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.483 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.483 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73793 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2836480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.483 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:16.483 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.898776967, 2024-09-14T05:30:16.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045441212000000002, 2024-09-14T05:30:16.483 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.483 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.483 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.483 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.484 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.484 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.484 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73793 2024-09-14T05:30:16.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.484 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:16.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:16.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559781+0000", 2024-09-14T05:30:16.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2899999999999999e-07 2024-09-14T05:30:16.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559783+0000", 2024-09-14T05:30:16.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.075e-06 2024-09-14T05:30:16.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.486 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559788+0000", 2024-09-14T05:30:16.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5369999999999997e-06 2024-09-14T05:30:16.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.486 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.486 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559793+0000", 2024-09-14T05:30:16.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:30:16.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.486 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.486 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605109+0000", 2024-09-14T05:30:16.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316229999999999 2024-09-14T05:30:16.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605168+0000", 2024-09-14T05:30:16.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8949000000000001e-05 2024-09-14T05:30:16.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605221+0000", 2024-09-14T05:30:16.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3260000000000002e-05 2024-09-14T05:30:16.487 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.487 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.487 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.487 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73806 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.487 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:16.488 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.898758341000001, 2024-09-14T05:30:16.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045526189000000002, 2024-09-14T05:30:16.488 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.488 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.488 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.488 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.488 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.488 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.488 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73806 2024-09-14T05:30:16.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.488 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:16.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:16.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:16.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:30:16.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559801+0000", 2024-09-14T05:30:16.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.995e-06 2024-09-14T05:30:16.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559807+0000", 2024-09-14T05:30:16.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5450000000000003e-06 2024-09-14T05:30:16.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559811+0000", 2024-09-14T05:30:16.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5090000000000004e-06 2024-09-14T05:30:16.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605237+0000", 2024-09-14T05:30:16.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045426133 2024-09-14T05:30:16.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605278+0000", 2024-09-14T05:30:16.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1084999999999997e-05 2024-09-14T05:30:16.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605325+0000", 2024-09-14T05:30:16.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6582999999999998e-05 2024-09-14T05:30:16.492 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.492 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.492 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.492 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73809 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2838528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.492 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:16.492 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.898734835999999, 2024-09-14T05:30:16.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045596062999999999, 2024-09-14T05:30:16.492 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.492 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.492 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.493 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.493 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.493 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.493 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73809 2024-09-14T05:30:16.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.493 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.493 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:16.493 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.493 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:16.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559823+0000", 2024-09-14T05:30:16.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.08e-07 2024-09-14T05:30:16.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559825+0000", 2024-09-14T05:30:16.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.26e-06 2024-09-14T05:30:16.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559831+0000", 2024-09-14T05:30:16.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6690000000000003e-06 2024-09-14T05:30:16.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559835+0000", 2024-09-14T05:30:16.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5120000000000002e-06 2024-09-14T05:30:16.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605342+0000", 2024-09-14T05:30:16.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045506957000000001 2024-09-14T05:30:16.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.496 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.496 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605375+0000", 2024-09-14T05:30:16.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3250999999999997e-05 2024-09-14T05:30:16.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.496 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.496 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605418+0000", 2024-09-14T05:30:16.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3006000000000001e-05 2024-09-14T05:30:16.497 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.497 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.497 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.497 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73822 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2807808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.497 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:16.497 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.898707182999999, 2024-09-14T05:30:16.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045661405000000002, 2024-09-14T05:30:16.497 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.497 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.497 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.497 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.497 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.498 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.498 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73822 2024-09-14T05:30:16.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.498 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:16.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:16.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559851+0000", 2024-09-14T05:30:16.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4000000000000002e-07 2024-09-14T05:30:16.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559853+0000", 2024-09-14T05:30:16.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5380000000000001e-06 2024-09-14T05:30:16.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559859+0000", 2024-09-14T05:30:16.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0619999999999997e-06 2024-09-14T05:30:16.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559864+0000", 2024-09-14T05:30:16.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.977e-06 2024-09-14T05:30:16.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605435+0000", 2024-09-14T05:30:16.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045570672 2024-09-14T05:30:16.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605467+0000", 2024-09-14T05:30:16.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2486e-05 2024-09-14T05:30:16.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.501 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.501 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605511+0000", 2024-09-14T05:30:16.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4129999999999999e-05 2024-09-14T05:30:16.501 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.501 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.501 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.501 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73825 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2840576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.501 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:16.501 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.898683818999999, 2024-09-14T05:30:16.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045728694, 2024-09-14T05:30:16.502 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.502 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.502 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.502 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.502 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.502 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.502 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73825 2024-09-14T05:30:16.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.502 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:16.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:16.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559874+0000", 2024-09-14T05:30:16.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5400000000000002e-07 2024-09-14T05:30:16.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559876+0000", 2024-09-14T05:30:16.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.131e-06 2024-09-14T05:30:16.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559888+0000", 2024-09-14T05:30:16.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2216e-05 2024-09-14T05:30:16.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559893+0000", 2024-09-14T05:30:16.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6539999999999998e-06 2024-09-14T05:30:16.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605526+0000", 2024-09-14T05:30:16.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045633119 2024-09-14T05:30:16.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605560+0000", 2024-09-14T05:30:16.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3825999999999998e-05 2024-09-14T05:30:16.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605602+0000", 2024-09-14T05:30:16.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2293999999999999e-05 2024-09-14T05:30:16.505 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.506 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.506 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.506 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74023 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2863104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.506 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:16.506 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.897497844, 2024-09-14T05:30:16.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044635385999999999, 2024-09-14T05:30:16.506 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.506 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.506 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.506 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.506 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.506 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.507 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74023 2024-09-14T05:30:16.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.507 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:16.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:16.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561060+0000", 2024-09-14T05:30:16.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0699999999999998e-07 2024-09-14T05:30:16.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561072+0000", 2024-09-14T05:30:16.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2261e-05 2024-09-14T05:30:16.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561079+0000", 2024-09-14T05:30:16.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8399999999999997e-06 2024-09-14T05:30:16.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561084+0000", 2024-09-14T05:30:16.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7380000000000002e-06 2024-09-14T05:30:16.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605620+0000", 2024-09-14T05:30:16.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044536146999999998 2024-09-14T05:30:16.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605653+0000", 2024-09-14T05:30:16.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2969000000000002e-05 2024-09-14T05:30:16.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605695+0000", 2024-09-14T05:30:16.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2024000000000003e-05 2024-09-14T05:30:16.510 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.510 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.510 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.510 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74026 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2895872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.510 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:16.510 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.897466382000001, 2024-09-14T05:30:16.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044692717, 2024-09-14T05:30:16.511 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.511 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.511 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.511 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.511 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.511 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.511 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74026 2024-09-14T05:30:16.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.511 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:16.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:16.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:16.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:16.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561093+0000", 2024-09-14T05:30:16.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-14T05:30:16.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561099+0000", 2024-09-14T05:30:16.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0399999999999998e-06 2024-09-14T05:30:16.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561104+0000", 2024-09-14T05:30:16.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8180000000000003e-06 2024-09-14T05:30:16.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605710+0000", 2024-09-14T05:30:16.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044605657 2024-09-14T05:30:16.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605742+0000", 2024-09-14T05:30:16.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1949000000000002e-05 2024-09-14T05:30:16.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605784+0000", 2024-09-14T05:30:16.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1884000000000001e-05 2024-09-14T05:30:16.514 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.514 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.514 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.515 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74039 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.515 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:16.515 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.897339784, 2024-09-14T05:30:16.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044654905000000002, 2024-09-14T05:30:16.515 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.515 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.515 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.515 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.515 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.515 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.515 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74039 2024-09-14T05:30:16.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.516 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:16.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:16.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561218+0000", 2024-09-14T05:30:16.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4000000000000001e-07 2024-09-14T05:30:16.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561221+0000", 2024-09-14T05:30:16.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9730000000000002e-06 2024-09-14T05:30:16.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561229+0000", 2024-09-14T05:30:16.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1759999999999993e-06 2024-09-14T05:30:16.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561235+0000", 2024-09-14T05:30:16.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0109999999999997e-06 2024-09-14T05:30:16.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605798+0000", 2024-09-14T05:30:16.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044563285000000001 2024-09-14T05:30:16.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605831+0000", 2024-09-14T05:30:16.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2054999999999997e-05 2024-09-14T05:30:16.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.519 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.519 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605872+0000", 2024-09-14T05:30:16.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1764999999999997e-05 2024-09-14T05:30:16.519 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.519 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.519 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.519 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74042 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.519 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:16.519 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.897291331000002, 2024-09-14T05:30:16.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044773234000000002, 2024-09-14T05:30:16.519 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.520 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.520 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.520 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.520 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.520 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.520 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74042 2024-09-14T05:30:16.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.520 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:16.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:16.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:16.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4099999999999998e-07 2024-09-14T05:30:16.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.522 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561269+0000", 2024-09-14T05:30:16.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7930000000000002e-06 2024-09-14T05:30:16.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.522 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561278+0000", 2024-09-14T05:30:16.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5150000000000006e-06 2024-09-14T05:30:16.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.522 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561284+0000", 2024-09-14T05:30:16.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2029999999999997e-06 2024-09-14T05:30:16.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.523 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605934+0000", 2024-09-14T05:30:16.523 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044650449000000002 2024-09-14T05:30:16.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.523 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605983+0000", 2024-09-14T05:30:16.523 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8943999999999998e-05 2024-09-14T05:30:16.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.523 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606039+0000", 2024-09-14T05:30:16.524 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5689000000000002e-05 2024-09-14T05:30:16.524 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.524 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.524 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.524 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74052 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.524 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:16.524 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.897264455999998, 2024-09-14T05:30:16.524 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044830474000000002, 2024-09-14T05:30:16.524 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.524 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.524 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.525 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.525 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.525 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.525 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74052 2024-09-14T05:30:16.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.525 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:16.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:16.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:16.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1099999999999996e-07 2024-09-14T05:30:16.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561295+0000", 2024-09-14T05:30:16.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7770000000000001e-06 2024-09-14T05:30:16.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561303+0000", 2024-09-14T05:30:16.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6860000000000006e-06 2024-09-14T05:30:16.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561308+0000", 2024-09-14T05:30:16.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.023e-06 2024-09-14T05:30:16.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605893+0000", 2024-09-14T05:30:16.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044585458000000001 2024-09-14T05:30:16.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605933+0000", 2024-09-14T05:30:16.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9551999999999997e-05 2024-09-14T05:30:16.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606123+0000", 2024-09-14T05:30:16.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019046699999999999 2024-09-14T05:30:16.528 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.529 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.529 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.529 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74054 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.529 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:16.529 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.897242428999999, 2024-09-14T05:30:16.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093610999999999, 2024-09-14T05:30:16.529 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.529 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.529 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.529 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.529 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.530 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.530 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74054 2024-09-14T05:30:16.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.530 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:16.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:16.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.531 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:16.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9699999999999996e-07 2024-09-14T05:30:16.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.531 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561317+0000", 2024-09-14T05:30:16.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9740000000000001e-06 2024-09-14T05:30:16.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.531 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561324+0000", 2024-09-14T05:30:16.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4590000000000003e-06 2024-09-14T05:30:16.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561329+0000", 2024-09-14T05:30:16.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5969999999999999e-06 2024-09-14T05:30:16.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606142+0000", 2024-09-14T05:30:16.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044812347000000002 2024-09-14T05:30:16.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606184+0000", 2024-09-14T05:30:16.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2441999999999998e-05 2024-09-14T05:30:16.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606408+0000", 2024-09-14T05:30:16.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022429500000000001 2024-09-14T05:30:16.533 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.533 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.533 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.533 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74056 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.534 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:16.534 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.897047726, 2024-09-14T05:30:16.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045191685000000002, 2024-09-14T05:30:16.534 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.534 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.534 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.534 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.534 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.534 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.534 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74056 2024-09-14T05:30:16.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.534 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:16.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:16.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561510+0000", 2024-09-14T05:30:16.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2399999999999998e-07 2024-09-14T05:30:16.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.536 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561513+0000", 2024-09-14T05:30:16.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7219999999999999e-06 2024-09-14T05:30:16.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.536 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561524+0000", 2024-09-14T05:30:16.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1357e-05 2024-09-14T05:30:16.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.536 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561530+0000", 2024-09-14T05:30:16.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2890000000000003e-06 2024-09-14T05:30:16.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.537 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606438+0000", 2024-09-14T05:30:16.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044907326999999997 2024-09-14T05:30:16.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.537 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606520+0000", 2024-09-14T05:30:16.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1924999999999999e-05 2024-09-14T05:30:16.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606701+0000", 2024-09-14T05:30:16.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00018144100000000001 2024-09-14T05:30:16.538 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.538 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.538 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.538 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74058 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.538 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:16.538 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.896569053, 2024-09-14T05:30:16.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093368000000002, 2024-09-14T05:30:16.539 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.539 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.539 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.539 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.539 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.539 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.539 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74058 2024-09-14T05:30:16.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.539 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:16.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:16.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561989+0000", 2024-09-14T05:30:16.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3300000000000001e-07 2024-09-14T05:30:16.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561993+0000", 2024-09-14T05:30:16.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1790000000000001e-06 2024-09-14T05:30:16.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562007+0000", 2024-09-14T05:30:16.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3434e-05 2024-09-14T05:30:16.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562014+0000", 2024-09-14T05:30:16.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.537e-06 2024-09-14T05:30:16.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606736+0000", 2024-09-14T05:30:16.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044722117999999998 2024-09-14T05:30:16.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606829+0000", 2024-09-14T05:30:16.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2410999999999993e-05 2024-09-14T05:30:16.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.543 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607082+0000", 2024-09-14T05:30:16.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025285599999999998 2024-09-14T05:30:16.543 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.543 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.543 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.543 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74059 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.543 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:16.543 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.896511061999998, 2024-09-14T05:30:16.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045479928000000003, 2024-09-14T05:30:16.543 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.543 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.543 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.544 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.544 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.544 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.544 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74059 2024-09-14T05:30:16.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.544 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:16.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:16.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562047+0000", 2024-09-14T05:30:16.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6899999999999997e-07 2024-09-14T05:30:16.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562050+0000", 2024-09-14T05:30:16.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.943e-06 2024-09-14T05:30:16.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562058+0000", 2024-09-14T05:30:16.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0120000000000005e-06 2024-09-14T05:30:16.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562063+0000", 2024-09-14T05:30:16.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:30:16.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607186+0000", 2024-09-14T05:30:16.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045122348999999999 2024-09-14T05:30:16.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607265+0000", 2024-09-14T05:30:16.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9462999999999995e-05 2024-09-14T05:30:16.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607526+0000", 2024-09-14T05:30:16.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026090400000000001 2024-09-14T05:30:16.547 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.547 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.547 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.548 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74060 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2850816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:16.548 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:16.548 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 26.896485685000002, 2024-09-14T05:30:16.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045200092999999997, 2024-09-14T05:30:16.548 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.548 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.548 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.548 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.548 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.548 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.548 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74060 2024-09-14T05:30:16.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.549 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:16.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:16.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:16.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5200000000000002e-07 2024-09-14T05:30:16.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562075+0000", 2024-09-14T05:30:16.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5200000000000002e-06 2024-09-14T05:30:16.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562082+0000", 2024-09-14T05:30:16.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8929999999999999e-06 2024-09-14T05:30:16.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562088+0000", 2024-09-14T05:30:16.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5570000000000003e-06 2024-09-14T05:30:16.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607111+0000", 2024-09-14T05:30:16.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045023464999999999 2024-09-14T05:30:16.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607190+0000", 2024-09-14T05:30:16.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8721999999999996e-05 2024-09-14T05:30:16.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607272+0000", 2024-09-14T05:30:16.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1483999999999994e-05 2024-09-14T05:30:16.552 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.552 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.552 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.552 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.552 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.552 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:16.554 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:30:16.554 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:16.552+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:16.576 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-14T05:30:16.599 INFO:teuthology.orchestra.run.smithi071.stdout:4805 writesame 0x7432909 thru 0x74379a4 data_size 0x43(0x509c bytes) 2024-09-14T05:30:16.607 INFO:teuthology.orchestra.run.smithi071.stdout:4806 write 0xc77935b thru 0xc77ca84 (0x372a bytes) 2024-09-14T05:30:16.611 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:16.611 INFO:teuthology.orchestra.run.smithi071.stdout:4810 punch from 0x3e0ca04 to 0x3e0fa95, (0x3091 bytes) 2024-09-14T05:30:16.611 INFO:teuthology.orchestra.run.smithi071.stdout:4811 writesame 0x1e61f7d thru 0x1e68ed0 data_size 0x6(0x6f54 bytes) 2024-09-14T05:30:16.620 INFO:teuthology.orchestra.run.smithi071.stdout:4812 read 0xd84f23c thru 0xd856c99 (0x7a5e bytes) 2024-09-14T05:30:16.626 INFO:teuthology.orchestra.run.smithi071.stdout:4814 trunc from 0xe7ab7d1 to 0xac806d8 2024-09-14T05:30:16.641 INFO:teuthology.orchestra.run.smithi071.stdout:4815 punch from 0x76e96b8 to 0x76e9d63, (0x6ab bytes) 2024-09-14T05:30:16.641 INFO:teuthology.orchestra.run.smithi071.stdout:4816 trunc from 0xac806d8 to 0xa420b2f 2024-09-14T05:30:16.654 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:16.653+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:16.656 INFO:teuthology.orchestra.run.smithi071.stdout:4817 writesame 0xc52aad2 thru 0xc52d714 data_size 0x3(0x2c43 bytes) 2024-09-14T05:30:16.657 INFO:teuthology.orchestra.run.smithi071.stdout:4818 punch from 0x580b1eb to 0x5810849, (0x565e bytes) 2024-09-14T05:30:16.657 INFO:teuthology.orchestra.run.smithi071.stdout:4819 write 0x75e79c5 thru 0x75f2d25 (0xb361 bytes) 2024-09-14T05:30:16.661 INFO:teuthology.orchestra.run.smithi071.stdout:4820 read 0xaacaeb thru 0xab01f5 (0x370b bytes) 2024-09-14T05:30:16.668 INFO:teuthology.orchestra.run.smithi071.stdout:4821 writesame 0xd44e820 thru 0xd45de45 data_size 0xe(0xf626 bytes) 2024-09-14T05:30:16.669 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:16.669 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:16.673 INFO:teuthology.orchestra.run.smithi071.stdout:4826 trunc from 0xd45de46 to 0xa58bdea 2024-09-14T05:30:16.687 INFO:teuthology.orchestra.run.smithi071.stdout:4827 trunc from 0xa58bdea to 0x40d45fb 2024-09-14T05:30:16.698 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:16.698 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:16.699 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.699 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:120899 3.12 3:48bcfa63:::rbd_object_map.112d5502ed06:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-14T05:30:16.699 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:16.694845+0000", 2024-09-14T05:30:16.699 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.003477342, 2024-09-14T05:30:16.699 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035020210000000001, 2024-09-14T05:30:16.699 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.699 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.699 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "waiting for sub ops", 2024-09-14T05:30:16.699 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.699 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.699 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.699 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 120899 2024-09-14T05:30:16.699 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.700 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.700 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.700 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.694845+0000", 2024-09-14T05:30:16.700 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.700 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.700 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.694845+0000", 2024-09-14T05:30:16.700 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.700 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.701 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.694845+0000", 2024-09-14T05:30:16.701 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8700000000000005e-07 2024-09-14T05:30:16.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.701 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.701 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.701 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.694849+0000", 2024-09-14T05:30:16.701 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3299999999999999e-06 2024-09-14T05:30:16.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.701 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.701 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.701 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.694854+0000", 2024-09-14T05:30:16.701 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.891e-06 2024-09-14T05:30:16.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.701 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.702 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.702 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.694859+0000", 2024-09-14T05:30:16.702 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0200000000000002e-06 2024-09-14T05:30:16.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.702 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.702 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.696202+0000", 2024-09-14T05:30:16.702 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001342219 2024-09-14T05:30:16.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.702 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.702 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.696286+0000", 2024-09-14T05:30:16.702 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4738000000000001e-05 2024-09-14T05:30:16.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.703 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 6", 2024-09-14T05:30:16.703 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.696678+0000", 2024-09-14T05:30:16.703 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00039147200000000002 2024-09-14T05:30:16.703 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.703 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.703 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.703 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.703 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.703 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:120900 3.12 3:48bcfa63:::rbd_object_map.112d5502ed06:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-14T05:30:16.703 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:16.694910+0000", 2024-09-14T05:30:16.703 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0034116250000000002, 2024-09-14T05:30:16.703 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0035088530000000001, 2024-09-14T05:30:16.703 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.704 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.704 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "waiting for sub ops", 2024-09-14T05:30:16.704 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.704 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.704 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.704 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 120900 2024-09-14T05:30:16.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.704 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.704 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.704 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.694910+0000", 2024-09-14T05:30:16.704 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.705 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.705 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.694910+0000", 2024-09-14T05:30:16.705 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.705 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.705 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.694911+0000", 2024-09-14T05:30:16.705 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4099999999999998e-07 2024-09-14T05:30:16.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.706 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.706 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.694914+0000", 2024-09-14T05:30:16.706 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.4400000000000001e-06 2024-09-14T05:30:16.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.706 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.706 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.694920+0000", 2024-09-14T05:30:16.706 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8379999999999998e-06 2024-09-14T05:30:16.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.706 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.707 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.694924+0000", 2024-09-14T05:30:16.707 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9519999999999996e-06 2024-09-14T05:30:16.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.707 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.707 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.697470+0000", 2024-09-14T05:30:16.707 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.002546118 2024-09-14T05:30:16.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.707 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.707 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.697535+0000", 2024-09-14T05:30:16.707 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5034000000000001e-05 2024-09-14T05:30:16.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.707 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 6", 2024-09-14T05:30:16.708 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.697920+0000", 2024-09-14T05:30:16.708 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00038480799999999999 2024-09-14T05:30:16.708 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.708 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.708 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.708 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop_reply(client.4397.0:120899 3.12 e70/39 ondisk, result = 0)", 2024-09-14T05:30:16.708 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:16.697367+0000", 2024-09-14T05:30:16.708 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.00095505600000000003, 2024-09-14T05:30:16.708 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001107568, 2024-09-14T05:30:16.708 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.708 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.708 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.709 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.697367+0000", 2024-09-14T05:30:16.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.697367+0000", 2024-09-14T05:30:16.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.710 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.710 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.697367+0000", 2024-09-14T05:30:16.710 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1200000000000003e-07 2024-09-14T05:30:16.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.710 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.710 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.697370+0000", 2024-09-14T05:30:16.710 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0900000000000001e-06 2024-09-14T05:30:16.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.710 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.710 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.697377+0000", 2024-09-14T05:30:16.710 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5010000000000001e-06 2024-09-14T05:30:16.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.711 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.711 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.697384+0000", 2024-09-14T05:30:16.711 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9280000000000004e-06 2024-09-14T05:30:16.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.711 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.711 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.698441+0000", 2024-09-14T05:30:16.711 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.001057311 2024-09-14T05:30:16.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.711 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.711 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:16.698449+0000", 2024-09-14T05:30:16.711 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8339999999999999e-06 2024-09-14T05:30:16.712 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.712 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.712 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.712 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.712 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:30:16.712 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 3 2024-09-14T05:30:16.712 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:16.713 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:30:16.714 INFO:teuthology.orchestra.run.smithi071.stdout:4828 writesame 0x661f687 thru 0x6621ea2 data_size 0x2(0x281c bytes) 2024-09-14T05:30:16.718 INFO:teuthology.orchestra.run.smithi071.stdout:4829 write 0x668162c thru 0x6690ded (0xf7c2 bytes) 2024-09-14T05:30:16.728 INFO:teuthology.orchestra.run.smithi071.stdout:4830 trunc from 0x6690dee to 0xca52fb9 2024-09-14T05:30:16.735 INFO:teuthology.orchestra.run.smithi071.stdout:4831 read 0x94777e thru 0x954b72 (0xd3f5 bytes) 2024-09-14T05:30:16.736 INFO:teuthology.orchestra.run.smithi071.stdout:4833 punch from 0x6456184 to 0x645ec16, (0x8a92 bytes) 2024-09-14T05:30:16.736 INFO:teuthology.orchestra.run.smithi071.stdout:4834 read 0x5fc80cf thru 0x5fca2cd (0x21ff bytes) 2024-09-14T05:30:16.749 INFO:teuthology.orchestra.run.smithi071.stdout:4836 trunc from 0xca52fb9 to 0x17c7ae5 2024-09-14T05:30:16.755 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:16.754+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:16.769 INFO:teuthology.orchestra.run.smithi071.stdout:4839 write 0x48cfeb5 thru 0x48dd4ec (0xd638 bytes) 2024-09-14T05:30:16.770 INFO:teuthology.orchestra.run.smithi071.stdout:4840 write 0x576f53 thru 0x585658 (0xe706 bytes) 2024-09-14T05:30:16.772 INFO:teuthology.orchestra.run.smithi071.stdout:4841 read 0x1a4abde thru 0x1a54f1c (0xa33f bytes) 2024-09-14T05:30:16.789 INFO:teuthology.orchestra.run.smithi071.stdout:4842 write 0xe3079a6 thru 0xe30f748 (0x7da3 bytes) 2024-09-14T05:30:16.793 INFO:teuthology.orchestra.run.smithi071.stdout:4843 punch from 0x676a8d4 to 0x67740dc, (0x9808 bytes) 2024-09-14T05:30:16.793 INFO:teuthology.orchestra.run.smithi071.stdout:4845 write 0xce30f10 thru 0xce39f78 (0x9069 bytes) 2024-09-14T05:30:16.795 INFO:teuthology.orchestra.run.smithi071.stdout:4847 read 0xcf9a61b thru 0xcf9efb6 (0x499c bytes) 2024-09-14T05:30:16.796 INFO:teuthology.orchestra.run.smithi071.stdout:4849 write 0xc1ee605 thru 0xc1f3091 (0x4a8d bytes) 2024-09-14T05:30:16.799 INFO:teuthology.orchestra.run.smithi071.stdout:4851 read 0x934157c thru 0x9347053 (0x5ad8 bytes) 2024-09-14T05:30:16.799 INFO:teuthology.orchestra.run.smithi071.stdout:4852 write 0x2d68d77 thru 0x2d6a1ac (0x1436 bytes) 2024-09-14T05:30:16.814 INFO:teuthology.orchestra.run.smithi071.stdout:4853 trunc from 0xe30f749 to 0x27fce44 2024-09-14T05:30:16.832 INFO:teuthology.orchestra.run.smithi071.stdout:4854 writesame 0x3199b9a thru 0x31a28be data_size 0x1(0x8d25 bytes) 2024-09-14T05:30:16.832 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:16.832 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:16.833 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:16.833 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:16.833 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:16.833 INFO:teuthology.orchestra.run.smithi071.stdout:4855 read 0x1c774fe thru 0x1c7e874 (0x7377 bytes) 2024-09-14T05:30:16.833 INFO:teuthology.orchestra.run.smithi071.stdout:4857 read 0x252a2a thru 0x2624bd (0xfa94 bytes) 2024-09-14T05:30:16.834 INFO:teuthology.orchestra.run.smithi071.stdout:4858 read 0x2ed8813 thru 0x2edfce6 (0x74d4 bytes) 2024-09-14T05:30:16.834 INFO:teuthology.orchestra.run.smithi071.stdout:4860 read 0x263e492 thru 0x2640ca9 (0x2818 bytes) 2024-09-14T05:30:16.835 INFO:teuthology.orchestra.run.smithi071.stdout:4863 punch from 0x2ab492f to 0x2abdf9c, (0x966d bytes) 2024-09-14T05:30:16.835 INFO:teuthology.orchestra.run.smithi071.stdout:4864 punch from 0x1b9fe6d to 0x1ba0743, (0x8d6 bytes) 2024-09-14T05:30:16.846 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:30:16.851 INFO:teuthology.orchestra.run.smithi071.stdout:4865 write 0xbd3bb72 thru 0xbd43ec6 (0x8355 bytes) 2024-09-14T05:30:16.853 INFO:teuthology.orchestra.run.smithi071.stdout:4866 punch from 0x3a5ce87 to 0x3a5fac6, (0x2c3f bytes) 2024-09-14T05:30:16.854 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:16.856 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:16.855+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:16.860 INFO:teuthology.orchestra.run.smithi071.stdout:4870 write 0xd7c4059 thru 0xd7c8038 (0x3fe0 bytes) 2024-09-14T05:30:16.863 INFO:teuthology.orchestra.run.smithi071.stdout:4871 write 0x2c4c7f thru 0x2d4698 (0xfa1a bytes) 2024-09-14T05:30:16.865 INFO:teuthology.orchestra.run.smithi071.stdout:4873 punch from 0x1370d88 to 0x13739ae, (0x2c26 bytes) 2024-09-14T05:30:16.871 INFO:teuthology.orchestra.run.smithi071.stdout:4874 write 0xe8a7415 thru 0xe8b259c (0xb188 bytes) 2024-09-14T05:30:16.874 INFO:teuthology.orchestra.run.smithi071.stdout:4875 read 0x7d5abde thru 0x7d69c37 (0xf05a bytes) 2024-09-14T05:30:16.874 INFO:teuthology.orchestra.run.smithi071.stdout:4876 read 0x341906b thru 0x3427a99 (0xea2f bytes) 2024-09-14T05:30:16.891 INFO:teuthology.orchestra.run.smithi071.stdout:4877 trunc from 0xe8b259d to 0xa1cd1f 2024-09-14T05:30:16.902 INFO:teuthology.orchestra.run.smithi071.stdout:4878 punch from 0x80e960 to 0x81974a, (0xadea bytes) 2024-09-14T05:30:16.903 INFO:teuthology.orchestra.run.smithi071.stdout:4879 write 0x92179f thru 0x930199 (0xe9fb bytes) 2024-09-14T05:30:16.923 INFO:teuthology.orchestra.run.smithi071.stdout:4880 write 0xca08d0d thru 0xca12ed5 (0xa1c9 bytes) 2024-09-14T05:30:16.926 INFO:teuthology.orchestra.run.smithi071.stdout:4882 write 0x868fe87 thru 0x869202e (0x21a8 bytes) 2024-09-14T05:30:16.929 INFO:teuthology.orchestra.run.smithi071.stdout:4885 write 0x766091d thru 0x766349f (0x2b83 bytes) 2024-09-14T05:30:16.938 INFO:teuthology.orchestra.run.smithi071.stdout:4886 trunc from 0xca12ed6 to 0x7fbbde9 2024-09-14T05:30:16.951 INFO:teuthology.orchestra.run.smithi071.stdout:4887 punch from 0x3de4b7a to 0x3deb024, (0x64aa bytes) 2024-09-14T05:30:16.951 INFO:teuthology.orchestra.run.smithi071.stdout:4888 writesame 0x3827a4f thru 0x3834314 data_size 0x1f(0xc8c6 bytes) 2024-09-14T05:30:16.952 INFO:teuthology.orchestra.run.smithi071.stdout:4889 writesame 0xb7d869 thru 0xb866a6 data_size 0x22(0x8e3e bytes) 2024-09-14T05:30:16.954 INFO:teuthology.orchestra.run.smithi071.stdout:4890 read 0x1ab37f9 thru 0x1ac2eb1 (0xf6b9 bytes) 2024-09-14T05:30:16.954 INFO:teuthology.orchestra.run.smithi071.stdout:4892 read 0x7e77e1 thru 0x7f3f5a (0xc77a bytes) 2024-09-14T05:30:16.955 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:16.955 INFO:teuthology.orchestra.run.smithi071.stdout:4894 read 0x7b8785e thru 0x7b8c588 (0x4d2b bytes) 2024-09-14T05:30:16.955 INFO:teuthology.orchestra.run.smithi071.stdout:4895 writesame 0x20c485f thru 0x20d1925 data_size 0x1d(0xd0c7 bytes) 2024-09-14T05:30:16.956 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:16.956+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:16.957 INFO:teuthology.orchestra.run.smithi071.stdout:4896 write 0x357a7ad thru 0x358a7a5 (0xfff9 bytes) 2024-09-14T05:30:16.959 INFO:teuthology.orchestra.run.smithi071.stdout:4897 write 0x2f84c1a thru 0x2f8cb8f (0x7f76 bytes) 2024-09-14T05:30:16.960 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:16.960 INFO:teuthology.orchestra.run.smithi071.stdout:4899 write 0x573b2f7 thru 0x5746302 (0xb00c bytes) 2024-09-14T05:30:16.970 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:16.970 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:16.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:16.970 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:16.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.970 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4235.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-14T05:30:16.970 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:30:16.970 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.128453131000001, 2024-09-14T05:30:16.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12887758399999999, 2024-09-14T05:30:16.970 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.970 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.971 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:30:16.971 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.971 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4235", 2024-09-14T05:30:16.971 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/3211559243", 2024-09-14T05:30:16.971 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 2 2024-09-14T05:30:16.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.971 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:30:16.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:30:16.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839833+0000", 2024-09-14T05:30:16.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:30:16.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839838+0000", 2024-09-14T05:30:16.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2440000000000001e-06 2024-09-14T05:30:16.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839848+0000", 2024-09-14T05:30:16.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0030000000000001e-05 2024-09-14T05:30:16.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839854+0000", 2024-09-14T05:30:16.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3750000000000002e-06 2024-09-14T05:30:16.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839908+0000", 2024-09-14T05:30:16.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3965999999999998e-05 2024-09-14T05:30:16.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:30:16.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839914+0000", 2024-09-14T05:30:16.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6139999999999997e-06 2024-09-14T05:30:16.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968558+0000", 2024-09-14T05:30:16.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12864361199999999 2024-09-14T05:30:16.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968710+0000", 2024-09-14T05:30:16.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015177300000000001 2024-09-14T05:30:16.975 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.975 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.975 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.975 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27764 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 188416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:16.975 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:30:16.975 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.757126749000001, 2024-09-14T05:30:16.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080826920999999996, 2024-09-14T05:30:16.975 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.975 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.975 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.976 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.976 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.976 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.976 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27764 2024-09-14T05:30:16.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.976 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:30:16.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:30:16.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211159+0000", 2024-09-14T05:30:16.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1800000000000001e-07 2024-09-14T05:30:16.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211161+0000", 2024-09-14T05:30:16.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.968e-06 2024-09-14T05:30:16.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211169+0000", 2024-09-14T05:30:16.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4800000000000001e-06 2024-09-14T05:30:16.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211174+0000", 2024-09-14T05:30:16.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1880000000000001e-06 2024-09-14T05:30:16.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291670+0000", 2024-09-14T05:30:16.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080495199000000003 2024-09-14T05:30:16.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291775+0000", 2024-09-14T05:30:16.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000105375 2024-09-14T05:30:16.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291985+0000", 2024-09-14T05:30:16.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021029299999999999 2024-09-14T05:30:16.979 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.979 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.979 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.980 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27765 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:16.980 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:30:16.980 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.757103999000002, 2024-09-14T05:30:16.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081115197999999999, 2024-09-14T05:30:16.980 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.980 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.980 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.980 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.980 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.980 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.980 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27765 2024-09-14T05:30:16.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.981 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:30:16.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:30:16.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211182+0000", 2024-09-14T05:30:16.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3599999999999999e-07 2024-09-14T05:30:16.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211184+0000", 2024-09-14T05:30:16.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.03e-06 2024-09-14T05:30:16.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211192+0000", 2024-09-14T05:30:16.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2309999999999996e-06 2024-09-14T05:30:16.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211197+0000", 2024-09-14T05:30:16.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4199999999999998e-06 2024-09-14T05:30:16.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292017+0000", 2024-09-14T05:30:16.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080819516999999994 2024-09-14T05:30:16.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292087+0000", 2024-09-14T05:30:16.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0056999999999997e-05 2024-09-14T05:30:16.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292296+0000", 2024-09-14T05:30:16.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000209507 2024-09-14T05:30:16.985 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.985 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.985 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.985 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27766 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:16.985 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:30:16.985 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.757080784999999, 2024-09-14T05:30:16.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081359077000000002, 2024-09-14T05:30:16.986 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.986 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.986 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.986 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.986 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.986 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.986 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27766 2024-09-14T05:30:16.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.986 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:30:16.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:30:16.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211205+0000", 2024-09-14T05:30:16.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7e-07 2024-09-14T05:30:16.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211207+0000", 2024-09-14T05:30:16.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4540000000000001e-06 2024-09-14T05:30:16.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211216+0000", 2024-09-14T05:30:16.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3680000000000002e-06 2024-09-14T05:30:16.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211221+0000", 2024-09-14T05:30:16.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3499999999999996e-06 2024-09-14T05:30:16.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292329+0000", 2024-09-14T05:30:16.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081108240999999998 2024-09-14T05:30:16.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292370+0000", 2024-09-14T05:30:16.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.121e-05 2024-09-14T05:30:16.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292563+0000", 2024-09-14T05:30:16.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019298399999999999 2024-09-14T05:30:16.990 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.990 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.990 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.990 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27767 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:16.990 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:16.990 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.757057271000001, 2024-09-14T05:30:16.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081623119999999993, 2024-09-14T05:30:16.990 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.990 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.990 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.990 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.991 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.991 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.991 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27767 2024-09-14T05:30:16.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.991 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:16.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:16.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:16.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0499999999999999e-07 2024-09-14T05:30:16.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211230+0000", 2024-09-14T05:30:16.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0700000000000001e-06 2024-09-14T05:30:16.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211236+0000", 2024-09-14T05:30:16.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8429999999999998e-06 2024-09-14T05:30:16.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211241+0000", 2024-09-14T05:30:16.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:30:16.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292594+0000", 2024-09-14T05:30:16.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081352868999999994 2024-09-14T05:30:16.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292631+0000", 2024-09-14T05:30:16.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7083999999999999e-05 2024-09-14T05:30:16.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292851+0000", 2024-09-14T05:30:16.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022011700000000001 2024-09-14T05:30:16.994 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.994 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.994 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.995 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27768 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 274432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:16.995 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:30:16.995 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.757038734999998, 2024-09-14T05:30:16.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081875032, 2024-09-14T05:30:16.995 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.995 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:16.995 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:16.995 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:16.995 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:16.995 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:16.995 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27768 2024-09-14T05:30:16.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.995 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:16.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:16.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:30:16.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:16.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:30:16.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:16.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:16.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211247+0000", 2024-09-14T05:30:16.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.46e-07 2024-09-14T05:30:16.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:16.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211248+0000", 2024-09-14T05:30:16.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7349999999999999e-06 2024-09-14T05:30:16.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:16.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211253+0000", 2024-09-14T05:30:16.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.899e-06 2024-09-14T05:30:16.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:16.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211258+0000", 2024-09-14T05:30:16.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2810000000000002e-06 2024-09-14T05:30:16.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:16.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292888+0000", 2024-09-14T05:30:16.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081629955000000004 2024-09-14T05:30:16.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:16.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292926+0000", 2024-09-14T05:30:16.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7947000000000001e-05 2024-09-14T05:30:16.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:16.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293121+0000", 2024-09-14T05:30:16.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019586899999999999 2024-09-14T05:30:16.999 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.999 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:16.999 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:16.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:16.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:16.999 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27769 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 294912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:16.999 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:30:16.999 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.757020646000001, 2024-09-14T05:30:16.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082181899000000003, 2024-09-14T05:30:16.999 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:16.999 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:17.000 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:17.000 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:17.000 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:17.000 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.000 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27769 2024-09-14T05:30:17.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.000 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:17.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:17.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:30:17.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:17.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:30:17.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:17.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211265+0000", 2024-09-14T05:30:17.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2199999999999999e-07 2024-09-14T05:30:17.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:17.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211267+0000", 2024-09-14T05:30:17.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2239999999999998e-06 2024-09-14T05:30:17.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:17.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211273+0000", 2024-09-14T05:30:17.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6540000000000004e-06 2024-09-14T05:30:17.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211277+0000", 2024-09-14T05:30:17.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6299999999999997e-06 2024-09-14T05:30:17.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:17.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293150+0000", 2024-09-14T05:30:17.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081872575000000003 2024-09-14T05:30:17.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:17.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293245+0000", 2024-09-14T05:30:17.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5327000000000004e-05 2024-09-14T05:30:17.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:17.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293446+0000", 2024-09-14T05:30:17.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000201067 2024-09-14T05:30:17.003 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.003 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:17.003 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.004 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27770 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:17.004 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:17.004 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.757000483999999, 2024-09-14T05:30:17.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082663707000000003, 2024-09-14T05:30:17.004 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:17.004 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:17.004 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:17.004 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:17.004 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:17.004 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.004 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27770 2024-09-14T05:30:17.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.005 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:17.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:17.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:17.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:17.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:17.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:17.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:17.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1199999999999998e-07 2024-09-14T05:30:17.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:17.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211292+0000", 2024-09-14T05:30:17.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1609999999999997e-06 2024-09-14T05:30:17.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:17.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211299+0000", 2024-09-14T05:30:17.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3300000000000004e-06 2024-09-14T05:30:17.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211303+0000", 2024-09-14T05:30:17.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7859999999999996e-06 2024-09-14T05:30:17.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:17.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293628+0000", 2024-09-14T05:30:17.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082324283999999998 2024-09-14T05:30:17.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:17.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293694+0000", 2024-09-14T05:30:17.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6277000000000003e-05 2024-09-14T05:30:17.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:17.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293948+0000", 2024-09-14T05:30:17.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025445699999999998 2024-09-14T05:30:17.008 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.008 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:17.008 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.008 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27790 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:17.008 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:30:17.008 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.756974079999999, 2024-09-14T05:30:17.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082461006000000003, 2024-09-14T05:30:17.009 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:17.009 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:17.009 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:17.009 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:17.009 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:17.009 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.009 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27790 2024-09-14T05:30:17.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.009 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:17.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:17.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:30:17.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:17.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:30:17.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:17.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211312+0000", 2024-09-14T05:30:17.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9299999999999998e-07 2024-09-14T05:30:17.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:17.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211314+0000", 2024-09-14T05:30:17.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000002e-06 2024-09-14T05:30:17.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:17.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211320+0000", 2024-09-14T05:30:17.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1589999999999998e-06 2024-09-14T05:30:17.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211325+0000", 2024-09-14T05:30:17.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9609999999999996e-06 2024-09-14T05:30:17.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:17.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293474+0000", 2024-09-14T05:30:17.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082149080999999999 2024-09-14T05:30:17.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:17.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293542+0000", 2024-09-14T05:30:17.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8497000000000005e-05 2024-09-14T05:30:17.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:17.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293772+0000", 2024-09-14T05:30:17.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022969900000000001 2024-09-14T05:30:17.012 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.012 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:17.012 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.013 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27791 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:17.013 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:30:17.013 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.756953765999999, 2024-09-14T05:30:17.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082700838999999998, 2024-09-14T05:30:17.013 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:17.013 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:17.013 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:17.013 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:17.013 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:17.013 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.013 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27791 2024-09-14T05:30:17.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.014 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:17.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:17.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:30:17.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:17.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:30:17.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:17.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211332+0000", 2024-09-14T05:30:17.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7399999999999999e-07 2024-09-14T05:30:17.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:17.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211334+0000", 2024-09-14T05:30:17.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8500000000000001e-06 2024-09-14T05:30:17.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:17.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211340+0000", 2024-09-14T05:30:17.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3010000000000003e-06 2024-09-14T05:30:17.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211345+0000", 2024-09-14T05:30:17.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.938e-06 2024-09-14T05:30:17.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:17.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293797+0000", 2024-09-14T05:30:17.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082452152000000001 2024-09-14T05:30:17.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:17.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293869+0000", 2024-09-14T05:30:17.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2236000000000004e-05 2024-09-14T05:30:17.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:17.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294032+0000", 2024-09-14T05:30:17.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016298800000000001 2024-09-14T05:30:17.017 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.017 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:17.017 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.017 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27792 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:17.017 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:17.017 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.756934234999999, 2024-09-14T05:30:17.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082897304000000005, 2024-09-14T05:30:17.018 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:17.018 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:17.018 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:17.018 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:17.018 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:17.018 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.018 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27792 2024-09-14T05:30:17.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.018 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:17.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:17.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:17.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:17.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:17.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:17.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:17.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2000000000000001e-07 2024-09-14T05:30:17.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:17.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211353+0000", 2024-09-14T05:30:17.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.761e-06 2024-09-14T05:30:17.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:17.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211359+0000", 2024-09-14T05:30:17.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2430000000000002e-06 2024-09-14T05:30:17.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211364+0000", 2024-09-14T05:30:17.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0239999999999997e-06 2024-09-14T05:30:17.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:17.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294059+0000", 2024-09-14T05:30:17.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082694834999999994 2024-09-14T05:30:17.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:17.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294114+0000", 2024-09-14T05:30:17.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4823e-05 2024-09-14T05:30:17.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:17.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294248+0000", 2024-09-14T05:30:17.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013429800000000001 2024-09-14T05:30:17.021 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.022 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:17.022 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.022 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27793 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:17.022 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:17.022 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.756914435999999, 2024-09-14T05:30:17.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083153362999999994, 2024-09-14T05:30:17.022 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:17.022 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:17.022 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:17.022 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:17.022 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:17.022 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.023 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27793 2024-09-14T05:30:17.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.023 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:17.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:17.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:17.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:17.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:17.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:17.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:17.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5100000000000001e-07 2024-09-14T05:30:17.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:17.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211373+0000", 2024-09-14T05:30:17.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.001e-06 2024-09-14T05:30:17.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:17.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211379+0000", 2024-09-14T05:30:17.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4559999999999997e-06 2024-09-14T05:30:17.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211384+0000", 2024-09-14T05:30:17.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9749999999999998e-06 2024-09-14T05:30:17.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:17.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294279+0000", 2024-09-14T05:30:17.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082894282999999999 2024-09-14T05:30:17.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:17.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294327+0000", 2024-09-14T05:30:17.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8208000000000003e-05 2024-09-14T05:30:17.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:17.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294524+0000", 2024-09-14T05:30:17.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019708900000000001 2024-09-14T05:30:17.026 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.026 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:17.026 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.026 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27794 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:17.026 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:17.026 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.756894112000001, 2024-09-14T05:30:17.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588590000000004, 2024-09-14T05:30:17.027 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:17.027 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:17.027 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:17.027 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:17.027 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:17.027 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.027 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27794 2024-09-14T05:30:17.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.027 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:17.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:17.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:17.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:17.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:17.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:17.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:17.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4499999999999997e-07 2024-09-14T05:30:17.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:17.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211393+0000", 2024-09-14T05:30:17.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.815e-06 2024-09-14T05:30:17.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:17.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211400+0000", 2024-09-14T05:30:17.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2899999999999999e-06 2024-09-14T05:30:17.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211404+0000", 2024-09-14T05:30:17.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:30:17.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:17.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294560+0000", 2024-09-14T05:30:17.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083155534000000003 2024-09-14T05:30:17.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:17.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294631+0000", 2024-09-14T05:30:17.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1378999999999995e-05 2024-09-14T05:30:17.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:17.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294980+0000", 2024-09-14T05:30:17.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00034844900000000002 2024-09-14T05:30:17.031 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.031 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:17.031 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.031 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27795 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:17.031 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:30:17.031 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.756874754000002, 2024-09-14T05:30:17.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083970367000000004, 2024-09-14T05:30:17.031 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:17.031 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:17.031 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:17.031 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:17.031 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:17.032 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.032 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27795 2024-09-14T05:30:17.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.032 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:17.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:17.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:30:17.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:17.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:30:17.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:17.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211411+0000", 2024-09-14T05:30:17.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5499999999999999e-07 2024-09-14T05:30:17.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:17.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211413+0000", 2024-09-14T05:30:17.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:30:17.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:17.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211419+0000", 2024-09-14T05:30:17.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1720000000000004e-06 2024-09-14T05:30:17.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211424+0000", 2024-09-14T05:30:17.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0690000000000001e-06 2024-09-14T05:30:17.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:17.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295013+0000", 2024-09-14T05:30:17.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588898999999994 2024-09-14T05:30:17.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:17.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295102+0000", 2024-09-14T05:30:17.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9462000000000005e-05 2024-09-14T05:30:17.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:17.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295381+0000", 2024-09-14T05:30:17.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000278566 2024-09-14T05:30:17.035 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.035 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:17.035 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.035 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27796 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:17.036 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:17.036 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.756855025, 2024-09-14T05:30:17.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084379107999999994, 2024-09-14T05:30:17.036 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:17.036 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:17.036 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:17.036 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:17.036 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:17.036 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.036 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27796 2024-09-14T05:30:17.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.036 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:17.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:17.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:17.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:17.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:17.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:17.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:17.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2800000000000002e-07 2024-09-14T05:30:17.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:17.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211432+0000", 2024-09-14T05:30:17.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:30:17.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:17.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211439+0000", 2024-09-14T05:30:17.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5239999999999997e-06 2024-09-14T05:30:17.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211444+0000", 2024-09-14T05:30:17.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.95e-06 2024-09-14T05:30:17.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:17.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295602+0000", 2024-09-14T05:30:17.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084158017000000002 2024-09-14T05:30:17.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:17.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295674+0000", 2024-09-14T05:30:17.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2385999999999994e-05 2024-09-14T05:30:17.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:17.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295809+0000", 2024-09-14T05:30:17.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013495900000000001 2024-09-14T05:30:17.040 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.040 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:17.040 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.040 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27797 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 266240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:17.040 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:17.040 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.756835042999999, 2024-09-14T05:30:17.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084898597000000006, 2024-09-14T05:30:17.040 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:17.040 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:17.040 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:17.040 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:17.040 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:17.041 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.041 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27797 2024-09-14T05:30:17.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.041 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:17.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:17.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:17.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:17.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:17.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:17.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:17.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:30:17.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:17.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211452+0000", 2024-09-14T05:30:17.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.708e-06 2024-09-14T05:30:17.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:17.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211458+0000", 2024-09-14T05:30:17.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3489999999999997e-06 2024-09-14T05:30:17.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211464+0000", 2024-09-14T05:30:17.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3820000000000003e-06 2024-09-14T05:30:17.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:17.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295839+0000", 2024-09-14T05:30:17.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084375055000000004 2024-09-14T05:30:17.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:17.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295918+0000", 2024-09-14T05:30:17.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9253999999999997e-05 2024-09-14T05:30:17.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:17.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296349+0000", 2024-09-14T05:30:17.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043051 2024-09-14T05:30:17.044 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.044 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:17.044 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.044 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27798 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:17.045 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:17.045 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.756815327999998, 2024-09-14T05:30:17.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085300275999999994, 2024-09-14T05:30:17.045 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:17.045 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:17.045 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:17.045 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:17.045 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:17.045 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.045 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27798 2024-09-14T05:30:17.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.045 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:17.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:17.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:17.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:17.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:17.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:17.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:17.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:17.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:17.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211472+0000", 2024-09-14T05:30:17.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7579999999999999e-06 2024-09-14T05:30:17.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:17.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211478+0000", 2024-09-14T05:30:17.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0310000000000004e-06 2024-09-14T05:30:17.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211483+0000", 2024-09-14T05:30:17.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8110000000000002e-06 2024-09-14T05:30:17.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.048 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:17.048 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296469+0000", 2024-09-14T05:30:17.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084986575999999994 2024-09-14T05:30:17.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.048 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:17.048 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296545+0000", 2024-09-14T05:30:17.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5749999999999998e-05 2024-09-14T05:30:17.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.048 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:17.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296770+0000", 2024-09-14T05:30:17.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022498900000000001 2024-09-14T05:30:17.049 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.049 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:17.049 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.049 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27799 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 307200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:17.049 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:17.049 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.756796415, 2024-09-14T05:30:17.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085718461999999995, 2024-09-14T05:30:17.049 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:17.049 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:17.049 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:17.050 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:17.050 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:17.050 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.050 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27799 2024-09-14T05:30:17.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.050 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:17.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:17.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:17.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:17.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:17.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:17.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:17.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8000000000000001e-07 2024-09-14T05:30:17.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:17.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211491+0000", 2024-09-14T05:30:17.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-14T05:30:17.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:17.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211497+0000", 2024-09-14T05:30:17.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.857e-06 2024-09-14T05:30:17.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211502+0000", 2024-09-14T05:30:17.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9790000000000001e-06 2024-09-14T05:30:17.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:17.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296800+0000", 2024-09-14T05:30:17.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085298099000000002 2024-09-14T05:30:17.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.053 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:17.053 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296871+0000", 2024-09-14T05:30:17.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0879999999999999e-05 2024-09-14T05:30:17.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.053 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:17.053 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297207+0000", 2024-09-14T05:30:17.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033651699999999999 2024-09-14T05:30:17.053 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.053 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:17.053 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.054 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27826 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:17.054 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:30:17.054 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.755081294, 2024-09-14T05:30:17.054 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084306729999999996, 2024-09-14T05:30:17.054 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:17.054 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:17.054 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:17.054 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:17.054 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:17.054 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.054 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27826 2024-09-14T05:30:17.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.054 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:17.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:17.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:30:17.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:17.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:30:17.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:17.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213205+0000", 2024-09-14T05:30:17.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.277e-06 2024-09-14T05:30:17.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:17.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213210+0000", 2024-09-14T05:30:17.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3120000000000001e-06 2024-09-14T05:30:17.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:17.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213226+0000", 2024-09-14T05:30:17.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5279000000000001e-05 2024-09-14T05:30:17.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213234+0000", 2024-09-14T05:30:17.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7020000000000007e-06 2024-09-14T05:30:17.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:17.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297239+0000", 2024-09-14T05:30:17.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084004876000000006 2024-09-14T05:30:17.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:17.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297311+0000", 2024-09-14T05:30:17.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1336999999999998e-05 2024-09-14T05:30:17.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:17.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297511+0000", 2024-09-14T05:30:17.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019994699999999999 2024-09-14T05:30:17.058 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.058 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:17.058 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.058 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.058 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.058 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27827 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:17.058 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:30:17.058 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 30.755036106999999, 2024-09-14T05:30:17.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084520537000000007, 2024-09-14T05:30:17.058 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:17.058 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:17.058 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:17.059 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:17.059 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:17.059 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.059 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27827 2024-09-14T05:30:17.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.059 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:17.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.059 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:17.059 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:30:17.059 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.059 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:17.059 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:30:17.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:17.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:17.060 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213250+0000", 2024-09-14T05:30:17.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1099999999999995e-07 2024-09-14T05:30:17.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:17.060 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213252+0000", 2024-09-14T05:30:17.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6699999999999998e-06 2024-09-14T05:30:17.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:17.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213259+0000", 2024-09-14T05:30:17.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8469999999999998e-06 2024-09-14T05:30:17.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213265+0000", 2024-09-14T05:30:17.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4940000000000003e-06 2024-09-14T05:30:17.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:17.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297541+0000", 2024-09-14T05:30:17.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084276731999999993 2024-09-14T05:30:17.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:17.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297579+0000", 2024-09-14T05:30:17.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8108999999999997e-05 2024-09-14T05:30:17.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:17.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:17.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:17.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297770+0000", 2024-09-14T05:30:17.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019007400000000001 2024-09-14T05:30:17.062 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.062 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:17.062 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.062 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:17.062 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:17.062 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:17.063 INFO:teuthology.orchestra.run.smithi071.stdout:4901 trunc from 0x7fbbde9 to 0x1f559dd 2024-09-14T05:30:17.064 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:30:17.067 INFO:teuthology.orchestra.run.smithi071.stdout:4902 write 0x5838513 thru 0x583ffab (0x7a99 bytes) 2024-09-14T05:30:17.067 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:17.068 INFO:teuthology.orchestra.run.smithi071.stdout:4904 read 0x48bb21a thru 0x48be393 (0x317a bytes) 2024-09-14T05:30:17.068 INFO:teuthology.orchestra.run.smithi071.stdout:4905 read 0x18f578d thru 0x1902e9f (0xd713 bytes) 2024-09-14T05:30:17.068 INFO:teuthology.orchestra.run.smithi071.stdout:4906 write 0x3fdfad4 thru 0x3fe981b (0x9d48 bytes) 2024-09-14T05:30:17.068 INFO:teuthology.orchestra.run.smithi071.stdout:4908 read 0x531bf8d thru 0x5321172 (0x51e6 bytes) 2024-09-14T05:30:17.068 INFO:teuthology.orchestra.run.smithi071.stdout:4909 write 0xb0ac058 thru 0xb0b0f43 (0x4eec bytes) 2024-09-14T05:30:17.068 INFO:teuthology.orchestra.run.smithi071.stdout:4910 writesame 0xcc9bdd0 thru 0xcca7640 data_size 0x1(0xb871 bytes) 2024-09-14T05:30:17.068 INFO:teuthology.orchestra.run.smithi071.stdout:4911 read 0x22af88f thru 0x22b3eb7 (0x4629 bytes) 2024-09-14T05:30:17.068 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:17.068 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:17.068 INFO:teuthology.orchestra.run.smithi071.stdout:4916 read 0xcb586d6 thru 0xcb6194b (0x9276 bytes) 2024-09-14T05:30:17.068 INFO:teuthology.orchestra.run.smithi071.stdout:4917 write 0x487d9b4 thru 0x48887bc (0xae09 bytes) 2024-09-14T05:30:17.068 INFO:teuthology.orchestra.run.smithi071.stdout:4918 punch from 0x280e3fa to 0x28147d1, (0x63d7 bytes) 2024-09-14T05:30:17.068 INFO:teuthology.orchestra.run.smithi071.stdout:4919 punch from 0x4b0e61f to 0x4b12415, (0x3df6 bytes) 2024-09-14T05:30:17.069 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:17.069 INFO:teuthology.orchestra.run.smithi071.stdout:4921 read 0x3422813 thru 0x342ff23 (0xd711 bytes) 2024-09-14T05:30:17.069 INFO:teuthology.orchestra.run.smithi071.stdout:4922 read 0x5714bc6 thru 0x5719e8d (0x52c8 bytes) 2024-09-14T05:30:17.069 INFO:teuthology.orchestra.run.smithi071.stdout:4923 punch from 0x9500f9c to 0x95033bc, (0x2420 bytes) 2024-09-14T05:30:17.069 INFO:teuthology.orchestra.run.smithi071.stdout:4924 trunc from 0xcca7641 to 0xada2b59 2024-09-14T05:30:17.069 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:17.069 INFO:teuthology.orchestra.run.smithi071.stdout:4926 write 0x6416380 thru 0x641831c (0x1f9d bytes) 2024-09-14T05:30:17.069 INFO:teuthology.orchestra.run.smithi071.stdout:4927 read 0x2fc8e61 thru 0x2fcf740 (0x68e0 bytes) 2024-09-14T05:30:17.069 INFO:teuthology.orchestra.run.smithi071.stdout:4928 write 0xc27dfa thru 0xc2a0da (0x22e1 bytes) 2024-09-14T05:30:17.069 INFO:teuthology.orchestra.run.smithi071.stdout:4930 read 0x2ec203a thru 0x2ecf420 (0xd3e7 bytes) 2024-09-14T05:30:17.069 INFO:teuthology.orchestra.run.smithi071.stdout:4931 write 0x2636669 thru 0x2642d95 (0xc72d bytes) 2024-09-14T05:30:17.069 INFO:teuthology.orchestra.run.smithi071.stdout:4932 read 0x92599fd thru 0x925f497 (0x5a9b bytes) 2024-09-14T05:30:17.069 INFO:teuthology.orchestra.run.smithi071.stdout:4934 read 0x1e8fcbd thru 0x1e9ef6c (0xf2b0 bytes) 2024-09-14T05:30:17.069 INFO:teuthology.orchestra.run.smithi071.stdout:4937 trunc from 0xada2b59 to 0xdfbbb6a 2024-09-14T05:30:17.070 INFO:teuthology.orchestra.run.smithi071.stdout:4938 punch from 0x2138971 to 0x2141d75, (0x9404 bytes) 2024-09-14T05:30:17.070 INFO:teuthology.orchestra.run.smithi071.stdout:4941 write 0x2c5dd0d thru 0x2c6667b (0x896f bytes) 2024-09-14T05:30:17.070 INFO:teuthology.orchestra.run.smithi071.stdout:4942 write 0x7df135e thru 0x7e00d24 (0xf9c7 bytes) 2024-09-14T05:30:17.070 INFO:teuthology.orchestra.run.smithi071.stdout:4943 write 0x5ec02e2 thru 0x5ec8202 (0x7f21 bytes) 2024-09-14T05:30:17.070 INFO:teuthology.orchestra.run.smithi071.stdout:4944 write 0x1dc1e2e thru 0x1dd0673 (0xe846 bytes) 2024-09-14T05:30:17.070 INFO:teuthology.orchestra.run.smithi071.stdout:4945 read 0x9f85aa2 thru 0x9f9389d (0xddfc bytes) 2024-09-14T05:30:17.070 INFO:teuthology.orchestra.run.smithi071.stdout:4946 read 0xc9b72 thru 0xd68fc (0xcd8b bytes) 2024-09-14T05:30:17.070 INFO:teuthology.orchestra.run.smithi071.stdout:4947 punch from 0xdeaf5b9 to 0xdeb00ed, (0xb34 bytes) 2024-09-14T05:30:17.070 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:17.063+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:17.073 INFO:teuthology.orchestra.run.smithi071.stdout:4949 trunc from 0xdfbbb6a to 0x1df72e4 2024-09-14T05:30:17.099 INFO:teuthology.orchestra.run.smithi071.stdout:4950 writesame 0x857359a thru 0x8576f1e data_size 0x1f(0x3985 bytes) 2024-09-14T05:30:17.099 INFO:teuthology.orchestra.run.smithi071.stdout:4951 punch from 0x3d0977d to 0x3d14b29, (0xb3ac bytes) 2024-09-14T05:30:17.099 INFO:teuthology.orchestra.run.smithi071.stdout:4952 punch from 0x1e994e0 to 0x1ea00ae, (0x6bce bytes) 2024-09-14T05:30:17.100 INFO:teuthology.orchestra.run.smithi071.stdout:4953 read 0x7bf5105 thru 0x7bffb33 (0xaa2f bytes) 2024-09-14T05:30:17.100 INFO:teuthology.orchestra.run.smithi071.stdout:4956 read 0xf35703 thru 0xf3c5a5 (0x6ea3 bytes) 2024-09-14T05:30:17.100 INFO:teuthology.orchestra.run.smithi071.stdout:4957 read 0x69fc7f2 thru 0x6a0b524 (0xed33 bytes) 2024-09-14T05:30:17.100 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:17.100 INFO:teuthology.orchestra.run.smithi071.stdout:4960 write 0x65db4d thru 0x664811 (0x6cc5 bytes) 2024-09-14T05:30:17.100 INFO:teuthology.orchestra.run.smithi071.stdout:4962 read 0x536fd20 thru 0x5373131 (0x3412 bytes) 2024-09-14T05:30:17.101 INFO:teuthology.orchestra.run.smithi071.stdout:4964 writesame 0x231d436 thru 0x231f61c data_size 0xb(0x21e7 bytes) 2024-09-14T05:30:17.102 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:17.102 INFO:teuthology.orchestra.run.smithi071.stdout:4967 read 0x433a668 thru 0x433f360 (0x4cf9 bytes) 2024-09-14T05:30:17.102 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:17.105 INFO:teuthology.orchestra.run.smithi071.stdout:4970 trunc from 0x8576f1f to 0x6718965 2024-09-14T05:30:17.127 INFO:teuthology.orchestra.run.smithi071.stdout:4971 write 0xe83aa5e thru 0xe83ccc9 (0x226c bytes) 2024-09-14T05:30:17.130 INFO:teuthology.orchestra.run.smithi071.stdout:4972 writesame 0x1f93449 thru 0x1f961ae data_size 0x27(0x2d66 bytes) 2024-09-14T05:30:17.132 INFO:teuthology.orchestra.run.smithi071.stdout:4973 write 0xe4a32fc thru 0xe4ad9bb (0xa6c0 bytes) 2024-09-14T05:30:17.134 INFO:teuthology.orchestra.run.smithi071.stdout:4974 write 0x8a4f8a7 thru 0x8a5bd44 (0xc49e bytes) 2024-09-14T05:30:17.148 INFO:teuthology.orchestra.run.smithi071.stdout:4975 trunc from 0xe83ccca to 0x5906966 2024-09-14T05:30:17.163 INFO:teuthology.orchestra.run.smithi071.stdout:4976 trunc from 0x5906966 to 0x7bd7ab4 2024-09-14T05:30:17.164 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:17.164+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:17.168 INFO:teuthology.orchestra.run.smithi071.stdout:4977 punch from 0x12687f3 to 0x1276cd3, (0xe4e0 bytes) 2024-09-14T05:30:17.168 INFO:teuthology.orchestra.run.smithi071.stdout:4980 read 0x51c8a2a thru 0x51cdde8 (0x53bf bytes) 2024-09-14T05:30:17.177 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:17.177 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:17.177 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:17.177 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:17.181 INFO:teuthology.orchestra.run.smithi071.stdout:4981 write 0xe2d8844 thru 0xe2d996c (0x1129 bytes) 2024-09-14T05:30:17.184 INFO:teuthology.orchestra.run.smithi071.stdout:4982 write 0x9f07498 thru 0x9f095b7 (0x2120 bytes) 2024-09-14T05:30:17.187 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:30:17.200 INFO:teuthology.orchestra.run.smithi071.stdout:4983 trunc from 0xe2d996d to 0x335b586 2024-09-14T05:30:17.212 INFO:teuthology.orchestra.run.smithi071.stdout:4984 read 0x48fda9 thru 0x49993b (0x9b93 bytes) 2024-09-14T05:30:17.212 INFO:teuthology.orchestra.run.smithi071.stdout:4986 read 0x1c76b74 thru 0x1c7f37f (0x880c bytes) 2024-09-14T05:30:17.213 INFO:teuthology.orchestra.run.smithi071.stdout:4987 writesame 0xeaad26 thru 0xeae4e9 data_size 0x53(0x37c4 bytes) 2024-09-14T05:30:17.220 INFO:teuthology.orchestra.run.smithi071.stdout:4988 writesame 0x40b1565 thru 0x40bd62e data_size 0x1(0xc0ca bytes) 2024-09-14T05:30:17.221 INFO:teuthology.orchestra.run.smithi071.stdout:4989 read 0x3c74f52 thru 0x3c7c326 (0x73d5 bytes) 2024-09-14T05:30:17.222 INFO:teuthology.orchestra.run.smithi071.stdout:4990 punch from 0x1883df1 to 0x1890d8b, (0xcf9a bytes) 2024-09-14T05:30:17.222 INFO:teuthology.orchestra.run.smithi071.stdout:4991 read 0x47bb2e thru 0x483e21 (0x82f4 bytes) 2024-09-14T05:30:17.222 INFO:teuthology.orchestra.run.smithi071.stdout:4992 punch from 0x5e635 to 0x5f7d0, (0x119b bytes) 2024-09-14T05:30:17.235 INFO:teuthology.orchestra.run.smithi071.stdout:4993 write 0x9486eff thru 0x948865c (0x175e bytes) 2024-09-14T05:30:17.237 INFO:teuthology.orchestra.run.smithi071.stdout:4995 punch from 0x323ec08 to 0x324dcf1, (0xf0e9 bytes) 2024-09-14T05:30:17.238 INFO:teuthology.orchestra.run.smithi071.stdout:4996 write 0x7039889 thru 0x703c782 (0x2efa bytes) 2024-09-14T05:30:17.240 INFO:teuthology.orchestra.run.smithi071.stdout:4997 read 0x9efacc thru 0x9fad67 (0xb29c bytes) 2024-09-14T05:30:17.250 INFO:teuthology.orchestra.run.smithi071.stdout:4998 trunc from 0x948865d to 0x1a3db25 2024-09-14T05:30:17.261 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:17.261 INFO:teuthology.orchestra.run.smithi071.stdout:5000 read 0x6b3ca3 thru 0x6b7535 (0x3893 bytes) 2024-09-14T05:30:17.261 INFO:teuthology.orchestra.run.smithi071.stdout:5002 clone 37 order 21 su 8192 sc 13 2024-09-14T05:30:17.265 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:17.265+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:17.293 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:17.293 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:17.293 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:17.293 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:17.293 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:17.302 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:30:17.306 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-14T05:30:17.364 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone36 from 0x1a3db25 (overlap 0xa1cd1f) to 0x8170e5 2024-09-14T05:30:17.366 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:17.365+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:17.414 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:17.415 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:17.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:17.415 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:17.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.415 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3523 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6062080~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.415 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:17.415 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.89857464, 2024-09-14T05:30:17.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096911030999999995, 2024-09-14T05:30:17.415 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.415 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.415 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.416 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3523 2024-09-14T05:30:17.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.416 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:17.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:17.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514359+0000", 2024-09-14T05:30:17.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9299999999999999e-07 2024-09-14T05:30:17.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514361+0000", 2024-09-14T05:30:17.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:17.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514366+0000", 2024-09-14T05:30:17.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1660000000000002e-06 2024-09-14T05:30:17.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514371+0000", 2024-09-14T05:30:17.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.648e-06 2024-09-14T05:30:17.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.610954+0000", 2024-09-14T05:30:17.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096582170999999994 2024-09-14T05:30:17.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611002+0000", 2024-09-14T05:30:17.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7923999999999997e-05 2024-09-14T05:30:17.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611269+0000", 2024-09-14T05:30:17.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026789499999999998 2024-09-14T05:30:17.419 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.419 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.419 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.420 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3524 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6553600~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.420 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:17.420 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.898555448000003, 2024-09-14T05:30:17.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097263347999999999, 2024-09-14T05:30:17.420 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.420 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.420 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.420 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3524 2024-09-14T05:30:17.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.420 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:17.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:17.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:17.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:17.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514380+0000", 2024-09-14T05:30:17.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6500000000000001e-06 2024-09-14T05:30:17.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514386+0000", 2024-09-14T05:30:17.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7360000000000004e-06 2024-09-14T05:30:17.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514391+0000", 2024-09-14T05:30:17.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2830000000000003e-06 2024-09-14T05:30:17.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611295+0000", 2024-09-14T05:30:17.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096904814000000006 2024-09-14T05:30:17.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611343+0000", 2024-09-14T05:30:17.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7506999999999998e-05 2024-09-14T05:30:17.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611641+0000", 2024-09-14T05:30:17.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029798800000000001 2024-09-14T05:30:17.424 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.424 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.424 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.424 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3525 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7045120~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.424 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:17.424 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.898533561000001, 2024-09-14T05:30:17.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097644422999999994, 2024-09-14T05:30:17.424 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.424 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.424 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.425 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.425 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3525 2024-09-14T05:30:17.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.425 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:17.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:17.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514400+0000", 2024-09-14T05:30:17.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.77e-07 2024-09-14T05:30:17.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514402+0000", 2024-09-14T05:30:17.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6530000000000001e-06 2024-09-14T05:30:17.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514411+0000", 2024-09-14T05:30:17.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7429999999999997e-06 2024-09-14T05:30:17.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514417+0000", 2024-09-14T05:30:17.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2249999999999997e-06 2024-09-14T05:30:17.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611668+0000", 2024-09-14T05:30:17.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097250681000000005 2024-09-14T05:30:17.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611716+0000", 2024-09-14T05:30:17.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7336000000000002e-05 2024-09-14T05:30:17.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612044+0000", 2024-09-14T05:30:17.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032840799999999998 2024-09-14T05:30:17.429 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.429 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.429 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.429 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3526 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7536640~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.429 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:17.429 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.898509077, 2024-09-14T05:30:17.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097943884999999994, 2024-09-14T05:30:17.430 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.430 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.430 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.430 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.430 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.430 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.430 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3526 2024-09-14T05:30:17.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.430 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:17.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:17.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:17.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0400000000000002e-07 2024-09-14T05:30:17.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514426+0000", 2024-09-14T05:30:17.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:17.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514432+0000", 2024-09-14T05:30:17.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.105e-06 2024-09-14T05:30:17.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514438+0000", 2024-09-14T05:30:17.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5149999999999997e-06 2024-09-14T05:30:17.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612072+0000", 2024-09-14T05:30:17.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097633947999999998 2024-09-14T05:30:17.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612120+0000", 2024-09-14T05:30:17.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8458999999999998e-05 2024-09-14T05:30:17.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612368+0000", 2024-09-14T05:30:17.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024761999999999999 2024-09-14T05:30:17.434 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.434 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.434 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.435 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3527 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 8028160~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.435 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:17.435 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.898488364000002, 2024-09-14T05:30:17.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098244955999999994, 2024-09-14T05:30:17.435 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.435 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.435 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.435 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.435 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.436 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3527 2024-09-14T05:30:17.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.436 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:17.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:17.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519842+0000", 2024-09-14T05:30:17.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053975230000000004 2024-09-14T05:30:17.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519851+0000", 2024-09-14T05:30:17.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5599999999999994e-06 2024-09-14T05:30:17.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519858+0000", 2024-09-14T05:30:17.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2409999999999998e-06 2024-09-14T05:30:17.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519863+0000", 2024-09-14T05:30:17.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:30:17.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612395+0000", 2024-09-14T05:30:17.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092532552000000004 2024-09-14T05:30:17.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612442+0000", 2024-09-14T05:30:17.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6486000000000002e-05 2024-09-14T05:30:17.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612690+0000", 2024-09-14T05:30:17.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024791599999999999 2024-09-14T05:30:17.440 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.440 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.440 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.440 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3603 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5177344~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.440 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:17.440 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.892935919000003, 2024-09-14T05:30:17.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096796436, 2024-09-14T05:30:17.440 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.440 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.440 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.441 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3603 2024-09-14T05:30:17.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.441 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:17.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:17.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:17.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1899999999999999e-07 2024-09-14T05:30:17.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519998+0000", 2024-09-14T05:30:17.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5000000000000001e-07 2024-09-14T05:30:17.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520002+0000", 2024-09-14T05:30:17.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7299999999999999e-06 2024-09-14T05:30:17.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520004+0000", 2024-09-14T05:30:17.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.429e-06 2024-09-14T05:30:17.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616493+0000", 2024-09-14T05:30:17.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096488345000000003 2024-09-14T05:30:17.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616544+0000", 2024-09-14T05:30:17.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1724000000000001e-05 2024-09-14T05:30:17.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616794+0000", 2024-09-14T05:30:17.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000249139 2024-09-14T05:30:17.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.444 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.444 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3604 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5668864~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.445 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:17.445 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.892921281, 2024-09-14T05:30:17.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108222999999994, 2024-09-14T05:30:17.445 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.445 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.445 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.445 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3604 2024-09-14T05:30:17.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.445 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:17.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:17.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:17.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7000000000000001e-07 2024-09-14T05:30:17.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520013+0000", 2024-09-14T05:30:17.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.389e-06 2024-09-14T05:30:17.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520018+0000", 2024-09-14T05:30:17.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3019999999999996e-06 2024-09-14T05:30:17.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520026+0000", 2024-09-14T05:30:17.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6170000000000007e-06 2024-09-14T05:30:17.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616816+0000", 2024-09-14T05:30:17.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096789766999999999 2024-09-14T05:30:17.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616871+0000", 2024-09-14T05:30:17.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4811000000000001e-05 2024-09-14T05:30:17.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617120+0000", 2024-09-14T05:30:17.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024906700000000001 2024-09-14T05:30:17.449 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.449 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.449 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.449 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3605 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6160384~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.449 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:17.449 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.892902821, 2024-09-14T05:30:17.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097423945999999997, 2024-09-14T05:30:17.449 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.449 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.450 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.450 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.450 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.450 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.450 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3605 2024-09-14T05:30:17.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.450 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:17.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:17.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:17.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3999999999999998e-07 2024-09-14T05:30:17.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520032+0000", 2024-09-14T05:30:17.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.083e-06 2024-09-14T05:30:17.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520036+0000", 2024-09-14T05:30:17.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.019e-06 2024-09-14T05:30:17.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520038+0000", 2024-09-14T05:30:17.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4849999999999999e-06 2024-09-14T05:30:17.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617147+0000", 2024-09-14T05:30:17.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108665999999996 2024-09-14T05:30:17.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617199+0000", 2024-09-14T05:30:17.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2558000000000001e-05 2024-09-14T05:30:17.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617454+0000", 2024-09-14T05:30:17.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025489499999999999 2024-09-14T05:30:17.453 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.453 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.453 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.454 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3606 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6651904~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.454 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:17.454 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.892892099000001, 2024-09-14T05:30:17.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097745705000000002, 2024-09-14T05:30:17.454 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.454 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.454 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.454 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.454 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.454 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.454 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3606 2024-09-14T05:30:17.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.454 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:17.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:17.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:17.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-14T05:30:17.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520042+0000", 2024-09-14T05:30:17.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4799999999999997e-07 2024-09-14T05:30:17.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520046+0000", 2024-09-14T05:30:17.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8149999999999999e-06 2024-09-14T05:30:17.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520049+0000", 2024-09-14T05:30:17.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.576e-06 2024-09-14T05:30:17.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617476+0000", 2024-09-14T05:30:17.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097427558999999997 2024-09-14T05:30:17.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617522+0000", 2024-09-14T05:30:17.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5927999999999998e-05 2024-09-14T05:30:17.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617787+0000", 2024-09-14T05:30:17.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000264663 2024-09-14T05:30:17.457 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.458 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.458 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.458 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3607 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7143424~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.458 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:17.458 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.892881572, 2024-09-14T05:30:17.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097871125000000003, 2024-09-14T05:30:17.458 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.458 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.458 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.458 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.458 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.458 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.458 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3607 2024-09-14T05:30:17.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.459 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:17.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:17.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520052+0000", 2024-09-14T05:30:17.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.03e-07 2024-09-14T05:30:17.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520053+0000", 2024-09-14T05:30:17.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.85e-07 2024-09-14T05:30:17.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520056+0000", 2024-09-14T05:30:17.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9140000000000001e-06 2024-09-14T05:30:17.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520059+0000", 2024-09-14T05:30:17.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4600000000000002e-06 2024-09-14T05:30:17.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617809+0000", 2024-09-14T05:30:17.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097750216000000001 2024-09-14T05:30:17.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617864+0000", 2024-09-14T05:30:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5211999999999999e-05 2024-09-14T05:30:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617923+0000", 2024-09-14T05:30:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8235000000000001e-05 2024-09-14T05:30:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3608 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7634944~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.892871264, 2024-09-14T05:30:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098199212999999994, 2024-09-14T05:30:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3608 2024-09-14T05:30:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525953+0000", 2024-09-14T05:30:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058911850000000002 2024-09-14T05:30:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525957+0000", 2024-09-14T05:30:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2379999999999999e-06 2024-09-14T05:30:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525967+0000", 2024-09-14T05:30:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5389999999999996e-06 2024-09-14T05:30:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525972+0000", 2024-09-14T05:30:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.541e-06 2024-09-14T05:30:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617945+0000", 2024-09-14T05:30:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091972597000000003 2024-09-14T05:30:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617992+0000", 2024-09-14T05:30:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7397e-05 2024-09-14T05:30:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.618261+0000", 2024-09-14T05:30:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026871600000000001 2024-09-14T05:30:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3640 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6586368~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.886763422000001, 2024-09-14T05:30:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097068580000000002, 2024-09-14T05:30:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3640 2024-09-14T05:30:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.23e-07 2024-09-14T05:30:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526171+0000", 2024-09-14T05:30:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:30:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526175+0000", 2024-09-14T05:30:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8369999999999999e-06 2024-09-14T05:30:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526177+0000", 2024-09-14T05:30:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4959999999999999e-06 2024-09-14T05:30:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622847+0000", 2024-09-14T05:30:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096669790000000005 2024-09-14T05:30:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622902+0000", 2024-09-14T05:30:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5157000000000001e-05 2024-09-14T05:30:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623238+0000", 2024-09-14T05:30:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033603399999999997 2024-09-14T05:30:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3641 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7077888~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.886752911999999, 2024-09-14T05:30:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097371141999999994, 2024-09-14T05:30:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3641 2024-09-14T05:30:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531851+0000", 2024-09-14T05:30:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005671265 2024-09-14T05:30:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531855+0000", 2024-09-14T05:30:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0849999999999999e-06 2024-09-14T05:30:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531867+0000", 2024-09-14T05:30:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1028e-05 2024-09-14T05:30:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531872+0000", 2024-09-14T05:30:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000003e-06 2024-09-14T05:30:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623251+0000", 2024-09-14T05:30:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091378980999999998 2024-09-14T05:30:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623295+0000", 2024-09-14T05:30:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3881999999999998e-05 2024-09-14T05:30:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623551+0000", 2024-09-14T05:30:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025636099999999998 2024-09-14T05:30:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3721 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.880894636999997, 2024-09-14T05:30:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096806472000000005, 2024-09-14T05:30:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3721 2024-09-14T05:30:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532039+0000", 2024-09-14T05:30:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.05e-07 2024-09-14T05:30:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532040+0000", 2024-09-14T05:30:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9099999999999991e-07 2024-09-14T05:30:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532043+0000", 2024-09-14T05:30:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6610000000000002e-06 2024-09-14T05:30:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532046+0000", 2024-09-14T05:30:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4080000000000001e-06 2024-09-14T05:30:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628545+0000", 2024-09-14T05:30:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096499615999999996 2024-09-14T05:30:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628592+0000", 2024-09-14T05:30:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6907000000000003e-05 2024-09-14T05:30:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628845+0000", 2024-09-14T05:30:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025268399999999998 2024-09-14T05:30:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3722 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6193152~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.880884557000002, 2024-09-14T05:30:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097146587000000006, 2024-09-14T05:30:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3722 2024-09-14T05:30:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538191+0000", 2024-09-14T05:30:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061429969999999999 2024-09-14T05:30:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538200+0000", 2024-09-14T05:30:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7069999999999998e-06 2024-09-14T05:30:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538212+0000", 2024-09-14T05:30:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1504e-05 2024-09-14T05:30:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538219+0000", 2024-09-14T05:30:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7500000000000003e-06 2024-09-14T05:30:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628869+0000", 2024-09-14T05:30:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090649224 2024-09-14T05:30:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628919+0000", 2024-09-14T05:30:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0144e-05 2024-09-14T05:30:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.629195+0000", 2024-09-14T05:30:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027626099999999998 2024-09-14T05:30:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3758 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7110656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.874290340999998, 2024-09-14T05:30:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096977544999999998, 2024-09-14T05:30:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3758 2024-09-14T05:30:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538645+0000", 2024-09-14T05:30:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.843e-06 2024-09-14T05:30:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538651+0000", 2024-09-14T05:30:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6169999999999998e-06 2024-09-14T05:30:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538655+0000", 2024-09-14T05:30:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5399999999999997e-06 2024-09-14T05:30:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635225+0000", 2024-09-14T05:30:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096570149999999993 2024-09-14T05:30:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635272+0000", 2024-09-14T05:30:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7199e-05 2024-09-14T05:30:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635620+0000", 2024-09-14T05:30:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034782600000000003 2024-09-14T05:30:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3759 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.874272429999998, 2024-09-14T05:30:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097335331999999997, 2024-09-14T05:30:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3759 2024-09-14T05:30:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544013+0000", 2024-09-14T05:30:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053520829999999997 2024-09-14T05:30:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544016+0000", 2024-09-14T05:30:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6660000000000001e-06 2024-09-14T05:30:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544026+0000", 2024-09-14T05:30:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7089999999999997e-06 2024-09-14T05:30:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544043+0000", 2024-09-14T05:30:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833999999999998e-05 2024-09-14T05:30:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635646+0000", 2024-09-14T05:30:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091603544999999995 2024-09-14T05:30:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635696+0000", 2024-09-14T05:30:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9307000000000001e-05 2024-09-14T05:30:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635996+0000", 2024-09-14T05:30:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030018800000000001 2024-09-14T05:30:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3838 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6225920~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.868610750000002, 2024-09-14T05:30:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096894235999999995, 2024-09-14T05:30:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3838 2024-09-14T05:30:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544323+0000", 2024-09-14T05:30:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-14T05:30:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544325+0000", 2024-09-14T05:30:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.75e-06 2024-09-14T05:30:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544331+0000", 2024-09-14T05:30:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5019999999999998e-06 2024-09-14T05:30:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544335+0000", 2024-09-14T05:30:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2989999999999998e-06 2024-09-14T05:30:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640869+0000", 2024-09-14T05:30:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096533351000000003 2024-09-14T05:30:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640919+0000", 2024-09-14T05:30:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999999e-05 2024-09-14T05:30:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641217+0000", 2024-09-14T05:30:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029717600000000002 2024-09-14T05:30:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3839 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.868591811999998, 2024-09-14T05:30:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097240339999999995, 2024-09-14T05:30:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3839 2024-09-14T05:30:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549834+0000", 2024-09-14T05:30:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054927819999999999 2024-09-14T05:30:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549838+0000", 2024-09-14T05:30:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8689999999999997e-06 2024-09-14T05:30:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549848+0000", 2024-09-14T05:30:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0154e-05 2024-09-14T05:30:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549853+0000", 2024-09-14T05:30:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9740000000000001e-06 2024-09-14T05:30:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641245+0000", 2024-09-14T05:30:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091392245999999996 2024-09-14T05:30:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641295+0000", 2024-09-14T05:30:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0018999999999997e-05 2024-09-14T05:30:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641582+0000", 2024-09-14T05:30:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028629600000000001 2024-09-14T05:30:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3868 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.862929233999999, 2024-09-14T05:30:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096859555999999999, 2024-09-14T05:30:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3868 2024-09-14T05:30:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556173+0000", 2024-09-14T05:30:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006169038 2024-09-14T05:30:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556177+0000", 2024-09-14T05:30:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0010000000000003e-06 2024-09-14T05:30:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556188+0000", 2024-09-14T05:30:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0675e-05 2024-09-14T05:30:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556199+0000", 2024-09-14T05:30:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.155e-05 2024-09-14T05:30:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646511+0000", 2024-09-14T05:30:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090312187000000002 2024-09-14T05:30:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646563+0000", 2024-09-14T05:30:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2219000000000003e-05 2024-09-14T05:30:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646863+0000", 2024-09-14T05:30:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029988600000000002 2024-09-14T05:30:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.506 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:17.507 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-14T05:30:17.508 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:30:17.509 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:17.507+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:17.608 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:17.608+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:17.611 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:17.611 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:17.611 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:17.611 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:17.620 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:30:17.694 INFO:teuthology.orchestra.run.smithi071.stdout:5003 write 0xcb0cb84 thru 0xcb1a48c (0xd909 bytes) 2024-09-14T05:30:17.704 INFO:teuthology.orchestra.run.smithi071.stdout:5004 write 0xac331b1 thru 0xac39bdf (0x6a2f bytes) 2024-09-14T05:30:17.710 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:17.709+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:17.719 INFO:teuthology.orchestra.run.smithi071.stdout:5005 write 0xea2060d thru 0xea2a5ef (0x9fe3 bytes) 2024-09-14T05:30:17.728 INFO:teuthology.orchestra.run.smithi071.stdout:5007 read 0x7afb176 thru 0x7b05a2f (0xa8ba bytes) 2024-09-14T05:30:17.728 INFO:teuthology.orchestra.run.smithi071.stdout:5008 writesame 0x944627c thru 0x944788c data_size 0x3(0x1611 bytes) 2024-09-14T05:30:17.742 INFO:teuthology.orchestra.run.smithi071.stdout:5009 trunc from 0xea2a5f0 to 0x563042a 2024-09-14T05:30:17.755 INFO:teuthology.orchestra.run.smithi071.stdout:5010 write 0x17e778 thru 0x186027 (0x78b0 bytes) 2024-09-14T05:30:17.773 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:17.773 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:17.773 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:17.773 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:17.773 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:17.783 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:30:17.811 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:17.810+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:17.836 INFO:teuthology.orchestra.run.smithi071.stdout:5012 trunc from 0x563042a to 0x208bbbc 2024-09-14T05:30:17.858 INFO:teuthology.orchestra.run.smithi071.stdout:5013 write 0x59cbe thru 0x6903e (0xf381 bytes) 2024-09-14T05:30:17.902 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:17.902 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:17.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:17.902 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:17.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.902 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1989 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 16326080~54496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:17.902 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:17.902 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.339130345999997, 2024-09-14T05:30:17.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096664322999999996, 2024-09-14T05:30:17.903 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.903 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.903 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:17.903 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.903 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.903 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.903 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1989 2024-09-14T05:30:17.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.903 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:17.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:17.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561455+0000", 2024-09-14T05:30:17.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1699999999999997e-07 2024-09-14T05:30:17.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590703+0000", 2024-09-14T05:30:17.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029248225999999999 2024-09-14T05:30:17.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590732+0000", 2024-09-14T05:30:17.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8679000000000001e-05 2024-09-14T05:30:17.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590759+0000", 2024-09-14T05:30:17.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7319999999999999e-05 2024-09-14T05:30:17.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590835+0000", 2024-09-14T05:30:17.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5543999999999994e-05 2024-09-14T05:30:17.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590956+0000", 2024-09-14T05:30:17.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012168 2024-09-14T05:30:17.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 1", 2024-09-14T05:30:17.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.591261+0000", 2024-09-14T05:30:17.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030484799999999998 2024-09-14T05:30:17.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:17.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.637682+0000", 2024-09-14T05:30:17.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046420550999999997 2024-09-14T05:30:17.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:17.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.657912+0000", 2024-09-14T05:30:17.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020230011999999999 2024-09-14T05:30:17.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:17.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658007+0000", 2024-09-14T05:30:17.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5814000000000001e-05 2024-09-14T05:30:17.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658118+0000", 2024-09-14T05:30:17.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110832 2024-09-14T05:30:17.908 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.908 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.908 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.908 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2033 3.1e 3:7e47f6eb:::rbd_data.112d1f36d097.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30853661~63531] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:17.908 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:17.909 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.227855679999998, 2024-09-14T05:30:17.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065167630000000004, 2024-09-14T05:30:17.909 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.909 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.909 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:17.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.909 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2033 2024-09-14T05:30:17.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.909 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:17.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:17.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672730+0000", 2024-09-14T05:30:17.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.393e-06 2024-09-14T05:30:17.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691289+0000", 2024-09-14T05:30:17.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018558991 2024-09-14T05:30:17.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691316+0000", 2024-09-14T05:30:17.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7498999999999999e-05 2024-09-14T05:30:17.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691331+0000", 2024-09-14T05:30:17.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4652e-05 2024-09-14T05:30:17.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691397+0000", 2024-09-14T05:30:17.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6112000000000007e-05 2024-09-14T05:30:17.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691509+0000", 2024-09-14T05:30:17.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111599 2024-09-14T05:30:17.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:17.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.692253+0000", 2024-09-14T05:30:17.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074456200000000004 2024-09-14T05:30:17.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:17.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.719140+0000", 2024-09-14T05:30:17.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026886542999999999 2024-09-14T05:30:17.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:17.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737789+0000", 2024-09-14T05:30:17.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018649203999999999 2024-09-14T05:30:17.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:17.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737846+0000", 2024-09-14T05:30:17.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6501e-05 2024-09-14T05:30:17.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737896+0000", 2024-09-14T05:30:17.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0574000000000001e-05 2024-09-14T05:30:17.914 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.914 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.915 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.915 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3008 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13729792~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.915 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:17.915 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.596931810000001, 2024-09-14T05:30:17.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061007651000000003, 2024-09-14T05:30:17.915 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.915 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.915 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.915 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.915 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.915 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.915 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3008 2024-09-14T05:30:17.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.916 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:17.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:17.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303653+0000", 2024-09-14T05:30:17.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0699999999999999e-07 2024-09-14T05:30:17.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303654+0000", 2024-09-14T05:30:17.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-07 2024-09-14T05:30:17.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303657+0000", 2024-09-14T05:30:17.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2890000000000002e-06 2024-09-14T05:30:17.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303661+0000", 2024-09-14T05:30:17.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3290000000000004e-06 2024-09-14T05:30:17.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364233+0000", 2024-09-14T05:30:17.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060572134 2024-09-14T05:30:17.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364315+0000", 2024-09-14T05:30:17.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1549000000000001e-05 2024-09-14T05:30:17.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364660+0000", 2024-09-14T05:30:17.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034525300000000001 2024-09-14T05:30:17.919 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.919 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.919 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.919 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3009 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14221312~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.919 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:17.919 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.59691772, 2024-09-14T05:30:17.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061445063000000001, 2024-09-14T05:30:17.920 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.920 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.920 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.920 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.920 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.920 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.920 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3009 2024-09-14T05:30:17.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.920 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:17.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:17.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:17.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:17.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303669+0000", 2024-09-14T05:30:17.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7150000000000001e-06 2024-09-14T05:30:17.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303675+0000", 2024-09-14T05:30:17.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5339999999999996e-06 2024-09-14T05:30:17.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303681+0000", 2024-09-14T05:30:17.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3639999999999998e-06 2024-09-14T05:30:17.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364705+0000", 2024-09-14T05:30:17.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061024123 2024-09-14T05:30:17.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364786+0000", 2024-09-14T05:30:17.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1378999999999995e-05 2024-09-14T05:30:17.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365112+0000", 2024-09-14T05:30:17.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032558700000000001 2024-09-14T05:30:17.923 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.923 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.923 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.924 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3010 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14712832~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.924 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:17.924 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.596896856000001, 2024-09-14T05:30:17.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061834630000000002, 2024-09-14T05:30:17.924 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.924 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.924 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.924 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.924 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.924 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.924 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3010 2024-09-14T05:30:17.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.925 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:17.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:17.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303688+0000", 2024-09-14T05:30:17.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:30:17.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303690+0000", 2024-09-14T05:30:17.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9460000000000001e-06 2024-09-14T05:30:17.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303696+0000", 2024-09-14T05:30:17.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2230000000000004e-06 2024-09-14T05:30:17.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303700+0000", 2024-09-14T05:30:17.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7290000000000002e-06 2024-09-14T05:30:17.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365163+0000", 2024-09-14T05:30:17.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061463093000000003 2024-09-14T05:30:17.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365214+0000", 2024-09-14T05:30:17.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0977999999999999e-05 2024-09-14T05:30:17.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365522+0000", 2024-09-14T05:30:17.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000308227 2024-09-14T05:30:17.928 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.928 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.928 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.928 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3241 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.928 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:17.928 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.571222433999999, 2024-09-14T05:30:17.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061041892, 2024-09-14T05:30:17.928 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.929 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.929 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.929 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.929 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.929 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.929 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3241 2024-09-14T05:30:17.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.929 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:17.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:17.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329496+0000", 2024-09-14T05:30:17.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013421799999999999 2024-09-14T05:30:17.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329499+0000", 2024-09-14T05:30:17.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8459999999999999e-06 2024-09-14T05:30:17.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329505+0000", 2024-09-14T05:30:17.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4320000000000004e-06 2024-09-14T05:30:17.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329509+0000", 2024-09-14T05:30:17.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0339999999999998e-06 2024-09-14T05:30:17.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.389947+0000", 2024-09-14T05:30:17.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060438093999999998 2024-09-14T05:30:17.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390038+0000", 2024-09-14T05:30:17.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0253999999999993e-05 2024-09-14T05:30:17.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390404+0000", 2024-09-14T05:30:17.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036601400000000001 2024-09-14T05:30:17.932 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.932 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.932 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.933 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3242 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.933 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:17.933 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.571067526, 2024-09-14T05:30:17.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061282426000000001, 2024-09-14T05:30:17.933 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.933 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.933 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.933 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.933 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.933 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.933 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3242 2024-09-14T05:30:17.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.934 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:17.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:17.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:17.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9900000000000001e-07 2024-09-14T05:30:17.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329518+0000", 2024-09-14T05:30:17.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.328e-06 2024-09-14T05:30:17.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329522+0000", 2024-09-14T05:30:17.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.563e-06 2024-09-14T05:30:17.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329525+0000", 2024-09-14T05:30:17.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7379999999999999e-06 2024-09-14T05:30:17.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390437+0000", 2024-09-14T05:30:17.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060912463 2024-09-14T05:30:17.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390498+0000", 2024-09-14T05:30:17.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0615999999999997e-05 2024-09-14T05:30:17.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390799+0000", 2024-09-14T05:30:17.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030131899999999998 2024-09-14T05:30:17.937 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.937 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.937 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.937 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3243 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.937 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:17.937 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.571055739999998, 2024-09-14T05:30:17.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061728980000000003, 2024-09-14T05:30:17.937 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.937 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.937 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.937 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.938 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.938 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.938 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3243 2024-09-14T05:30:17.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.938 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:17.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:17.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329692+0000", 2024-09-14T05:30:17.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000163948 2024-09-14T05:30:17.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329696+0000", 2024-09-14T05:30:17.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5540000000000002e-06 2024-09-14T05:30:17.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329706+0000", 2024-09-14T05:30:17.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7960000000000007e-06 2024-09-14T05:30:17.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329712+0000", 2024-09-14T05:30:17.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6109999999999999e-06 2024-09-14T05:30:17.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390861+0000", 2024-09-14T05:30:17.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061148767 2024-09-14T05:30:17.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390942+0000", 2024-09-14T05:30:17.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1266000000000004e-05 2024-09-14T05:30:17.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391258+0000", 2024-09-14T05:30:17.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031503800000000002 2024-09-14T05:30:17.941 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.941 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.941 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.941 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3244 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14778368~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.941 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:17.942 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.570860691999997, 2024-09-14T05:30:17.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062050594000000001, 2024-09-14T05:30:17.942 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.942 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.942 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.942 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3244 2024-09-14T05:30:17.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.942 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:17.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:17.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329916+0000", 2024-09-14T05:30:17.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019290799999999999 2024-09-14T05:30:17.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329919+0000", 2024-09-14T05:30:17.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9019999999999999e-06 2024-09-14T05:30:17.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329926+0000", 2024-09-14T05:30:17.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9e-06 2024-09-14T05:30:17.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329932+0000", 2024-09-14T05:30:17.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.088e-06 2024-09-14T05:30:17.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391299+0000", 2024-09-14T05:30:17.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061366136000000002 2024-09-14T05:30:17.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391391+0000", 2024-09-14T05:30:17.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2204e-05 2024-09-14T05:30:17.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391774+0000", 2024-09-14T05:30:17.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038345600000000001 2024-09-14T05:30:17.945 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.946 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.946 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.946 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3921 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 5308416~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.946 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:17.946 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.499316882000002, 2024-09-14T05:30:17.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060842694000000003, 2024-09-14T05:30:17.946 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.946 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.946 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.947 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.947 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3921 2024-09-14T05:30:17.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.947 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:17.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:17.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401705+0000", 2024-09-14T05:30:17.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768499999999999 2024-09-14T05:30:17.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401709+0000", 2024-09-14T05:30:17.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0319999999999997e-06 2024-09-14T05:30:17.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401720+0000", 2024-09-14T05:30:17.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1005000000000001e-05 2024-09-14T05:30:17.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401727+0000", 2024-09-14T05:30:17.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1879999999999996e-06 2024-09-14T05:30:17.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461748+0000", 2024-09-14T05:30:17.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060021199999999997 2024-09-14T05:30:17.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461793+0000", 2024-09-14T05:30:17.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4530999999999998e-05 2024-09-14T05:30:17.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462110+0000", 2024-09-14T05:30:17.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031705299999999997 2024-09-14T05:30:17.950 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.950 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.950 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.950 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3923 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 6291456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.950 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:17.950 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.498670853999997, 2024-09-14T05:30:17.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060891273000000003, 2024-09-14T05:30:17.951 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.951 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.951 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.951 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3923 2024-09-14T05:30:17.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.951 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:17.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:17.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407054+0000", 2024-09-14T05:30:17.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005140787 2024-09-14T05:30:17.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407059+0000", 2024-09-14T05:30:17.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.04e-06 2024-09-14T05:30:17.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407069+0000", 2024-09-14T05:30:17.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4429999999999992e-06 2024-09-14T05:30:17.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407076+0000", 2024-09-14T05:30:17.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1679999999999998e-06 2024-09-14T05:30:17.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462514+0000", 2024-09-14T05:30:17.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055438551000000003 2024-09-14T05:30:17.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462562+0000", 2024-09-14T05:30:17.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7649999999999999e-05 2024-09-14T05:30:17.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462805+0000", 2024-09-14T05:30:17.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024263400000000001 2024-09-14T05:30:17.955 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.955 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.955 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.955 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4052 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 12550144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.955 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:17.955 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.483098845000001, 2024-09-14T05:30:17.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060872904999999998, 2024-09-14T05:30:17.955 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.955 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.955 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.955 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.955 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.956 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4052 2024-09-14T05:30:17.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.956 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:17.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:17.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417486+0000", 2024-09-14T05:30:17.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-14T05:30:17.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417487+0000", 2024-09-14T05:30:17.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4499999999999995e-07 2024-09-14T05:30:17.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417489+0000", 2024-09-14T05:30:17.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8899999999999999e-06 2024-09-14T05:30:17.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417492+0000", 2024-09-14T05:30:17.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4389999999999999e-06 2024-09-14T05:30:17.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478027+0000", 2024-09-14T05:30:17.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060534959999999999 2024-09-14T05:30:17.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478079+0000", 2024-09-14T05:30:17.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2104999999999997e-05 2024-09-14T05:30:17.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478358+0000", 2024-09-14T05:30:17.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000279351 2024-09-14T05:30:17.959 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.959 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.959 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.959 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4053 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13041664~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.959 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:17.959 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.483089460999999, 2024-09-14T05:30:17.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061258866000000002, 2024-09-14T05:30:17.960 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.960 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.960 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.960 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4053 2024-09-14T05:30:17.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.960 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:17.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:17.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:17.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.17e-07 2024-09-14T05:30:17.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417496+0000", 2024-09-14T05:30:17.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0890000000000001e-06 2024-09-14T05:30:17.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417499+0000", 2024-09-14T05:30:17.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8629999999999999e-06 2024-09-14T05:30:17.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417502+0000", 2024-09-14T05:30:17.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6290000000000002e-06 2024-09-14T05:30:17.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478387+0000", 2024-09-14T05:30:17.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060885433000000003 2024-09-14T05:30:17.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478435+0000", 2024-09-14T05:30:17.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8167999999999997e-05 2024-09-14T05:30:17.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478754+0000", 2024-09-14T05:30:17.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031846700000000001 2024-09-14T05:30:17.963 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.963 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.964 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.964 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4054 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13533184~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.964 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:17.964 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.483079576999998, 2024-09-14T05:30:17.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061604694000000002, 2024-09-14T05:30:17.964 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.964 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.964 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.964 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.964 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.964 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.964 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4054 2024-09-14T05:30:17.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.965 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:17.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:17.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:17.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.04e-07 2024-09-14T05:30:17.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417506+0000", 2024-09-14T05:30:17.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8299999999999995e-07 2024-09-14T05:30:17.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417509+0000", 2024-09-14T05:30:17.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.88e-06 2024-09-14T05:30:17.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417511+0000", 2024-09-14T05:30:17.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5950000000000001e-06 2024-09-14T05:30:17.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478784+0000", 2024-09-14T05:30:17.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061272971000000002 2024-09-14T05:30:17.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478841+0000", 2024-09-14T05:30:17.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-14T05:30:17.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479109+0000", 2024-09-14T05:30:17.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026794600000000003 2024-09-14T05:30:17.968 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.968 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.968 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.968 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4055 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14024704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.968 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:17.968 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.483069874999998, 2024-09-14T05:30:17.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061932299000000003, 2024-09-14T05:30:17.969 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.969 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.969 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.969 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4055 2024-09-14T05:30:17.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.969 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:17.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:17.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417515+0000", 2024-09-14T05:30:17.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-14T05:30:17.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417516+0000", 2024-09-14T05:30:17.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7199999999999997e-07 2024-09-14T05:30:17.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417519+0000", 2024-09-14T05:30:17.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.937e-06 2024-09-14T05:30:17.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417522+0000", 2024-09-14T05:30:17.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1420000000000002e-06 2024-09-14T05:30:17.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479140+0000", 2024-09-14T05:30:17.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061617933999999999 2024-09-14T05:30:17.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479188+0000", 2024-09-14T05:30:17.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8717000000000001e-05 2024-09-14T05:30:17.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479447+0000", 2024-09-14T05:30:17.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025838800000000002 2024-09-14T05:30:17.972 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.972 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.972 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.973 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4056 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14516224~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.973 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:17.973 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.483059701999998, 2024-09-14T05:30:17.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062263185999999998, 2024-09-14T05:30:17.973 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.973 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.973 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.973 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4056 2024-09-14T05:30:17.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.973 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:17.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:17.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:17.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8700000000000002e-07 2024-09-14T05:30:17.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417528+0000", 2024-09-14T05:30:17.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.579e-06 2024-09-14T05:30:17.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417533+0000", 2024-09-14T05:30:17.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2400000000000001e-06 2024-09-14T05:30:17.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417536+0000", 2024-09-14T05:30:17.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0309999999999999e-06 2024-09-14T05:30:17.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479477+0000", 2024-09-14T05:30:17.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061940967 2024-09-14T05:30:17.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479523+0000", 2024-09-14T05:30:17.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6335000000000003e-05 2024-09-14T05:30:17.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479788+0000", 2024-09-14T05:30:17.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026474700000000002 2024-09-14T05:30:17.977 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.977 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.977 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.977 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4057 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15007744~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.977 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:17.977 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.483045240999999, 2024-09-14T05:30:17.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062589928000000003, 2024-09-14T05:30:17.977 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.977 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.977 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.978 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.978 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.978 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.978 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4057 2024-09-14T05:30:17.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.978 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:17.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:17.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422838+0000", 2024-09-14T05:30:17.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052990349999999997 2024-09-14T05:30:17.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422854+0000", 2024-09-14T05:30:17.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5523999999999999e-05 2024-09-14T05:30:17.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422864+0000", 2024-09-14T05:30:17.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0084e-05 2024-09-14T05:30:17.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422872+0000", 2024-09-14T05:30:17.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-14T05:30:17.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479813+0000", 2024-09-14T05:30:17.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056941389000000002 2024-09-14T05:30:17.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479868+0000", 2024-09-14T05:30:17.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4973999999999999e-05 2024-09-14T05:30:17.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.480129+0000", 2024-09-14T05:30:17.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026084200000000001 2024-09-14T05:30:17.981 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.981 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.981 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.982 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4171 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14057472~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.982 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:17.982 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.471047681000002, 2024-09-14T05:30:17.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061184929999999998, 2024-09-14T05:30:17.982 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.982 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.982 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.982 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4171 2024-09-14T05:30:17.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.982 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:17.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:17.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:17.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.39e-07 2024-09-14T05:30:17.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429541+0000", 2024-09-14T05:30:17.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:30:17.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429547+0000", 2024-09-14T05:30:17.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3280000000000002e-06 2024-09-14T05:30:17.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429552+0000", 2024-09-14T05:30:17.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4859999999999997e-06 2024-09-14T05:30:17.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490379+0000", 2024-09-14T05:30:17.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060826681 2024-09-14T05:30:17.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490434+0000", 2024-09-14T05:30:17.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4577999999999998e-05 2024-09-14T05:30:17.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490722+0000", 2024-09-14T05:30:17.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028781400000000001 2024-09-14T05:30:17.985 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.986 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.986 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.986 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4172 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14548992~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.986 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:17.986 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.471025673, 2024-09-14T05:30:17.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061498256000000001, 2024-09-14T05:30:17.986 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.986 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.986 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.987 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4172 2024-09-14T05:30:17.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.987 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:17.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:17.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:17.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.72e-07 2024-09-14T05:30:17.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429561+0000", 2024-09-14T05:30:17.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.717e-06 2024-09-14T05:30:17.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429566+0000", 2024-09-14T05:30:17.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6529999999999999e-06 2024-09-14T05:30:17.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429571+0000", 2024-09-14T05:30:17.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5979999999999999e-06 2024-09-14T05:30:17.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490751+0000", 2024-09-14T05:30:17.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061180336000000002 2024-09-14T05:30:17.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490798+0000", 2024-09-14T05:30:17.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6875000000000001e-05 2024-09-14T05:30:17.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491057+0000", 2024-09-14T05:30:17.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025870500000000002 2024-09-14T05:30:17.990 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.990 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.990 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.991 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4173 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15040512~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:17.991 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:17.991 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.471008046999998, 2024-09-14T05:30:17.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061894820000000003, 2024-09-14T05:30:17.991 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:17.991 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:17.991 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:17.991 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:17.991 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:17.991 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:17.991 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4173 2024-09-14T05:30:17.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.991 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:17.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:17.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:17.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:17.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:17.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:17.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:17.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429577+0000", 2024-09-14T05:30:17.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-14T05:30:17.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:17.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429578+0000", 2024-09-14T05:30:17.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6980000000000001e-06 2024-09-14T05:30:17.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:17.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429585+0000", 2024-09-14T05:30:17.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.725e-06 2024-09-14T05:30:17.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:17.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429590+0000", 2024-09-14T05:30:17.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.938e-06 2024-09-14T05:30:17.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:17.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491091+0000", 2024-09-14T05:30:17.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061500773000000002 2024-09-14T05:30:17.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:17.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491136+0000", 2024-09-14T05:30:17.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5284000000000002e-05 2024-09-14T05:30:17.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:17.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:17.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:17.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491471+0000", 2024-09-14T05:30:17.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033485699999999998 2024-09-14T05:30:17.995 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.995 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.995 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.995 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:17.995 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:17.995 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:17.997 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:30:17.998 INFO:teuthology.orchestra.run.smithi071.stdout:5014 read 0xcaab04 thru 0xcb9c6e (0xf16b bytes) 2024-09-14T05:30:17.998 INFO:teuthology.orchestra.run.smithi071.stdout:5015 trunc from 0x208bbbc to 0xae6c465 2024-09-14T05:30:17.998 INFO:teuthology.orchestra.run.smithi071.stdout:5016 read 0x9d8d1b4 thru 0x9d948bc (0x7709 bytes) 2024-09-14T05:30:17.998 INFO:teuthology.orchestra.run.smithi071.stdout:5017 writesame 0x4b5b499 thru 0x4b69a71 data_size 0x1(0xe5d9 bytes) 2024-09-14T05:30:17.998 INFO:teuthology.orchestra.run.smithi071.stdout:5018 read 0x151960d thru 0x151f765 (0x6159 bytes) 2024-09-14T05:30:17.998 INFO:teuthology.orchestra.run.smithi071.stdout:5019 trunc from 0xae6c465 to 0xe3b21a1 2024-09-14T05:30:17.998 INFO:teuthology.orchestra.run.smithi071.stdout:5020 write 0xcbac2d8 thru 0xcbb132e (0x5057 bytes) 2024-09-14T05:30:17.998 INFO:teuthology.orchestra.run.smithi071.stdout:5022 read 0x2375eee thru 0x238519b (0xf2ae bytes) 2024-09-14T05:30:17.998 INFO:teuthology.orchestra.run.smithi071.stdout:5023 read 0xb42f80a thru 0xb43dc6d (0xe464 bytes) 2024-09-14T05:30:17.998 INFO:teuthology.orchestra.run.smithi071.stdout:5024 read 0x71e8f62 thru 0x71f0ac5 (0x7b64 bytes) 2024-09-14T05:30:17.998 INFO:teuthology.orchestra.run.smithi071.stdout:5026 read 0xac25d41 thru 0xac35090 (0xf350 bytes) 2024-09-14T05:30:17.998 INFO:teuthology.orchestra.run.smithi071.stdout:5027 trunc from 0xe3b21a1 to 0x8d3226b 2024-09-14T05:30:17.999 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:17.995+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:18.000 INFO:teuthology.orchestra.run.smithi071.stdout:5028 trunc from 0x8d3226b to 0xb46e2ec 2024-09-14T05:30:18.005 INFO:teuthology.orchestra.run.smithi071.stdout:5029 write 0xd7dcec thru 0xd83396 (0x56ab bytes) 2024-09-14T05:30:18.007 INFO:teuthology.orchestra.run.smithi071.stdout:5030 read 0x46c7f0b thru 0x46d0375 (0x846b bytes) 2024-09-14T05:30:18.007 INFO:teuthology.orchestra.run.smithi071.stdout:5033 write 0x8c4618f thru 0x8c49109 (0x2f7b bytes) 2024-09-14T05:30:18.011 INFO:teuthology.orchestra.run.smithi071.stdout:5034 write 0xa3aa73 thru 0xa441d2 (0x9760 bytes) 2024-09-14T05:30:18.043 INFO:teuthology.orchestra.run.smithi071.stdout:5035 trunc from 0xb46e2ec to 0x55863c5 2024-09-14T05:30:18.059 INFO:teuthology.orchestra.run.smithi071.stdout:5038 write 0x7763a5d thru 0x77699f7 (0x5f9b bytes) 2024-09-14T05:30:18.065 INFO:teuthology.orchestra.run.smithi071.stdout:5039 read 0x1c3c8dd thru 0x1c41b00 (0x5224 bytes) 2024-09-14T05:30:18.066 INFO:teuthology.orchestra.run.smithi071.stdout:5040 punch from 0x252e27e to 0x252e824, (0x5a6 bytes) 2024-09-14T05:30:18.068 INFO:teuthology.orchestra.run.smithi071.stdout:5042 trunc from 0x77699f8 to 0x9093261 2024-09-14T05:30:18.074 INFO:teuthology.orchestra.run.smithi071.stdout:5043 trunc from 0x9093261 to 0x962d02f 2024-09-14T05:30:18.078 INFO:teuthology.orchestra.run.smithi071.stdout:5045 punch from 0x6b70656 to 0x6b7ed60, (0xe70a bytes) 2024-09-14T05:30:18.079 INFO:teuthology.orchestra.run.smithi071.stdout:5046 writesame 0x75a484a thru 0x75b3cd8 data_size 0x29(0xf48f bytes) 2024-09-14T05:30:18.084 INFO:teuthology.orchestra.run.smithi071.stdout:5048 write 0x477d779 thru 0x4780c17 (0x349f bytes) 2024-09-14T05:30:18.093 INFO:teuthology.orchestra.run.smithi071.stdout:5049 writesame 0xa368d56 thru 0xa369066 data_size 0x1(0x311 bytes) 2024-09-14T05:30:18.093 INFO:teuthology.orchestra.run.smithi071.stdout:5050 read 0x4e3af13 thru 0x4e407a3 (0x5891 bytes) 2024-09-14T05:30:18.096 INFO:teuthology.orchestra.run.smithi071.stdout:5051 trunc from 0xa369067 to 0xc70fe40 2024-09-14T05:30:18.096 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:18.096+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:18.101 INFO:teuthology.orchestra.run.smithi071.stdout:5052 write 0xb0cdf82 thru 0xb0d76c3 (0x9742 bytes) 2024-09-14T05:30:18.110 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:18.110 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:18.110 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:18.110 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:18.119 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:30:18.121 INFO:teuthology.orchestra.run.smithi071.stdout:5053 trunc from 0xc70fe40 to 0x4630cc0 2024-09-14T05:30:18.135 INFO:teuthology.orchestra.run.smithi071.stdout:5055 writesame 0x2fbc7c9 thru 0x2fbf58e data_size 0x15(0x2dc6 bytes) 2024-09-14T05:30:18.140 INFO:teuthology.orchestra.run.smithi071.stdout:5058 read 0x3178c38 thru 0x3187538 (0xe901 bytes) 2024-09-14T05:30:18.140 INFO:teuthology.orchestra.run.smithi071.stdout:5060 punch from 0x3ce8791 to 0x3cecfab, (0x481a bytes) 2024-09-14T05:30:18.141 INFO:teuthology.orchestra.run.smithi071.stdout:5061 trunc from 0x4630cc0 to 0x5096a48 2024-09-14T05:30:18.146 INFO:teuthology.orchestra.run.smithi071.stdout:5062 punch from 0x3ad7a1d to 0x3adba78, (0x405b bytes) 2024-09-14T05:30:18.161 INFO:teuthology.orchestra.run.smithi071.stdout:5064 write 0xd13ae51 thru 0xd13e00b (0x31bb bytes) 2024-09-14T05:30:18.167 INFO:teuthology.orchestra.run.smithi071.stdout:5066 read 0x36afbd2 thru 0x36b4acf (0x4efe bytes) 2024-09-14T05:30:18.167 INFO:teuthology.orchestra.run.smithi071.stdout:5067 write 0xc6b039c thru 0xc6b4da6 (0x4a0b bytes) 2024-09-14T05:30:18.178 INFO:teuthology.orchestra.run.smithi071.stdout:5068 trunc from 0xd13e00c to 0xddf8a88 2024-09-14T05:30:18.183 INFO:teuthology.orchestra.run.smithi071.stdout:5069 read 0x3817ab8 thru 0x381cac8 (0x5011 bytes) 2024-09-14T05:30:18.188 INFO:teuthology.orchestra.run.smithi071.stdout:5070 write 0xe4fb36f thru 0xe4fcd3f (0x19d1 bytes) 2024-09-14T05:30:18.195 INFO:teuthology.orchestra.run.smithi071.stdout:5072 read 0x475323e thru 0x4760162 (0xcf25 bytes) 2024-09-14T05:30:18.195 INFO:teuthology.orchestra.run.smithi071.stdout:5073 read 0x64bbf58 thru 0x64bf6e1 (0x378a bytes) 2024-09-14T05:30:18.195 INFO:teuthology.orchestra.run.smithi071.stdout:5075 read 0x3911019 thru 0x391708d (0x6075 bytes) 2024-09-14T05:30:18.197 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:18.197+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:18.208 INFO:teuthology.orchestra.run.smithi071.stdout:5076 trunc from 0xe4fcd40 to 0x3d0dfbd 2024-09-14T05:30:18.228 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:18.229 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:18.229 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:18.229 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:18.235 INFO:teuthology.orchestra.run.smithi071.stdout:5078 write 0x913836b thru 0x913eaa6 (0x673c bytes) 2024-09-14T05:30:18.237 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:30:18.243 INFO:teuthology.orchestra.run.smithi071.stdout:5079 writesame 0x51133ce thru 0x511d092 data_size 0x1(0x9cc5 bytes) 2024-09-14T05:30:18.257 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.257 INFO:teuthology.orchestra.run.smithi071.stdout:5083 writesame 0x198b811 thru 0x199479f data_size 0xd(0x8f8f bytes) 2024-09-14T05:30:18.260 INFO:teuthology.orchestra.run.smithi071.stdout:5084 write 0x322c0b8 thru 0x322cdc9 (0xd12 bytes) 2024-09-14T05:30:18.261 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.261 INFO:teuthology.orchestra.run.smithi071.stdout:5086 writesame 0x7a27b59 thru 0x7a2fd77 data_size 0x1(0x821f bytes) 2024-09-14T05:30:18.272 INFO:teuthology.orchestra.run.smithi071.stdout:5089 write 0x2963d63 thru 0x2968aa9 (0x4d47 bytes) 2024-09-14T05:30:18.273 INFO:teuthology.orchestra.run.smithi071.stdout:5090 read 0x6663591 thru 0x6669486 (0x5ef6 bytes) 2024-09-14T05:30:18.281 INFO:teuthology.orchestra.run.smithi071.stdout:5091 trunc from 0x913eaa7 to 0xbf3de98 2024-09-14T05:30:18.296 INFO:teuthology.orchestra.run.smithi071.stdout:5095 trunc from 0xbf3de98 to 0x578f19 2024-09-14T05:30:18.324 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:18.324+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:18.378 INFO:teuthology.orchestra.run.smithi071.stdout:5097 write 0xc26487e thru 0xc2649e6 (0x169 bytes) 2024-09-14T05:30:18.381 INFO:teuthology.orchestra.run.smithi071.stdout:5098 read 0x5b75d49 thru 0x5b7ae96 (0x514e bytes) 2024-09-14T05:30:18.381 INFO:teuthology.orchestra.run.smithi071.stdout:5101 punch from 0xba5913a to 0xba641a2, (0xb068 bytes) 2024-09-14T05:30:18.386 INFO:teuthology.orchestra.run.smithi071.stdout:5102 write 0xc8a7f5f thru 0xc8b4230 (0xc2d2 bytes) 2024-09-14T05:30:18.398 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:18.397+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:18.399 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:18.399 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:18.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:18.400 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:18.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.400 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1205 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:18.400 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:18.400 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 48.200374607000001, 2024-09-14T05:30:18.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0086364279999999998, 2024-09-14T05:30:18.400 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.400 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.400 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:18.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:18.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:18.401 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:18.401 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1205 2024-09-14T05:30:18.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.401 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:18.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:18.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197891+0000", 2024-09-14T05:30:18.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.364e-06 2024-09-14T05:30:18.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197896+0000", 2024-09-14T05:30:18.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3000000000000001e-06 2024-09-14T05:30:18.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197910+0000", 2024-09-14T05:30:18.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3339000000000001e-05 2024-09-14T05:30:18.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197919+0000", 2024-09-14T05:30:18.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8869999999999998e-06 2024-09-14T05:30:18.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197974+0000", 2024-09-14T05:30:18.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5720000000000002e-05 2024-09-14T05:30:18.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.198094+0000", 2024-09-14T05:30:18.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119739 2024-09-14T05:30:18.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206526+0000", 2024-09-14T05:30:18.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084320790000000003 2024-09-14T05:30:18.404 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.404 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.404 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.404 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1220 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:18.404 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:18.405 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 48.191618243000001, 2024-09-14T05:30:18.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012822337, 2024-09-14T05:30:18.405 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.405 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.405 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:18.405 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:18.405 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:18.405 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:18.405 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1220 2024-09-14T05:30:18.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.405 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:18.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:18.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206647+0000", 2024-09-14T05:30:18.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4600000000000003e-07 2024-09-14T05:30:18.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206652+0000", 2024-09-14T05:30:18.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6619999999999996e-06 2024-09-14T05:30:18.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206662+0000", 2024-09-14T05:30:18.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.995e-06 2024-09-14T05:30:18.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206667+0000", 2024-09-14T05:30:18.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-14T05:30:18.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206706+0000", 2024-09-14T05:30:18.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9523000000000001e-05 2024-09-14T05:30:18.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206755+0000", 2024-09-14T05:30:18.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000002e-05 2024-09-14T05:30:18.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:18.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206877+0000", 2024-09-14T05:30:18.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121938 2024-09-14T05:30:18.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:18.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.207310+0000", 2024-09-14T05:30:18.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000432452 2024-09-14T05:30:18.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:18.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219364+0000", 2024-09-14T05:30:18.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012054481000000001 2024-09-14T05:30:18.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:18.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219425+0000", 2024-09-14T05:30:18.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0618000000000001e-05 2024-09-14T05:30:18.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219468+0000", 2024-09-14T05:30:18.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3739000000000003e-05 2024-09-14T05:30:18.410 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.410 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.410 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.411 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1243 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:18.411 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:18.411 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 48.178653015999998, 2024-09-14T05:30:18.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078660580000000004, 2024-09-14T05:30:18.411 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.411 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.411 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:18.411 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:18.411 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:18.411 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:18.411 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1243 2024-09-14T05:30:18.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.412 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:18.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:18.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219612+0000", 2024-09-14T05:30:18.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.09e-07 2024-09-14T05:30:18.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219616+0000", 2024-09-14T05:30:18.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.216e-06 2024-09-14T05:30:18.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219630+0000", 2024-09-14T05:30:18.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3064999999999999e-05 2024-09-14T05:30:18.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219637+0000", 2024-09-14T05:30:18.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7409999999999992e-06 2024-09-14T05:30:18.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219697+0000", 2024-09-14T05:30:18.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.02e-05 2024-09-14T05:30:18.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219833+0000", 2024-09-14T05:30:18.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013595600000000001 2024-09-14T05:30:18.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227477+0000", 2024-09-14T05:30:18.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076439710000000003 2024-09-14T05:30:18.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.415 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.415 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1264 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:18.415 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:18.415 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 48.170642223000002, 2024-09-14T05:30:18.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080867990000000004, 2024-09-14T05:30:18.415 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.415 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.416 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:18.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:18.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:18.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:18.416 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1264 2024-09-14T05:30:18.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.416 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:18.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:18.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227623+0000", 2024-09-14T05:30:18.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:30:18.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227627+0000", 2024-09-14T05:30:18.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.194e-06 2024-09-14T05:30:18.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227639+0000", 2024-09-14T05:30:18.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1296e-05 2024-09-14T05:30:18.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227648+0000", 2024-09-14T05:30:18.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2280000000000004e-06 2024-09-14T05:30:18.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227689+0000", 2024-09-14T05:30:18.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1329999999999999e-05 2024-09-14T05:30:18.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227741+0000", 2024-09-14T05:30:18.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1641000000000002e-05 2024-09-14T05:30:18.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:30:18.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227914+0000", 2024-09-14T05:30:18.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000173215 2024-09-14T05:30:18.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:18.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228474+0000", 2024-09-14T05:30:18.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055980199999999998 2024-09-14T05:30:18.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:18.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235608+0000", 2024-09-14T05:30:18.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0071343830000000002 2024-09-14T05:30:18.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:18.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235669+0000", 2024-09-14T05:30:18.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1068999999999994e-05 2024-09-14T05:30:18.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235709+0000", 2024-09-14T05:30:18.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9671e-05 2024-09-14T05:30:18.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.421 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.421 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1520 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:18.421 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:30:18.421 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 48.075634604000001, 2024-09-14T05:30:18.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061963629999999999, 2024-09-14T05:30:18.421 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.421 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.421 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:18.421 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:18.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:18.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:18.422 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1520 2024-09-14T05:30:18.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.422 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:30:18.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:30:18.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322631+0000", 2024-09-14T05:30:18.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.54e-07 2024-09-14T05:30:18.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322635+0000", 2024-09-14T05:30:18.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0450000000000002e-06 2024-09-14T05:30:18.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322646+0000", 2024-09-14T05:30:18.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1919999999999999e-05 2024-09-14T05:30:18.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322654+0000", 2024-09-14T05:30:18.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3139999999999998e-06 2024-09-14T05:30:18.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322704+0000", 2024-09-14T05:30:18.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0420000000000002e-05 2024-09-14T05:30:18.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322794+0000", 2024-09-14T05:30:18.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0037000000000005e-05 2024-09-14T05:30:18.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328826+0000", 2024-09-14T05:30:18.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060318730000000001 2024-09-14T05:30:18.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.425 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.425 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1526 3.4 3:2695cbe4:::rbd_data.112dd25fb4c.0000000000000119:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:18.425 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:18.425 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 48.072101660000001, 2024-09-14T05:30:18.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0079537359999999994, 2024-09-14T05:30:18.426 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.426 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.426 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:18.426 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:18.426 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:18.426 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:18.426 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1526 2024-09-14T05:30:18.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.426 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:18.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:18.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:18.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3600000000000003e-07 2024-09-14T05:30:18.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326168+0000", 2024-09-14T05:30:18.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3460000000000004e-06 2024-09-14T05:30:18.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326178+0000", 2024-09-14T05:30:18.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0291e-05 2024-09-14T05:30:18.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326183+0000", 2024-09-14T05:30:18.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-14T05:30:18.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326230+0000", 2024-09-14T05:30:18.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6650999999999998e-05 2024-09-14T05:30:18.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326295+0000", 2024-09-14T05:30:18.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5158000000000002e-05 2024-09-14T05:30:18.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:18.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326408+0000", 2024-09-14T05:30:18.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113039 2024-09-14T05:30:18.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.331648+0000", 2024-09-14T05:30:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052404929999999997 2024-09-14T05:30:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334011+0000", 2024-09-14T05:30:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002362669 2024-09-14T05:30:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334064+0000", 2024-09-14T05:30:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3198999999999997e-05 2024-09-14T05:30:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334116+0000", 2024-09-14T05:30:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2133999999999999e-05 2024-09-14T05:30:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1529 3.1 3:809746f7:::rbd_data.112dd25fb4c.0000000000000118:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:30:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 48.070337109999997, 2024-09-14T05:30:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061981060000000001, 2024-09-14T05:30:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.431 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1529 2024-09-14T05:30:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:30:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:30:18.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327928+0000", 2024-09-14T05:30:18.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0100000000000004e-07 2024-09-14T05:30:18.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327933+0000", 2024-09-14T05:30:18.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4689999999999999e-06 2024-09-14T05:30:18.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327944+0000", 2024-09-14T05:30:18.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1623999999999999e-05 2024-09-14T05:30:18.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327951+0000", 2024-09-14T05:30:18.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9759999999999998e-06 2024-09-14T05:30:18.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328003+0000", 2024-09-14T05:30:18.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1521999999999999e-05 2024-09-14T05:30:18.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328087+0000", 2024-09-14T05:30:18.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3988999999999998e-05 2024-09-14T05:30:18.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:18.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328304+0000", 2024-09-14T05:30:18.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000217254 2024-09-14T05:30:18.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:18.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.333363+0000", 2024-09-14T05:30:18.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050586499999999996 2024-09-14T05:30:18.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:18.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334026+0000", 2024-09-14T05:30:18.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066303499999999997 2024-09-14T05:30:18.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:18.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334080+0000", 2024-09-14T05:30:18.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4261000000000001e-05 2024-09-14T05:30:18.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334125+0000", 2024-09-14T05:30:18.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5525000000000002e-05 2024-09-14T05:30:18.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.437 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.437 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1531 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:18.437 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:18.437 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 48.069283493999997, 2024-09-14T05:30:18.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082114309999999999, 2024-09-14T05:30:18.437 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.437 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.437 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:18.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:18.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:18.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:18.438 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1531 2024-09-14T05:30:18.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.438 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:18.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:18.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328982+0000", 2024-09-14T05:30:18.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:30:18.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328985+0000", 2024-09-14T05:30:18.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7900000000000001e-06 2024-09-14T05:30:18.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328996+0000", 2024-09-14T05:30:18.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.028e-05 2024-09-14T05:30:18.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329001+0000", 2024-09-14T05:30:18.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0610000000000003e-06 2024-09-14T05:30:18.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329035+0000", 2024-09-14T05:30:18.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4366999999999998e-05 2024-09-14T05:30:18.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329082+0000", 2024-09-14T05:30:18.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6646e-05 2024-09-14T05:30:18.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:18.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329195+0000", 2024-09-14T05:30:18.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113279 2024-09-14T05:30:18.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:18.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334293+0000", 2024-09-14T05:30:18.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050980399999999999 2024-09-14T05:30:18.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:18.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337101+0000", 2024-09-14T05:30:18.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0028076170000000001 2024-09-14T05:30:18.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:18.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337154+0000", 2024-09-14T05:30:18.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3641999999999999e-05 2024-09-14T05:30:18.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337192+0000", 2024-09-14T05:30:18.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7932000000000002e-05 2024-09-14T05:30:18.443 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.443 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.443 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.443 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:2131 3.b e30/24 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head v 30'36, mlcod=30'34)", 2024-09-14T05:30:18.443 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:18.443 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.605936139000001, 2024-09-14T05:30:18.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031871709999999998, 2024-09-14T05:30:18.443 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.443 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.443 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:18.443 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:18.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:18.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792331+0000", 2024-09-14T05:30:18.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6900000000000001e-06 2024-09-14T05:30:18.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797319+0000", 2024-09-14T05:30:18.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049876349999999998 2024-09-14T05:30:18.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797349+0000", 2024-09-14T05:30:18.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0046e-05 2024-09-14T05:30:18.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797359+0000", 2024-09-14T05:30:18.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0037999999999999e-05 2024-09-14T05:30:18.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797445+0000", 2024-09-14T05:30:18.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6132999999999997e-05 2024-09-14T05:30:18.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797472+0000", 2024-09-14T05:30:18.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7478999999999999e-05 2024-09-14T05:30:18.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:18.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824161+0000", 2024-09-14T05:30:18.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026688714999999998 2024-09-14T05:30:18.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824200+0000", 2024-09-14T05:30:18.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8974000000000003e-05 2024-09-14T05:30:18.447 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.447 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.447 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.447 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4259 3.4 3:25f21dc9:::rbd_data.112d3d611e54.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 294912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:18.447 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:18.447 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.006763524999997, 2024-09-14T05:30:18.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010043553, 2024-09-14T05:30:18.447 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.448 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.448 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:18.448 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:18.448 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:18.448 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:18.448 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4259 2024-09-14T05:30:18.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.448 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:18.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:18.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391503+0000", 2024-09-14T05:30:18.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9099999999999999e-06 2024-09-14T05:30:18.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391821+0000", 2024-09-14T05:30:18.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031871800000000001 2024-09-14T05:30:18.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391844+0000", 2024-09-14T05:30:18.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2045e-05 2024-09-14T05:30:18.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391853+0000", 2024-09-14T05:30:18.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7249999999999992e-06 2024-09-14T05:30:18.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391917+0000", 2024-09-14T05:30:18.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3458000000000001e-05 2024-09-14T05:30:18.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392037+0000", 2024-09-14T05:30:18.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120449 2024-09-14T05:30:18.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:18.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392305+0000", 2024-09-14T05:30:18.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026739299999999999 2024-09-14T05:30:18.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:18.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.393609+0000", 2024-09-14T05:30:18.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013042450000000001 2024-09-14T05:30:18.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:18.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401430+0000", 2024-09-14T05:30:18.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078211670000000004 2024-09-14T05:30:18.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:18.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401499+0000", 2024-09-14T05:30:18.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9374000000000004e-05 2024-09-14T05:30:18.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401544+0000", 2024-09-14T05:30:18.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5068999999999998e-05 2024-09-14T05:30:18.453 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.453 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.453 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.453 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4452 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:18.453 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:30:18.453 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.515950916999998, 2024-09-14T05:30:18.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066061380000000001, 2024-09-14T05:30:18.454 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.454 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.454 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:18.454 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:18.454 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:18.454 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:18.454 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4452 2024-09-14T05:30:18.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.454 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:30:18.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:30:18.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882314+0000", 2024-09-14T05:30:18.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5899999999999998e-07 2024-09-14T05:30:18.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882315+0000", 2024-09-14T05:30:18.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6169999999999999e-06 2024-09-14T05:30:18.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882319+0000", 2024-09-14T05:30:18.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3629999999999998e-06 2024-09-14T05:30:18.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882324+0000", 2024-09-14T05:30:18.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1370000000000001e-06 2024-09-14T05:30:18.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883369+0000", 2024-09-14T05:30:18.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010448320000000001 2024-09-14T05:30:18.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883421+0000", 2024-09-14T05:30:18.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2401000000000002e-05 2024-09-14T05:30:18.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:18.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883638+0000", 2024-09-14T05:30:18.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021725600000000001 2024-09-14T05:30:18.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:18.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.884464+0000", 2024-09-14T05:30:18.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00082569800000000004 2024-09-14T05:30:18.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:18.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888860+0000", 2024-09-14T05:30:18.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0043963370000000002 2024-09-14T05:30:18.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:18.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888896+0000", 2024-09-14T05:30:18.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6022999999999997e-05 2024-09-14T05:30:18.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888920+0000", 2024-09-14T05:30:18.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3215000000000001e-05 2024-09-14T05:30:18.459 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.459 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.459 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.459 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4467 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:18.459 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:18.459 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.497582467000001, 2024-09-14T05:30:18.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0065700189999999999, 2024-09-14T05:30:18.460 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.460 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.460 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:18.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:18.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:18.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:18.460 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4467 2024-09-14T05:30:18.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.460 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:18.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:18.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:18.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-14T05:30:18.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900684+0000", 2024-09-14T05:30:18.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.629e-06 2024-09-14T05:30:18.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900688+0000", 2024-09-14T05:30:18.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8469999999999998e-06 2024-09-14T05:30:18.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900691+0000", 2024-09-14T05:30:18.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-14T05:30:18.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.901775+0000", 2024-09-14T05:30:18.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001084177 2024-09-14T05:30:18.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.901862+0000", 2024-09-14T05:30:18.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7167000000000001e-05 2024-09-14T05:30:18.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:18.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.902223+0000", 2024-09-14T05:30:18.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036068800000000001 2024-09-14T05:30:18.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:18.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.902843+0000", 2024-09-14T05:30:18.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000620489 2024-09-14T05:30:18.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:18.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907167+0000", 2024-09-14T05:30:18.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0043242649999999999 2024-09-14T05:30:18.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:18.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907217+0000", 2024-09-14T05:30:18.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9648000000000003e-05 2024-09-14T05:30:18.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907252+0000", 2024-09-14T05:30:18.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4780000000000002e-05 2024-09-14T05:30:18.465 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.465 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.465 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.465 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4554 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:18.465 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:18.465 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.413862713999997, 2024-09-14T05:30:18.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069998999999999999, 2024-09-14T05:30:18.465 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.466 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.466 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:18.466 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:18.466 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:18.466 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:18.466 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4554 2024-09-14T05:30:18.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.466 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:18.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:18.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:18.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9299999999999999e-07 2024-09-14T05:30:18.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984404+0000", 2024-09-14T05:30:18.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.855e-06 2024-09-14T05:30:18.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984407+0000", 2024-09-14T05:30:18.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-14T05:30:18.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984410+0000", 2024-09-14T05:30:18.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8159999999999998e-06 2024-09-14T05:30:18.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985511+0000", 2024-09-14T05:30:18.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00110053 2024-09-14T05:30:18.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985561+0000", 2024-09-14T05:30:18.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0389000000000002e-05 2024-09-14T05:30:18.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:18.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985772+0000", 2024-09-14T05:30:18.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000210571 2024-09-14T05:30:18.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:18.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.987075+0000", 2024-09-14T05:30:18.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001303128 2024-09-14T05:30:18.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:18.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991303+0000", 2024-09-14T05:30:18.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042283310000000001 2024-09-14T05:30:18.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:18.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991359+0000", 2024-09-14T05:30:18.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6335999999999997e-05 2024-09-14T05:30:18.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991402+0000", 2024-09-14T05:30:18.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2074999999999999e-05 2024-09-14T05:30:18.471 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.471 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.471 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.471 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4711 3.4 3:226df663:::rbd_data.112d3d611e54.000000000000002d:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:18.471 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:18.471 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.231191557999999, 2024-09-14T05:30:18.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011304193000000001, 2024-09-14T05:30:18.472 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.472 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.472 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:18.472 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:18.472 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:18.472 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:18.472 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4711 2024-09-14T05:30:18.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.472 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:18.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:18.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167074+0000", 2024-09-14T05:30:18.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0999999999999997e-07 2024-09-14T05:30:18.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167078+0000", 2024-09-14T05:30:18.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.878e-06 2024-09-14T05:30:18.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167087+0000", 2024-09-14T05:30:18.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6940000000000007e-06 2024-09-14T05:30:18.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167093+0000", 2024-09-14T05:30:18.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7740000000000001e-06 2024-09-14T05:30:18.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167128+0000", 2024-09-14T05:30:18.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5231999999999997e-05 2024-09-14T05:30:18.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167207+0000", 2024-09-14T05:30:18.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9036999999999996e-05 2024-09-14T05:30:18.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:18.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.169140+0000", 2024-09-14T05:30:18.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001932897 2024-09-14T05:30:18.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:18.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178258+0000", 2024-09-14T05:30:18.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091180819999999996 2024-09-14T05:30:18.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:18.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178314+0000", 2024-09-14T05:30:18.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5571e-05 2024-09-14T05:30:18.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:18.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178348+0000", 2024-09-14T05:30:18.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3747000000000001e-05 2024-09-14T05:30:18.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178377+0000", 2024-09-14T05:30:18.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9471e-05 2024-09-14T05:30:18.477 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.477 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.477 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.477 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4721 3.1 3:86302a6e:::rbd_data.112d3d611e54.0000000000000011:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:18.477 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:30:18.477 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.199206072000003, 2024-09-14T05:30:18.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066831750000000004, 2024-09-14T05:30:18.477 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.478 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.478 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:18.478 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:18.478 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:18.478 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:18.478 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4721 2024-09-14T05:30:18.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.478 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:30:18.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:30:18.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199059+0000", 2024-09-14T05:30:18.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.06e-07 2024-09-14T05:30:18.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199062+0000", 2024-09-14T05:30:18.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2279999999999998e-06 2024-09-14T05:30:18.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199072+0000", 2024-09-14T05:30:18.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5999999999999996e-06 2024-09-14T05:30:18.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199078+0000", 2024-09-14T05:30:18.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9109999999999998e-06 2024-09-14T05:30:18.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199123+0000", 2024-09-14T05:30:18.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5278000000000003e-05 2024-09-14T05:30:18.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199202+0000", 2024-09-14T05:30:18.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8521000000000003e-05 2024-09-14T05:30:18.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:18.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199271+0000", 2024-09-14T05:30:18.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9857999999999994e-05 2024-09-14T05:30:18.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:18.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199941+0000", 2024-09-14T05:30:18.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066984599999999998 2024-09-14T05:30:18.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:18.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205653+0000", 2024-09-14T05:30:18.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057117319999999997 2024-09-14T05:30:18.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:18.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205701+0000", 2024-09-14T05:30:18.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8254000000000003e-05 2024-09-14T05:30:18.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205741+0000", 2024-09-14T05:30:18.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0241000000000003e-05 2024-09-14T05:30:18.486 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.486 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.486 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.486 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4819 3.12 3:4e8efced:::rbd_data.112d3d611e54.000000000000000e:head [stat,truncate 1114112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:18.486 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:30:18.486 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.113891913000003, 2024-09-14T05:30:18.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062422770000000001, 2024-09-14T05:30:18.486 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.486 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.486 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:18.486 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:18.486 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:18.486 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:18.487 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4819 2024-09-14T05:30:18.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.487 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:30:18.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:30:18.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284373+0000", 2024-09-14T05:30:18.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3399999999999998e-07 2024-09-14T05:30:18.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284377+0000", 2024-09-14T05:30:18.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8539999999999999e-06 2024-09-14T05:30:18.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284388+0000", 2024-09-14T05:30:18.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1328999999999999e-05 2024-09-14T05:30:18.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284395+0000", 2024-09-14T05:30:18.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9759999999999998e-06 2024-09-14T05:30:18.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284723+0000", 2024-09-14T05:30:18.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032775799999999999 2024-09-14T05:30:18.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284801+0000", 2024-09-14T05:30:18.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7562e-05 2024-09-14T05:30:18.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:30:18.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284933+0000", 2024-09-14T05:30:18.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000131989 2024-09-14T05:30:18.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:18.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.287450+0000", 2024-09-14T05:30:18.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025177699999999999 2024-09-14T05:30:18.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:18.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290531+0000", 2024-09-14T05:30:18.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0030809959999999999 2024-09-14T05:30:18.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:18.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290588+0000", 2024-09-14T05:30:18.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6177000000000001e-05 2024-09-14T05:30:18.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290615+0000", 2024-09-14T05:30:18.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7132e-05 2024-09-14T05:30:18.491 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.492 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.492 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.492 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:7067 3.5 e35/24 3:a65d0cff:::rbd_data.112df7f2fe0a.0000000000000028:head v 35'89, mlcod=34'86)", 2024-09-14T05:30:18.492 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:18.492 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 43.255498641999999, 2024-09-14T05:30:18.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0064513080000000002, 2024-09-14T05:30:18.492 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.492 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.492 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:18.492 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:18.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:18.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:18.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:18.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142771+0000", 2024-09-14T05:30:18.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8949999999999997e-06 2024-09-14T05:30:18.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142779+0000", 2024-09-14T05:30:18.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8699999999999992e-06 2024-09-14T05:30:18.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142785+0000", 2024-09-14T05:30:18.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4899999999999997e-06 2024-09-14T05:30:18.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142843+0000", 2024-09-14T05:30:18.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7377000000000003e-05 2024-09-14T05:30:18.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142871+0000", 2024-09-14T05:30:18.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8305000000000001e-05 2024-09-14T05:30:18.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:18.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149177+0000", 2024-09-14T05:30:18.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063058639999999996 2024-09-14T05:30:18.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149217+0000", 2024-09-14T05:30:18.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0080999999999998e-05 2024-09-14T05:30:18.496 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.496 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.496 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.497 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:72582 3.b e47/24 3:d5072567:::rbd_data.112d4a90444c.0000000000000119:head v 47'386, mlcod=46'383)", 2024-09-14T05:30:18.497 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:18.497 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.873237705000001, 2024-09-14T05:30:18.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0067720940000000002, 2024-09-14T05:30:18.497 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.497 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.497 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:18.497 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:18.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:18.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:18.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1100000000000002e-07 2024-09-14T05:30:18.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525031+0000", 2024-09-14T05:30:18.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0740000000000003e-06 2024-09-14T05:30:18.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525040+0000", 2024-09-14T05:30:18.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5329999999999994e-06 2024-09-14T05:30:18.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525045+0000", 2024-09-14T05:30:18.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:30:18.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525090+0000", 2024-09-14T05:30:18.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4743000000000002e-05 2024-09-14T05:30:18.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525102+0000", 2024-09-14T05:30:18.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2067e-05 2024-09-14T05:30:18.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:18.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531757+0000", 2024-09-14T05:30:18.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066550380000000003 2024-09-14T05:30:18.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531799+0000", 2024-09-14T05:30:18.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1640000000000001e-05 2024-09-14T05:30:18.501 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.501 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.501 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.501 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:79671 3.15 e47/24 3:ab572765:::rbd_data.112d956b542f.0000000000000013:head v 47'120, mlcod=47'118)", 2024-09-14T05:30:18.501 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:30:18.501 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.247257486999999, 2024-09-14T05:30:18.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066700609999999997, 2024-09-14T05:30:18.501 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.501 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.501 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:18.502 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:30:18.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:30:18.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151008+0000", 2024-09-14T05:30:18.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:30:18.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151264+0000", 2024-09-14T05:30:18.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025656099999999999 2024-09-14T05:30:18.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151278+0000", 2024-09-14T05:30:18.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3288999999999999e-05 2024-09-14T05:30:18.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151287+0000", 2024-09-14T05:30:18.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7909999999999993e-06 2024-09-14T05:30:18.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151361+0000", 2024-09-14T05:30:18.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3979000000000004e-05 2024-09-14T05:30:18.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151383+0000", 2024-09-14T05:30:18.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2039999999999999e-05 2024-09-14T05:30:18.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:18.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.157542+0000", 2024-09-14T05:30:18.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006158915 2024-09-14T05:30:18.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.157677+0000", 2024-09-14T05:30:18.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013544300000000001 2024-09-14T05:30:18.505 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.505 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.505 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.505 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:123378 3.4 e71/39 3:24969376:::rbd_data.112d471e22e6.0000000000000009:head v 71'440, mlcod=69'438)", 2024-09-14T05:30:18.506 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:17.791627+0000", 2024-09-14T05:30:18.506 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.60663777500000005, 2024-09-14T05:30:18.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066686189999999998, 2024-09-14T05:30:18.506 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:18.506 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:18.506 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:18.506 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:18.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:18.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.791627+0000", 2024-09-14T05:30:18.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:18.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.791627+0000", 2024-09-14T05:30:18.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:18.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:18.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.791627+0000", 2024-09-14T05:30:18.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.65e-07 2024-09-14T05:30:18.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:18.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792806+0000", 2024-09-14T05:30:18.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001179193 2024-09-14T05:30:18.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:18.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792832+0000", 2024-09-14T05:30:18.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5514999999999999e-05 2024-09-14T05:30:18.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:18.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792842+0000", 2024-09-14T05:30:18.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0481e-05 2024-09-14T05:30:18.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:18.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792919+0000", 2024-09-14T05:30:18.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6720999999999999e-05 2024-09-14T05:30:18.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:18.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792945+0000", 2024-09-14T05:30:18.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6367999999999999e-05 2024-09-14T05:30:18.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:18.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.798239+0000", 2024-09-14T05:30:18.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052942509999999998 2024-09-14T05:30:18.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:18.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:18.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:18.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.798295+0000", 2024-09-14T05:30:18.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5825000000000002e-05 2024-09-14T05:30:18.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.510 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:18.510 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:18.510 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:18.510 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:30:18.512 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:30:18.513 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:18.511+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:18.514 INFO:teuthology.orchestra.run.smithi071.stdout:5103 writesame 0xd00a920 thru 0xd018cf2 data_size 0x1(0xe3d3 bytes) 2024-09-14T05:30:18.514 INFO:teuthology.orchestra.run.smithi071.stdout:5104 writesame 0x877fdd6 thru 0x8787c30 data_size 0x1(0x7e5b bytes) 2024-09-14T05:30:18.515 INFO:teuthology.orchestra.run.smithi071.stdout:5105 punch from 0x244b39b to 0x245a7e9, (0xf44e bytes) 2024-09-14T05:30:18.515 INFO:teuthology.orchestra.run.smithi071.stdout:5106 trunc from 0xd018cf3 to 0x28f5c7c 2024-09-14T05:30:18.515 INFO:teuthology.orchestra.run.smithi071.stdout:5107 write 0x9b3ebfd thru 0x9b44897 (0x5c9b bytes) 2024-09-14T05:30:18.515 INFO:teuthology.orchestra.run.smithi071.stdout:5108 read 0x2e6155d thru 0x2e663d1 (0x4e75 bytes) 2024-09-14T05:30:18.515 INFO:teuthology.orchestra.run.smithi071.stdout:5110 read 0x19d09e9 thru 0x19d2342 (0x195a bytes) 2024-09-14T05:30:18.515 INFO:teuthology.orchestra.run.smithi071.stdout:5112 write 0x3a3c8f4 thru 0x3a46af1 (0xa1fe bytes) 2024-09-14T05:30:18.515 INFO:teuthology.orchestra.run.smithi071.stdout:5113 trunc from 0x9b44898 to 0x3ee237c 2024-09-14T05:30:18.515 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.515 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.515 INFO:teuthology.orchestra.run.smithi071.stdout:5118 read 0x2104f59 thru 0x2112175 (0xd21d bytes) 2024-09-14T05:30:18.515 INFO:teuthology.orchestra.run.smithi071.stdout:5121 write 0x55d6f69 thru 0x55e3af8 (0xcb90 bytes) 2024-09-14T05:30:18.515 INFO:teuthology.orchestra.run.smithi071.stdout:5122 punch from 0x51318fe to 0x5137dd7, (0x64d9 bytes) 2024-09-14T05:30:18.515 INFO:teuthology.orchestra.run.smithi071.stdout:5123 read 0x2724892 thru 0x272cf3b (0x86aa bytes) 2024-09-14T05:30:18.515 INFO:teuthology.orchestra.run.smithi071.stdout:5124 write 0x6130157 thru 0x61304fe (0x3a8 bytes) 2024-09-14T05:30:18.516 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.516 INFO:teuthology.orchestra.run.smithi071.stdout:5126 read 0x46aab7b thru 0x46ba413 (0xf899 bytes) 2024-09-14T05:30:18.516 INFO:teuthology.orchestra.run.smithi071.stdout:5127 writesame 0x69ca059 thru 0x69cfa14 data_size 0x4(0x59bc bytes) 2024-09-14T05:30:18.516 INFO:teuthology.orchestra.run.smithi071.stdout:5128 punch from 0xf873e to 0x103221, (0xaae3 bytes) 2024-09-14T05:30:18.516 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.516 INFO:teuthology.orchestra.run.smithi071.stdout:5132 punch from 0x4b3d715 to 0x4b45ece, (0x87b9 bytes) 2024-09-14T05:30:18.516 INFO:teuthology.orchestra.run.smithi071.stdout:5133 read 0x61794ce thru 0x61816f4 (0x8227 bytes) 2024-09-14T05:30:18.516 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.517 INFO:teuthology.orchestra.run.smithi071.stdout:5135 trunc from 0x69cfa15 to 0x9b9456f 2024-09-14T05:30:18.522 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.523 INFO:teuthology.orchestra.run.smithi071.stdout:5137 trunc from 0x9b9456f to 0xa8a45e9 2024-09-14T05:30:18.528 INFO:teuthology.orchestra.run.smithi071.stdout:5138 write 0x1c8c7bd thru 0x1c8d53a (0xd7e bytes) 2024-09-14T05:30:18.531 INFO:teuthology.orchestra.run.smithi071.stdout:5139 writesame 0x396d040 thru 0x3975773 data_size 0x22(0x8734 bytes) 2024-09-14T05:30:18.533 INFO:teuthology.orchestra.run.smithi071.stdout:5140 read 0xdd8cb5 thru 0xddf8f7 (0x6c43 bytes) 2024-09-14T05:30:18.534 INFO:teuthology.orchestra.run.smithi071.stdout:5142 read 0x87bc413 thru 0x87cb559 (0xf147 bytes) 2024-09-14T05:30:18.534 INFO:teuthology.orchestra.run.smithi071.stdout:5145 writesame 0x58bf1bd thru 0x58cdbb2 data_size 0x1(0xe9f6 bytes) 2024-09-14T05:30:18.547 INFO:teuthology.orchestra.run.smithi071.stdout:5146 punch from 0x8c24f06 to 0x8c3351a, (0xe614 bytes) 2024-09-14T05:30:18.560 INFO:teuthology.orchestra.run.smithi071.stdout:5147 trunc from 0xa8a45e9 to 0x71ad12 2024-09-14T05:30:18.593 INFO:teuthology.orchestra.run.smithi071.stdout:5148 trunc from 0x71ad12 to 0xc30a758 2024-09-14T05:30:18.599 INFO:teuthology.orchestra.run.smithi071.stdout:5149 read 0x2b218b5 thru 0x2b2ee6f (0xd5bb bytes) 2024-09-14T05:30:18.599 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.599 INFO:teuthology.orchestra.run.smithi071.stdout:5151 read 0x5ae4748 thru 0x5af2bf9 (0xe4b2 bytes) 2024-09-14T05:30:18.599 INFO:teuthology.orchestra.run.smithi071.stdout:5152 writesame 0x4cf42b7 thru 0x4cfd323 data_size 0x1(0x906d bytes) 2024-09-14T05:30:18.612 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:18.611+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:18.625 INFO:teuthology.orchestra.run.smithi071.stdout:5153 write 0xcc6cab5 thru 0xcc78464 (0xb9b0 bytes) 2024-09-14T05:30:18.637 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.637 INFO:teuthology.orchestra.run.smithi071.stdout:5156 writesame 0x842e254 thru 0x843668d data_size 0x19(0x843a bytes) 2024-09-14T05:30:18.647 INFO:teuthology.orchestra.run.smithi071.stdout:5157 write 0x2c958bb thru 0x2ca4364 (0xeaaa bytes) 2024-09-14T05:30:18.657 INFO:teuthology.orchestra.run.smithi071.stdout:5159 punch from 0x5b7cb8f to 0x5b824b0, (0x5921 bytes) 2024-09-14T05:30:18.658 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.658 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.658 INFO:teuthology.orchestra.run.smithi071.stdout:5164 write 0xbad02d8 thru 0xbad2a16 (0x273f bytes) 2024-09-14T05:30:18.661 INFO:teuthology.orchestra.run.smithi071.stdout:5166 read 0x6d1cc47 thru 0x6d21728 (0x4ae2 bytes) 2024-09-14T05:30:18.662 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:18.662 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:18.662 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:18.662 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:18.663 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:18.663 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:18.669 INFO:teuthology.orchestra.run.smithi071.stdout:5167 write 0xed30f6a thru 0xed3220c (0x12a3 bytes) 2024-09-14T05:30:18.671 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:30:18.673 INFO:teuthology.orchestra.run.smithi071.stdout:5170 write 0x58cc50b thru 0x58d8c54 (0xc74a bytes) 2024-09-14T05:30:18.681 INFO:teuthology.orchestra.run.smithi071.stdout:5171 writesame 0x9e8f175 thru 0x9e9058f data_size 0x1(0x141b bytes) 2024-09-14T05:30:18.685 INFO:teuthology.orchestra.run.smithi071.stdout:5172 write 0x730b369 thru 0x730c728 (0x13c0 bytes) 2024-09-14T05:30:18.686 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:18.686 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:18.689 INFO:teuthology.orchestra.run.smithi071.stdout:5173 write 0x9b0d6b1 thru 0x9b1c9b9 (0xf309 bytes) 2024-09-14T05:30:18.698 INFO:teuthology.orchestra.run.smithi071.stdout:5175 punch from 0x90c1c8 to 0x913972, (0x77aa bytes) 2024-09-14T05:30:18.698 INFO:teuthology.orchestra.run.smithi071.stdout:5176 write 0xdb7ce54 thru 0xdb8b7b6 (0xe963 bytes) 2024-09-14T05:30:18.707 INFO:teuthology.orchestra.run.smithi071.stdout:5177 write 0xc132758 thru 0xc13c3c8 (0x9c71 bytes) 2024-09-14T05:30:18.711 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:18.711 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:18.712 INFO:teuthology.orchestra.run.smithi071.stdout:5179 write 0x1ad0847 thru 0x1ad6292 (0x5a4c bytes) 2024-09-14T05:30:18.712 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:18.712+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:18.714 INFO:teuthology.orchestra.run.smithi071.stdout:5180 write 0x567ddf5 thru 0x568c10d (0xe319 bytes) 2024-09-14T05:30:18.723 INFO:teuthology.orchestra.run.smithi071.stdout:5182 writesame 0x230db64 thru 0x231cd2b data_size 0xc(0xf1c8 bytes) 2024-09-14T05:30:18.731 INFO:teuthology.orchestra.run.smithi071.stdout:5183 write 0x1bfb7af thru 0x1c05f82 (0xa7d4 bytes) 2024-09-14T05:30:18.733 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:18.733 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:18.740 INFO:teuthology.orchestra.run.smithi071.stdout:5184 writesame 0x2ccabc9 thru 0x2ccb83d data_size 0x1(0xc75 bytes) 2024-09-14T05:30:18.742 INFO:teuthology.orchestra.run.smithi071.stdout:5186 read 0x6e1f451 thru 0x6e23a64 (0x4614 bytes) 2024-09-14T05:30:18.742 INFO:teuthology.orchestra.run.smithi071.stdout:5187 write 0x7b2fad0 thru 0x7b30977 (0xea8 bytes) 2024-09-14T05:30:18.747 INFO:teuthology.orchestra.run.smithi071.stdout:5188 writesame 0xbd22dd3 thru 0xbd2c320 data_size 0x1d(0x954e bytes) 2024-09-14T05:30:18.749 INFO:teuthology.orchestra.run.smithi071.stdout:5189 punch from 0xeac0d96 to 0xead06f3, (0xf95d bytes) 2024-09-14T05:30:18.750 INFO:teuthology.orchestra.run.smithi071.stdout:5190 read 0xc561b03 thru 0xc562fe3 (0x14e1 bytes) 2024-09-14T05:30:18.750 INFO:teuthology.orchestra.run.smithi071.stdout:5192 punch from 0xaaf7a11 to 0xaaff74b, (0x7d3a bytes) 2024-09-14T05:30:18.750 INFO:teuthology.orchestra.run.smithi071.stdout:5193 write 0xc528fb5 thru 0xc52be75 (0x2ec1 bytes) 2024-09-14T05:30:18.751 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.751 INFO:teuthology.orchestra.run.smithi071.stdout:5195 writesame 0x7651f91 thru 0x765d786 data_size 0x2f(0xb7f6 bytes) 2024-09-14T05:30:18.753 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:18.753 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:18.762 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.762 INFO:teuthology.orchestra.run.smithi071.stdout:5197 writesame 0x991762c thru 0x9924865 data_size 0x47(0xd23a bytes) 2024-09-14T05:30:18.766 INFO:teuthology.orchestra.run.smithi071.stdout:5198 writesame 0x5b91e59 thru 0x5b96fbe data_size 0x45(0x5166 bytes) 2024-09-14T05:30:18.774 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:18.774 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:18.785 INFO:teuthology.orchestra.run.smithi071.stdout:5200 trunc from 0xed3220d to 0x11d8d1f 2024-09-14T05:30:18.795 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:18.795 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:18.813 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:18.812+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:18.815 INFO:teuthology.orchestra.run.smithi071.stdout:5202 punch from 0x104abaa to 0x104f3fb, (0x4851 bytes) 2024-09-14T05:30:18.818 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:18.818 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:18.827 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:18.828 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:18.834 INFO:teuthology.orchestra.run.smithi071.stdout:5203 writesame 0xcbd333b thru 0xcbdd49e data_size 0x42(0xa164 bytes) 2024-09-14T05:30:18.835 INFO:teuthology.orchestra.run.smithi071.stdout:5204 write 0x709e82a thru 0x70a9d3f (0xb516 bytes) 2024-09-14T05:30:18.836 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:30:18.849 INFO:teuthology.orchestra.run.smithi071.stdout:5205 write 0x42d3077 thru 0x42df013 (0xbf9d bytes) 2024-09-14T05:30:18.861 INFO:teuthology.orchestra.run.smithi071.stdout:5207 read 0x105829c thru 0x1064e7b (0xcbe0 bytes) 2024-09-14T05:30:18.862 INFO:teuthology.orchestra.run.smithi071.stdout:5208 read 0xc9dd718 thru 0xc9dfb17 (0x2400 bytes) 2024-09-14T05:30:18.871 INFO:teuthology.orchestra.run.smithi071.stdout:5209 trunc from 0xcbdd49f to 0x4972d84 2024-09-14T05:30:18.884 INFO:teuthology.orchestra.run.smithi071.stdout:5210 read 0x2a56a21 thru 0x2a57a4b (0x102b bytes) 2024-09-14T05:30:18.884 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.884 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.884 INFO:teuthology.orchestra.run.smithi071.stdout:5213 read 0x46bc3ef thru 0x46c3207 (0x6e19 bytes) 2024-09-14T05:30:18.892 INFO:teuthology.orchestra.run.smithi071.stdout:5215 write 0x7897f18 thru 0x789b8db (0x39c4 bytes) 2024-09-14T05:30:18.901 INFO:teuthology.orchestra.run.smithi071.stdout:5217 write 0x6a15b38 thru 0x6a1cb17 (0x6fe0 bytes) 2024-09-14T05:30:18.908 INFO:teuthology.orchestra.run.smithi071.stdout:5218 read 0x39c95fe thru 0x39d9070 (0xfa73 bytes) 2024-09-14T05:30:18.908 INFO:teuthology.orchestra.run.smithi071.stdout:5220 read 0x909ac8 thru 0x914b38 (0xb071 bytes) 2024-09-14T05:30:18.909 INFO:teuthology.orchestra.run.smithi071.stdout:5221 write 0x67fb207 thru 0x67fe0c4 (0x2ebe bytes) 2024-09-14T05:30:18.914 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:18.913+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:18.918 INFO:teuthology.orchestra.run.smithi071.stdout:5223 write 0x7a9b302 thru 0x7aa7976 (0xc675 bytes) 2024-09-14T05:30:18.923 INFO:teuthology.orchestra.run.smithi071.stdout:5224 trunc from 0x7aa7977 to 0x7b4f19b 2024-09-14T05:30:18.927 INFO:teuthology.orchestra.run.smithi071.stdout:5225 write 0x17741b1 thru 0x1779f69 (0x5db9 bytes) 2024-09-14T05:30:18.929 INFO:teuthology.orchestra.run.smithi071.stdout:5227 writesame 0x149907a thru 0x14a41bf data_size 0x1(0xb146 bytes) 2024-09-14T05:30:18.940 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.954 INFO:teuthology.orchestra.run.smithi071.stdout:5229 writesame 0xe3d4bc8 thru 0xe3dc159 data_size 0x65(0x7592 bytes) 2024-09-14T05:30:18.954 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.954 INFO:teuthology.orchestra.run.smithi071.stdout:5231 read 0xdf3c7b7 thru 0xdf41274 (0x4abe bytes) 2024-09-14T05:30:18.954 INFO:teuthology.orchestra.run.smithi071.stdout:5232 write 0xe20312e thru 0xe20ca67 (0x993a bytes) 2024-09-14T05:30:18.961 INFO:teuthology.orchestra.run.smithi071.stdout:5233 punch from 0x8d7d021 to 0x8d8b4b3, (0xe492 bytes) 2024-09-14T05:30:18.961 INFO:teuthology.orchestra.run.smithi071.stdout:5234 punch from 0xb0f51c to 0xb115fb, (0x20df bytes) 2024-09-14T05:30:18.961 INFO:teuthology.orchestra.run.smithi071.stdout:5235 read 0xbcacb37 thru 0xbcb8214 (0xb6de bytes) 2024-09-14T05:30:18.961 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.962 INFO:teuthology.orchestra.run.smithi071.stdout:5238 writesame 0xa216d86 thru 0xa21ba26 data_size 0xd(0x4ca1 bytes) 2024-09-14T05:30:18.968 INFO:teuthology.orchestra.run.smithi071.stdout:5239 punch from 0x9f60756 to 0x9f62073, (0x191d bytes) 2024-09-14T05:30:18.969 INFO:teuthology.orchestra.run.smithi071.stdout:5240 write 0x1e587b7 thru 0x1e6442c (0xbc76 bytes) 2024-09-14T05:30:18.976 INFO:teuthology.orchestra.run.smithi071.stdout:5242 read 0x302c708 thru 0x303b797 (0xf090 bytes) 2024-09-14T05:30:18.977 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.977 INFO:teuthology.orchestra.run.smithi071.stdout:5245 writesame 0xa1092a thru 0xa133b4 data_size 0x1(0x2a8b bytes) 2024-09-14T05:30:18.980 INFO:teuthology.orchestra.run.smithi071.stdout:5247 read 0x6ef9e3a thru 0x6f00b9d (0x6d64 bytes) 2024-09-14T05:30:18.980 INFO:teuthology.orchestra.run.smithi071.stdout:5248 read 0x731f361 thru 0x732a866 (0xb506 bytes) 2024-09-14T05:30:18.981 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:18.989 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:18.989 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:18.991 INFO:teuthology.orchestra.run.smithi071.stdout:5250 trunc from 0xe3dc15a to 0x5a11d11 2024-09-14T05:30:18.999 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:30:19.006 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.006 INFO:teuthology.orchestra.run.smithi071.stdout:5252 read 0x16eb57f thru 0x16f3ad7 (0x8559 bytes) 2024-09-14T05:30:19.006 INFO:teuthology.orchestra.run.smithi071.stdout:5257 read 0x29ecfd6 thru 0x29fbe8d (0xeeb8 bytes) 2024-09-14T05:30:19.012 INFO:teuthology.orchestra.run.smithi071.stdout:5258 trunc from 0x5a11d11 to 0xc66233 2024-09-14T05:30:19.015 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:19.014+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:19.039 INFO:teuthology.orchestra.run.smithi071.stdout:5261 trunc from 0xc66233 to 0x7f70d30 2024-09-14T05:30:19.044 INFO:teuthology.orchestra.run.smithi071.stdout:5263 read 0x53f4821 thru 0x53f9773 (0x4f53 bytes) 2024-09-14T05:30:19.045 INFO:teuthology.orchestra.run.smithi071.stdout:5264 read 0x7bcfc74 thru 0x7bd673e (0x6acb bytes) 2024-09-14T05:30:19.045 INFO:teuthology.orchestra.run.smithi071.stdout:5265 write 0x33803d3 thru 0x338d9bc (0xd5ea bytes) 2024-09-14T05:30:19.056 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.070 INFO:teuthology.orchestra.run.smithi071.stdout:5268 writesame 0xe8bfd60 thru 0xe8cae6c data_size 0x31(0xb10d bytes) 2024-09-14T05:30:19.071 INFO:teuthology.orchestra.run.smithi071.stdout:5270 writesame 0xe670b21 thru 0xe671dc2 data_size 0x1(0x12a2 bytes) 2024-09-14T05:30:19.075 INFO:teuthology.orchestra.run.smithi071.stdout:5271 read 0x5fcbaca thru 0x5fd9871 (0xdda8 bytes) 2024-09-14T05:30:19.075 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.075 INFO:teuthology.orchestra.run.smithi071.stdout:5275 read 0x783ab5d thru 0x783ffe4 (0x5488 bytes) 2024-09-14T05:30:19.076 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.076 INFO:teuthology.orchestra.run.smithi071.stdout:5277 punch from 0x9c6d2a7 to 0x9c73ca5, (0x69fe bytes) 2024-09-14T05:30:19.076 INFO:teuthology.orchestra.run.smithi071.stdout:5278 read 0x5b60454 thru 0x5b619d0 (0x157d bytes) 2024-09-14T05:30:19.076 INFO:teuthology.orchestra.run.smithi071.stdout:5279 writesame 0xc6191e1 thru 0xc622c2a data_size 0x1d(0x9a4a bytes) 2024-09-14T05:30:19.086 INFO:teuthology.orchestra.run.smithi071.stdout:5280 read 0x9893d6d thru 0x9899140 (0x53d4 bytes) 2024-09-14T05:30:19.086 INFO:teuthology.orchestra.run.smithi071.stdout:5283 read 0x79f2dc3 thru 0x7a012d2 (0xe510 bytes) 2024-09-14T05:30:19.086 INFO:teuthology.orchestra.run.smithi071.stdout:5285 punch from 0x49a17ed to 0x49a2f76, (0x1789 bytes) 2024-09-14T05:30:19.086 INFO:teuthology.orchestra.run.smithi071.stdout:5288 write 0x6dbf352 thru 0x6dc1a2a (0x26d9 bytes) 2024-09-14T05:30:19.089 INFO:teuthology.orchestra.run.smithi071.stdout:5289 read 0xd39abb5 thru 0xd39f8b6 (0x4d02 bytes) 2024-09-14T05:30:19.090 INFO:teuthology.orchestra.run.smithi071.stdout:5290 writesame 0xa5f25da thru 0xa5f4803 data_size 0x1(0x222a bytes) 2024-09-14T05:30:19.100 INFO:teuthology.orchestra.run.smithi071.stdout:5292 punch from 0xb3a1a4b to 0xb3ae98d, (0xcf42 bytes) 2024-09-14T05:30:19.100 INFO:teuthology.orchestra.run.smithi071.stdout:5293 write 0x18a9c49 thru 0x18aaad5 (0xe8d bytes) 2024-09-14T05:30:19.100 INFO:teuthology.orchestra.run.smithi071.stdout:5294 punch from 0xdc29891 to 0xdc3828b, (0xe9fa bytes) 2024-09-14T05:30:19.100 INFO:teuthology.orchestra.run.smithi071.stdout:5295 read 0x991a45d thru 0x9921285 (0x6e29 bytes) 2024-09-14T05:30:19.100 INFO:teuthology.orchestra.run.smithi071.stdout:5297 write 0x7f4d148 thru 0x7f4f13c (0x1ff5 bytes) 2024-09-14T05:30:19.116 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:19.115+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:19.121 INFO:teuthology.orchestra.run.smithi071.stdout:5298 trunc from 0xe8cae6d to 0x1c2962d 2024-09-14T05:30:19.122 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:19.123 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:19.134 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:30:19.139 INFO:teuthology.orchestra.run.smithi071.stdout:5299 punch from 0x74c046 to 0x7514c4, (0x547e bytes) 2024-09-14T05:30:19.151 INFO:teuthology.orchestra.run.smithi071.stdout:5300 write 0x72ee0ed thru 0x72f8e09 (0xad1d bytes) 2024-09-14T05:30:19.159 INFO:teuthology.orchestra.run.smithi071.stdout:5301 write 0x5fd4651 thru 0x5fe35f8 (0xefa8 bytes) 2024-09-14T05:30:19.173 INFO:teuthology.orchestra.run.smithi071.stdout:5303 writesame 0x5e2a57e thru 0x5e30adc data_size 0x1(0x655f bytes) 2024-09-14T05:30:19.185 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.199 INFO:teuthology.orchestra.run.smithi071.stdout:5306 write 0xe67b804 thru 0xe688160 (0xc95d bytes) 2024-09-14T05:30:19.211 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.212 INFO:teuthology.orchestra.run.smithi071.stdout:5309 read 0xdfd2b54 thru 0xdfd5efb (0x33a8 bytes) 2024-09-14T05:30:19.212 INFO:teuthology.orchestra.run.smithi071.stdout:5310 writesame 0x39ce685 thru 0x39d9b04 data_size 0x8(0xb480 bytes) 2024-09-14T05:30:19.216 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:19.215+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:19.224 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.224 INFO:teuthology.orchestra.run.smithi071.stdout:5312 read 0x2a4d09d thru 0x2a55601 (0x8565 bytes) 2024-09-14T05:30:19.225 INFO:teuthology.orchestra.run.smithi071.stdout:5313 read 0x29c5e9b thru 0x29ce20f (0x8375 bytes) 2024-09-14T05:30:19.225 INFO:teuthology.orchestra.run.smithi071.stdout:5314 punch from 0xc7d785e to 0xc7dfff9, (0x879b bytes) 2024-09-14T05:30:19.225 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.225 INFO:teuthology.orchestra.run.smithi071.stdout:5316 punch from 0x9b4dca0 to 0x9b5b813, (0xdb73 bytes) 2024-09-14T05:30:19.231 INFO:teuthology.orchestra.run.smithi071.stdout:5317 trunc from 0xe688161 to 0xa31b54b 2024-09-14T05:30:19.242 INFO:teuthology.orchestra.run.smithi071.stdout:5319 punch from 0x42f72ed to 0x42fe59c, (0x72af bytes) 2024-09-14T05:30:19.242 INFO:teuthology.orchestra.run.smithi071.stdout:5320 write 0x128987a thru 0x12938ab (0xa032 bytes) 2024-09-14T05:30:19.245 INFO:teuthology.orchestra.run.smithi071.stdout:5322 read 0x369cf22 thru 0x36a6945 (0x9a24 bytes) 2024-09-14T05:30:19.245 INFO:teuthology.orchestra.run.smithi071.stdout:5324 punch from 0x2777ac2 to 0x2786227, (0xe765 bytes) 2024-09-14T05:30:19.245 INFO:teuthology.orchestra.run.smithi071.stdout:5326 write 0x5dc428d thru 0x5dd0bc4 (0xc938 bytes) 2024-09-14T05:30:19.253 INFO:teuthology.orchestra.run.smithi071.stdout:5327 trunc from 0xa31b54b to 0x671aeea 2024-09-14T05:30:19.274 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:19.274 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:19.280 INFO:teuthology.orchestra.run.smithi071.stdout:5329 write 0x9ad059f thru 0x9adbb78 (0xb5da bytes) 2024-09-14T05:30:19.285 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:30:19.295 INFO:teuthology.orchestra.run.smithi071.stdout:5330 trunc from 0x9adbb79 to 0x3b202b4 2024-09-14T05:30:19.312 INFO:teuthology.orchestra.run.smithi071.stdout:5333 punch from 0x37e8608 to 0x37f5122, (0xcb1a bytes) 2024-09-14T05:30:19.317 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:19.317+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:19.320 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T05:30:19.327 INFO:teuthology.orchestra.run.smithi071.stdout:5335 write 0xb11fd04 thru 0xb12354f (0x384c bytes) 2024-09-14T05:30:19.337 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.337 INFO:teuthology.orchestra.run.smithi071.stdout:5338 read 0x755a66 thru 0x75c395 (0x6930 bytes) 2024-09-14T05:30:19.337 INFO:teuthology.orchestra.run.smithi071.stdout:5339 read 0xabbe93b thru 0xabc3284 (0x494a bytes) 2024-09-14T05:30:19.338 INFO:teuthology.orchestra.run.smithi071.stdout:5340 write 0x8feff05 thru 0x8ffde2b (0xdf27 bytes) 2024-09-14T05:30:19.353 INFO:teuthology.orchestra.run.smithi071.stdout:5341 read 0x819cbed thru 0x81a3c8e (0x70a2 bytes) 2024-09-14T05:30:19.360 INFO:teuthology.orchestra.run.smithi071.stdout:5342 write 0xc580524 thru 0xc58d306 (0xcde3 bytes) 2024-09-14T05:30:19.370 INFO:teuthology.orchestra.run.smithi071.stdout:5344 writesame 0x73e3f2b thru 0x73eb938 data_size 0x11(0x7a0e bytes) 2024-09-14T05:30:19.378 INFO:teuthology.orchestra.run.smithi071.stdout:5345 write 0xaf0adb4 thru 0xaf16c0b (0xbe58 bytes) 2024-09-14T05:30:19.396 INFO:teuthology.orchestra.run.smithi071.stdout:5346 writesame 0xe634702 thru 0xe641dbe data_size 0x1(0xd6bd bytes) 2024-09-14T05:30:19.396 INFO:teuthology.orchestra.run.smithi071.stdout:5347 punch from 0x9a5e5bc to 0x9a65b0a, (0x754e bytes) 2024-09-14T05:30:19.396 INFO:teuthology.orchestra.run.smithi071.stdout:5348 write 0x76f17d2 thru 0x76f7a86 (0x62b5 bytes) 2024-09-14T05:30:19.398 INFO:teuthology.orchestra.run.smithi071.stdout:5349 read 0x346e4c2 thru 0x347cec6 (0xea05 bytes) 2024-09-14T05:30:19.398 INFO:teuthology.orchestra.run.smithi071.stdout:5350 read 0xcc8cc7d thru 0xcc954fb (0x887f bytes) 2024-09-14T05:30:19.398 INFO:teuthology.orchestra.run.smithi071.stdout:5352 writesame 0x2455ab1 thru 0x245b0f8 data_size 0x2c(0x5648 bytes) 2024-09-14T05:30:19.405 INFO:teuthology.orchestra.run.smithi071.stdout:5355 write 0xe0e67c thru 0xe1a0c9 (0xba4e bytes) 2024-09-14T05:30:19.410 INFO:teuthology.orchestra.run.smithi071.stdout:5356 trunc from 0xe641dbf to 0xc127724 2024-09-14T05:30:19.418 INFO:teuthology.orchestra.run.smithi071.stdout:5358 read 0x83d8188 thru 0x83e015d (0x7fd6 bytes) 2024-09-14T05:30:19.418 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:19.418+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:19.419 INFO:teuthology.orchestra.run.smithi071.stdout:5360 writesame 0x101d18c thru 0x101d341 data_size 0x3(0x1b6 bytes) 2024-09-14T05:30:19.419 INFO:teuthology.orchestra.run.smithi071.stdout:5361 writesame 0x207ab1 thru 0x214ae4 data_size 0xd(0xd034 bytes) 2024-09-14T05:30:19.424 INFO:teuthology.orchestra.run.smithi071.stdout:5363 write 0x27ad938 thru 0x27b3d0d (0x63d6 bytes) 2024-09-14T05:30:19.429 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.429 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.429 INFO:teuthology.orchestra.run.smithi071.stdout:5367 punch from 0x3d9c773 to 0x3da0bdf, (0x446c bytes) 2024-09-14T05:30:19.429 INFO:teuthology.orchestra.run.smithi071.stdout:5369 read 0x933f928 thru 0x93446cb (0x4da4 bytes) 2024-09-14T05:30:19.429 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.429 INFO:teuthology.orchestra.run.smithi071.stdout:5371 read 0x14f70a1 thru 0x1502df9 (0xbd59 bytes) 2024-09-14T05:30:19.429 INFO:teuthology.orchestra.run.smithi071.stdout:5372 write 0x7869003 thru 0x7872ad4 (0x9ad2 bytes) 2024-09-14T05:30:19.430 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:19.430 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:19.441 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:30:19.442 INFO:teuthology.orchestra.run.smithi071.stdout:5373 write 0xbe7e74e thru 0xbe7f3d2 (0xc85 bytes) 2024-09-14T05:30:19.450 INFO:teuthology.orchestra.run.smithi071.stdout:5374 write 0xe577081 thru 0xe57bf4d (0x4ecd bytes) 2024-09-14T05:30:19.455 INFO:teuthology.orchestra.run.smithi071.stdout:5375 punch from 0x26913d2 to 0x2698738, (0x7366 bytes) 2024-09-14T05:30:19.455 INFO:teuthology.orchestra.run.smithi071.stdout:5376 read 0x15d48e3 thru 0x15d70db (0x27f9 bytes) 2024-09-14T05:30:19.456 INFO:teuthology.orchestra.run.smithi071.stdout:5377 read 0xc668e98 thru 0xc6729a1 (0x9b0a bytes) 2024-09-14T05:30:19.456 INFO:teuthology.orchestra.run.smithi071.stdout:5378 read 0x61bc28a thru 0x61c0009 (0x3d80 bytes) 2024-09-14T05:30:19.456 INFO:teuthology.orchestra.run.smithi071.stdout:5381 read 0x8d28cec thru 0x8d31e60 (0x9175 bytes) 2024-09-14T05:30:19.457 INFO:teuthology.orchestra.run.smithi071.stdout:5382 writesame 0x199c5b1 thru 0x199de0f data_size 0x1(0x185f bytes) 2024-09-14T05:30:19.460 INFO:teuthology.orchestra.run.smithi071.stdout:5383 write 0x1aebbd9 thru 0x1afb79e (0xfbc6 bytes) 2024-09-14T05:30:19.471 INFO:teuthology.orchestra.run.smithi071.stdout:5384 trunc from 0xe57bf4e to 0x8e995e8 2024-09-14T05:30:19.488 INFO:teuthology.orchestra.run.smithi071.stdout:5386 read 0x22ec001 thru 0x22f7d08 (0xbd08 bytes) 2024-09-14T05:30:19.489 INFO:teuthology.orchestra.run.smithi071.stdout:5387 punch from 0x49139c to 0x498e60, (0x7ac4 bytes) 2024-09-14T05:30:19.489 INFO:teuthology.orchestra.run.smithi071.stdout:5388 writesame 0x134dee8 thru 0x135225a data_size 0x1(0x4373 bytes) 2024-09-14T05:30:19.494 INFO:teuthology.orchestra.run.smithi071.stdout:5389 punch from 0x129b168 to 0x12a4176, (0x900e bytes) 2024-09-14T05:30:19.503 INFO:teuthology.orchestra.run.smithi071.stdout:5390 write 0xc023ff7 thru 0xc02ac8f (0x6c99 bytes) 2024-09-14T05:30:19.511 INFO:teuthology.orchestra.run.smithi071.stdout:5393 read 0x3b158dd thru 0x3b1c569 (0x6c8d bytes) 2024-09-14T05:30:19.519 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:19.519+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:19.521 INFO:teuthology.orchestra.run.smithi071.stdout:5395 write 0xeb466cc thru 0xeb50055 (0x998a bytes) 2024-09-14T05:30:19.532 INFO:teuthology.orchestra.run.smithi071.stdout:5396 writesame 0x4574908 thru 0x4582a34 data_size 0x87(0xe12d bytes) 2024-09-14T05:30:19.543 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.543 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.543 INFO:teuthology.orchestra.run.smithi071.stdout:5401 write 0x837fcbf thru 0x83890ee (0x9430 bytes) 2024-09-14T05:30:19.553 INFO:teuthology.orchestra.run.smithi071.stdout:5403 write 0x3b85a78 thru 0x3b94529 (0xeab2 bytes) 2024-09-14T05:30:19.560 INFO:teuthology.orchestra.run.smithi071.stdout:5404 punch from 0x765b626 to 0x765bea3, (0x87d bytes) 2024-09-14T05:30:19.560 INFO:teuthology.orchestra.run.smithi071.stdout:5405 read 0x5921fee thru 0x592d430 (0xb443 bytes) 2024-09-14T05:30:19.560 INFO:teuthology.orchestra.run.smithi071.stdout:5406 punch from 0xe5b94bc to 0xe5be5e3, (0x5127 bytes) 2024-09-14T05:30:19.560 INFO:teuthology.orchestra.run.smithi071.stdout:5408 writesame 0x2a9e8cd thru 0x2aa73f0 data_size 0x1a(0x8b24 bytes) 2024-09-14T05:30:19.563 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.563 INFO:teuthology.orchestra.run.smithi071.stdout:5411 writesame 0xaf08103 thru 0xaf0ec8d data_size 0x17(0x6b8b bytes) 2024-09-14T05:30:19.572 INFO:teuthology.orchestra.run.smithi071.stdout:5412 read 0x1fce3e6 thru 0x1fd5aa5 (0x76c0 bytes) 2024-09-14T05:30:19.576 INFO:teuthology.orchestra.run.smithi071.stdout:5413 trunc from 0xeb50056 to 0xc60eb05 2024-09-14T05:30:19.585 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:19.586 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:19.589 INFO:teuthology.orchestra.run.smithi071.stdout:5414 punch from 0x4093a56 to 0x409b57c, (0x7b26 bytes) 2024-09-14T05:30:19.589 INFO:teuthology.orchestra.run.smithi071.stdout:5415 punch from 0x6c4cdf2 to 0x6c4d3f5, (0x603 bytes) 2024-09-14T05:30:19.590 INFO:teuthology.orchestra.run.smithi071.stdout:5416 read 0x65c9f6b thru 0x65ccbef (0x2c85 bytes) 2024-09-14T05:30:19.590 INFO:teuthology.orchestra.run.smithi071.stdout:5417 write 0x41ace4a thru 0x41b4b6d (0x7d24 bytes) 2024-09-14T05:30:19.596 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:30:19.599 INFO:teuthology.orchestra.run.smithi071.stdout:5418 trunc from 0xc60eb05 to 0x83a4b73 2024-09-14T05:30:19.615 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.620 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:19.619+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:19.621 INFO:teuthology.orchestra.run.smithi071.stdout:5421 write 0x87f978d thru 0x87ff938 (0x61ac bytes) 2024-09-14T05:30:19.624 INFO:teuthology.orchestra.run.smithi071.stdout:5423 punch from 0x812f575 to 0x813adab, (0xb836 bytes) 2024-09-14T05:30:19.626 INFO:teuthology.orchestra.run.smithi071.stdout:5424 trunc from 0x87ff939 to 0x6de7468 2024-09-14T05:30:19.654 INFO:teuthology.orchestra.run.smithi071.stdout:5430 writesame 0xa340a41 thru 0xa343f75 data_size 0x1(0x3535 bytes) 2024-09-14T05:30:19.654 INFO:teuthology.orchestra.run.smithi071.stdout:5431 write 0x7085764 thru 0x708b628 (0x5ec5 bytes) 2024-09-14T05:30:19.655 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.659 INFO:teuthology.orchestra.run.smithi071.stdout:5434 trunc from 0xa343f76 to 0xd4897ae 2024-09-14T05:30:19.665 INFO:teuthology.orchestra.run.smithi071.stdout:5435 read 0x3d1b525 thru 0x3d22973 (0x744f bytes) 2024-09-14T05:30:19.666 INFO:teuthology.orchestra.run.smithi071.stdout:5436 write 0x6cbd7db thru 0x6cc4772 (0x6f98 bytes) 2024-09-14T05:30:19.676 INFO:teuthology.orchestra.run.smithi071.stdout:5437 trunc from 0xd4897ae to 0x698e3e5 2024-09-14T05:30:19.686 INFO:teuthology.orchestra.run.smithi071.stdout:5438 read 0xe8334e thru 0xe882e5 (0x4f98 bytes) 2024-09-14T05:30:19.686 INFO:teuthology.orchestra.run.smithi071.stdout:5439 writesame 0x23e8a0d thru 0x23f8065 data_size 0x1(0xf659 bytes) 2024-09-14T05:30:19.696 INFO:teuthology.orchestra.run.smithi071.stdout:5440 punch from 0x59e1267 to 0x59f078a, (0xf523 bytes) 2024-09-14T05:30:19.696 INFO:teuthology.orchestra.run.smithi071.stdout:5441 punch from 0x42737cd to 0x427a9f0, (0x7223 bytes) 2024-09-14T05:30:19.696 INFO:teuthology.orchestra.run.smithi071.stdout:5442 read 0x26ddd3b thru 0x26e4589 (0x684f bytes) 2024-09-14T05:30:19.697 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.697 INFO:teuthology.orchestra.run.smithi071.stdout:5445 write 0x7f65d8 thru 0x7ffc3d (0x9666 bytes) 2024-09-14T05:30:19.707 INFO:teuthology.orchestra.run.smithi071.stdout:5447 write 0x8a847c5 thru 0x8a8ede3 (0xa61f bytes) 2024-09-14T05:30:19.716 INFO:teuthology.orchestra.run.smithi071.stdout:5448 read 0x8458007 thru 0x8466af1 (0xeaeb bytes) 2024-09-14T05:30:19.717 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.717 INFO:teuthology.orchestra.run.smithi071.stdout:5451 read 0x563352f thru 0x564234c (0xee1e bytes) 2024-09-14T05:30:19.721 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:19.720+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:19.723 INFO:teuthology.orchestra.run.smithi071.stdout:5453 write 0x91c0958 thru 0x91c1333 (0x9dc bytes) 2024-09-14T05:30:19.728 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:19.728 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:19.736 INFO:teuthology.orchestra.run.smithi071.stdout:5454 write 0xe8963e1 thru 0xe89bbd2 (0x57f2 bytes) 2024-09-14T05:30:19.739 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:30:19.744 INFO:teuthology.orchestra.run.smithi071.stdout:5455 trunc from 0xe89bbd3 to 0xc44a814 2024-09-14T05:30:19.754 INFO:teuthology.orchestra.run.smithi071.stdout:5456 writesame 0x477dbbc thru 0x478c4ff data_size 0x2(0xe944 bytes) 2024-09-14T05:30:19.761 INFO:teuthology.orchestra.run.smithi071.stdout:5457 punch from 0x6d079a9 to 0x6d11bea, (0xa241 bytes) 2024-09-14T05:30:19.761 INFO:teuthology.orchestra.run.smithi071.stdout:5459 write 0xa2ed719 thru 0xa2fafaa (0xd892 bytes) 2024-09-14T05:30:19.772 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.778 INFO:teuthology.orchestra.run.smithi071.stdout:5461 write 0xce19392 thru 0xce23d6b (0xa9da bytes) 2024-09-14T05:30:19.791 INFO:teuthology.orchestra.run.smithi071.stdout:5464 punch from 0x2e1224f to 0x2e1b078, (0x8e29 bytes) 2024-09-14T05:30:19.791 INFO:teuthology.orchestra.run.smithi071.stdout:5465 read 0x88274c3 thru 0x88293d1 (0x1f0f bytes) 2024-09-14T05:30:19.791 INFO:teuthology.orchestra.run.smithi071.stdout:5467 read 0x46d1d23 thru 0x46e17ae (0xfa8c bytes) 2024-09-14T05:30:19.791 INFO:teuthology.orchestra.run.smithi071.stdout:5468 read 0xac7d06c thru 0xac85a0a (0x899f bytes) 2024-09-14T05:30:19.791 INFO:teuthology.orchestra.run.smithi071.stdout:5469 write 0x6a35ea7 thru 0x6a45c90 (0xfdea bytes) 2024-09-14T05:30:19.794 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.795 INFO:teuthology.orchestra.run.smithi071.stdout:5473 write 0x11e9e6e thru 0x11ef9ef (0x5b82 bytes) 2024-09-14T05:30:19.796 INFO:teuthology.orchestra.run.smithi071.stdout:5474 punch from 0x608df8e to 0x6096456, (0x84c8 bytes) 2024-09-14T05:30:19.796 INFO:teuthology.orchestra.run.smithi071.stdout:5475 read 0x2ed4dc thru 0x2ee708 (0x122d bytes) 2024-09-14T05:30:19.797 INFO:teuthology.orchestra.run.smithi071.stdout:5476 write 0x64a6ff1 thru 0x64af7b1 (0x87c1 bytes) 2024-09-14T05:30:19.808 INFO:teuthology.orchestra.run.smithi071.stdout:5477 read 0xcc0d0b7 thru 0xcc1a11a (0xd064 bytes) 2024-09-14T05:30:19.808 INFO:teuthology.orchestra.run.smithi071.stdout:5478 read 0x19b701f thru 0x19c52f0 (0xe2d2 bytes) 2024-09-14T05:30:19.809 INFO:teuthology.orchestra.run.smithi071.stdout:5479 trunc from 0xce23d6c to 0xc667e6d 2024-09-14T05:30:19.816 INFO:teuthology.orchestra.run.smithi071.stdout:5481 punch from 0x474e7c7 to 0x475b5d9, (0xce12 bytes) 2024-09-14T05:30:19.822 INFO:teuthology.orchestra.run.smithi071.stdout:5482 writesame 0xca28d13 thru 0xca32812 data_size 0x28(0x9b00 bytes) 2024-09-14T05:30:19.822 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:19.821+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:19.832 INFO:teuthology.orchestra.run.smithi071.stdout:5483 trunc from 0xca32813 to 0x4f73c1c 2024-09-14T05:30:19.850 INFO:teuthology.orchestra.run.smithi071.stdout:5485 read 0xad6446 thru 0xae5e35 (0xf9f0 bytes) 2024-09-14T05:30:19.851 INFO:teuthology.orchestra.run.smithi071.stdout:5486 punch from 0x4e559fd to 0x4e5faec, (0xa0ef bytes) 2024-09-14T05:30:19.851 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.851 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.851 INFO:teuthology.orchestra.run.smithi071.stdout:5493 punch from 0x1bd0fa7 to 0x1bdc408, (0xb461 bytes) 2024-09-14T05:30:19.851 INFO:teuthology.orchestra.run.smithi071.stdout:5494 write 0x1cdbc33 thru 0x1ce973d (0xdb0b bytes) 2024-09-14T05:30:19.864 INFO:teuthology.orchestra.run.smithi071.stdout:5495 write 0x8da3fc6 thru 0x8dacdd2 (0x8e0d bytes) 2024-09-14T05:30:19.874 INFO:teuthology.orchestra.run.smithi071.stdout:5496 read 0x35b159b thru 0x35b55bd (0x4023 bytes) 2024-09-14T05:30:19.874 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.874 INFO:teuthology.orchestra.run.smithi071.stdout:5498 read 0xad6be thru 0xb799c (0xa2df bytes) 2024-09-14T05:30:19.875 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:19.875 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:19.883 INFO:teuthology.orchestra.run.smithi071.stdout:5499 writesame 0xb6410a9 thru 0xb64feec data_size 0x66(0xee44 bytes) 2024-09-14T05:30:19.883 INFO:teuthology.orchestra.run.smithi071.stdout:5502 read 0x8f98ede thru 0x8f98f84 (0xa7 bytes) 2024-09-14T05:30:19.884 INFO:teuthology.orchestra.run.smithi071.stdout:5503 punch from 0x74abd0e to 0x74b637f, (0xa671 bytes) 2024-09-14T05:30:19.884 INFO:teuthology.orchestra.run.smithi071.stdout:5504 read 0x29ff9db thru 0x2a0f9a6 (0xffcc bytes) 2024-09-14T05:30:19.884 INFO:teuthology.orchestra.run.smithi071.stdout:5505 write 0xff4798 thru 0x1003ef1 (0xf75a bytes) 2024-09-14T05:30:19.887 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:30:19.902 INFO:teuthology.orchestra.run.smithi071.stdout:5508 trunc from 0xb64feed to 0x88b092 2024-09-14T05:30:19.939 INFO:teuthology.orchestra.run.smithi071.stdout:5509 writesame 0x831dbb7 thru 0x832732f data_size 0x1(0x9779 bytes) 2024-09-14T05:30:19.939 INFO:teuthology.orchestra.run.smithi071.stdout:5510 read 0x21d53cf thru 0x21e1521 (0xc153 bytes) 2024-09-14T05:30:19.939 INFO:teuthology.orchestra.run.smithi071.stdout:5512 punch from 0x2ea1b46 to 0x2eb1154, (0xf60e bytes) 2024-09-14T05:30:19.939 INFO:teuthology.orchestra.run.smithi071.stdout:5513 write 0x1228c9e thru 0x122d612 (0x4975 bytes) 2024-09-14T05:30:19.939 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:19.938+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:19.941 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.945 INFO:teuthology.orchestra.run.smithi071.stdout:5517 trunc from 0x8327330 to 0x4ca4e8d 2024-09-14T05:30:19.950 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:19.950 INFO:teuthology.orchestra.run.smithi071.stdout:5519 read 0x3461c61 thru 0x347035e (0xe6fe bytes) 2024-09-14T05:30:19.959 INFO:teuthology.orchestra.run.smithi071.stdout:5520 write 0x799d57b thru 0x79a6de8 (0x986e bytes) 2024-09-14T05:30:19.979 INFO:teuthology.orchestra.run.smithi071.stdout:5522 writesame 0x9c7122d thru 0x9c752f8 data_size 0x1d(0x40cc bytes) 2024-09-14T05:30:19.979 INFO:teuthology.orchestra.run.smithi071.stdout:5524 read 0x233da6a thru 0x234a656 (0xcbed bytes) 2024-09-14T05:30:19.990 INFO:teuthology.orchestra.run.smithi071.stdout:5525 trunc from 0x9c752f9 to 0xa1cdcd 2024-09-14T05:30:20.006 INFO:teuthology.orchestra.run.smithi071.stdout:5526 punch from 0x2dfc1e to 0x2ea744, (0xab26 bytes) 2024-09-14T05:30:20.006 INFO:teuthology.orchestra.run.smithi071.stdout:5527 read 0x8e137a thru 0x8ef697 (0xe31e bytes) 2024-09-14T05:30:20.006 INFO:teuthology.orchestra.run.smithi071.stdout:5528 read 0x56da77 thru 0x5700c5 (0x264f bytes) 2024-09-14T05:30:20.006 INFO:teuthology.orchestra.run.smithi071.stdout:5529 read 0x8e9e5e thru 0x8ebccb (0x1e6e bytes) 2024-09-14T05:30:20.016 INFO:teuthology.orchestra.run.smithi071.stdout:5531 write 0x338ff75 thru 0x339152a (0x15b6 bytes) 2024-09-14T05:30:20.024 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:20.024+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:20.028 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:20.028 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:20.039 INFO:teuthology.orchestra.run.smithi071.stdout:5532 write 0xea3f5ca thru 0xea428a1 (0x32d8 bytes) 2024-09-14T05:30:20.041 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:30:20.044 INFO:teuthology.orchestra.run.smithi071.stdout:5533 write 0xaac45e6 thru 0xaacb660 (0x707b bytes) 2024-09-14T05:30:20.051 INFO:teuthology.orchestra.run.smithi071.stdout:5534 read 0xe12b751 thru 0xe136549 (0xadf9 bytes) 2024-09-14T05:30:20.051 INFO:teuthology.orchestra.run.smithi071.stdout:5535 read 0x2716053 thru 0x2721414 (0xb3c2 bytes) 2024-09-14T05:30:20.061 INFO:teuthology.orchestra.run.smithi071.stdout:5536 trunc from 0xea428a2 to 0x5ef2faf 2024-09-14T05:30:20.072 INFO:teuthology.orchestra.run.smithi071.stdout:5537 read 0x7bea21 thru 0x7c2995 (0x3f75 bytes) 2024-09-14T05:30:20.072 INFO:teuthology.orchestra.run.smithi071.stdout:5538 writesame 0x3827dcf thru 0x382b9f0 data_size 0x2(0x3c22 bytes) 2024-09-14T05:30:20.079 INFO:teuthology.orchestra.run.smithi071.stdout:5539 read 0x1404604 thru 0x1410aa0 (0xc49d bytes) 2024-09-14T05:30:20.080 INFO:teuthology.orchestra.run.smithi071.stdout:5540 read 0x2070629 thru 0x207d2a5 (0xcc7d bytes) 2024-09-14T05:30:20.080 INFO:teuthology.orchestra.run.smithi071.stdout:5541 punch from 0x575f609 to 0x576c7e4, (0xd1db bytes) 2024-09-14T05:30:20.086 INFO:teuthology.orchestra.run.smithi071.stdout:5542 write 0x60869e8 thru 0x6092aeb (0xc104 bytes) 2024-09-14T05:30:20.114 INFO:teuthology.orchestra.run.smithi071.stdout:5543 write 0xd0fc1f8 thru 0xd1080d0 (0xbed9 bytes) 2024-09-14T05:30:20.124 INFO:teuthology.orchestra.run.smithi071.stdout:5544 punch from 0xbda1551 to 0xbdadc48, (0xc6f7 bytes) 2024-09-14T05:30:20.124 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:20.124 INFO:teuthology.orchestra.run.smithi071.stdout:5546 write 0x55996f0 thru 0x55a591a (0xc22b bytes) 2024-09-14T05:30:20.125 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:20.124+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:20.132 INFO:teuthology.orchestra.run.smithi071.stdout:5547 read 0xc04cc3a thru 0xc0508a9 (0x3c70 bytes) 2024-09-14T05:30:20.132 INFO:teuthology.orchestra.run.smithi071.stdout:5548 write 0x599f3ba thru 0x59a4ed3 (0x5b1a bytes) 2024-09-14T05:30:20.133 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:20.134 INFO:teuthology.orchestra.run.smithi071.stdout:5550 write 0x4a190a7 thru 0x4a2254a (0x94a4 bytes) 2024-09-14T05:30:20.140 INFO:teuthology.orchestra.run.smithi071.stdout:5551 read 0xdc96d8 thru 0xdd72ca (0xdbf3 bytes) 2024-09-14T05:30:20.141 INFO:teuthology.orchestra.run.smithi071.stdout:5552 write 0x5eac88d thru 0x5eb091a (0x408e bytes) 2024-09-14T05:30:20.144 INFO:teuthology.orchestra.run.smithi071.stdout:5553 writesame 0xb89d48c thru 0xb8aaf87 data_size 0xa(0xdafc bytes) 2024-09-14T05:30:20.155 INFO:teuthology.orchestra.run.smithi071.stdout:5554 punch from 0x7630afa to 0x7639a55, (0x8f5b bytes) 2024-09-14T05:30:20.155 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:20.155 INFO:teuthology.orchestra.run.smithi071.stdout:5556 writesame 0x438740b thru 0x4392f39 data_size 0x1(0xbb2f bytes) 2024-09-14T05:30:20.166 INFO:teuthology.orchestra.run.smithi071.stdout:5557 read 0xc9c7313 thru 0xc9d139d (0xa08b bytes) 2024-09-14T05:30:20.172 INFO:teuthology.orchestra.run.smithi071.stdout:5560 trunc from 0xd1080d1 to 0x88a6d16 2024-09-14T05:30:20.177 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:20.178 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:20.187 INFO:teuthology.orchestra.run.smithi071.stdout:5564 read 0x3ae476d thru 0x3ae575c (0xff0 bytes) 2024-09-14T05:30:20.188 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:30:20.188 INFO:teuthology.orchestra.run.smithi071.stdout:5565 punch from 0x12c04ca to 0x12c2ea4, (0x29da bytes) 2024-09-14T05:30:20.195 INFO:teuthology.orchestra.run.smithi071.stdout:5566 write 0xa899e18 thru 0xa8a22f6 (0x84df bytes) 2024-09-14T05:30:20.208 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:20.217 INFO:teuthology.orchestra.run.smithi071.stdout:5569 writesame 0xd11a8a1 thru 0xd124c49 data_size 0x1(0xa3a9 bytes) 2024-09-14T05:30:20.217 INFO:teuthology.orchestra.run.smithi071.stdout:5570 write 0x831bd96 thru 0x83277f9 (0xba64 bytes) 2024-09-14T05:30:20.226 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:20.225+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:20.233 INFO:teuthology.orchestra.run.smithi071.stdout:5572 write 0xd9d1a18 thru 0xd9d9737 (0x7d20 bytes) 2024-09-14T05:30:20.240 INFO:teuthology.orchestra.run.smithi071.stdout:5573 punch from 0x2f2ccd6 to 0x2f2f9dd, (0x2d07 bytes) 2024-09-14T05:30:20.240 INFO:teuthology.orchestra.run.smithi071.stdout:5574 read 0xb913f3f thru 0xb919ff9 (0x60bb bytes) 2024-09-14T05:30:20.241 INFO:teuthology.orchestra.run.smithi071.stdout:5575 trunc from 0xd9d9738 to 0xcbd9aea 2024-09-14T05:30:20.257 INFO:teuthology.orchestra.run.smithi071.stdout:5577 write 0xcc68910 thru 0xcc72898 (0x9f89 bytes) 2024-09-14T05:30:20.268 INFO:teuthology.orchestra.run.smithi071.stdout:5578 read 0xaddd2fd thru 0xade7e18 (0xab1c bytes) 2024-09-14T05:30:20.269 INFO:teuthology.orchestra.run.smithi071.stdout:5579 read 0x900f441 thru 0x9013ae2 (0x46a2 bytes) 2024-09-14T05:30:20.269 INFO:teuthology.orchestra.run.smithi071.stdout:5580 writesame 0x205a947 thru 0x206630c data_size 0xe(0xb9c6 bytes) 2024-09-14T05:30:20.285 INFO:teuthology.orchestra.run.smithi071.stdout:5581 write 0xea4a99e thru 0xea4f47f (0x4ae2 bytes) 2024-09-14T05:30:20.294 INFO:teuthology.orchestra.run.smithi071.stdout:5582 write 0x56edcc2 thru 0x56f1855 (0x3b94 bytes) 2024-09-14T05:30:20.298 INFO:teuthology.orchestra.run.smithi071.stdout:5583 punch from 0xe71677a to 0xe71c586, (0x5e0c bytes) 2024-09-14T05:30:20.298 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:20.298 INFO:teuthology.orchestra.run.smithi071.stdout:5585 writesame 0x425bd32 thru 0x426108f data_size 0x3(0x535e bytes) 2024-09-14T05:30:20.303 INFO:teuthology.orchestra.run.smithi071.stdout:5586 read 0x93a8e0a thru 0x93b3815 (0xaa0c bytes) 2024-09-14T05:30:20.303 INFO:teuthology.orchestra.run.smithi071.stdout:5588 clone 38 order 21 su 65536 sc 11 2024-09-14T05:30:20.318 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:20.318 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:20.324 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-14T05:30:20.329 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:30:20.330 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:20.328+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:20.391 INFO:teuthology.orchestra.run.smithi071.stdout:truncating image image_client.0-clone37 from 0xea4f480 (overlap 0x578f19) to 0x4ecd96 2024-09-14T05:30:20.403 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T05:30:20.403 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T05:30:20.403 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:30:20.447 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:20.446+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:20.506 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:20.507 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:20.516 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:30:20.516 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-14T05:30:20.528 INFO:teuthology.orchestra.run.smithi071.stdout:5589 punch from 0xe4eeb38 to 0xe4f602f, (0x74f7 bytes) 2024-09-14T05:30:20.532 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:20.530+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:20.533 INFO:teuthology.orchestra.run.smithi071.stdout:5590 write 0x1f0de74 thru 0x1f17486 (0x9613 bytes) 2024-09-14T05:30:20.566 INFO:teuthology.orchestra.run.smithi071.stdout:5591 trunc from 0xea4f480 to 0x3696b8e 2024-09-14T05:30:20.630 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:20.630+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:20.678 INFO:teuthology.orchestra.run.smithi071.stdout:5592 read 0xbe315e thru 0xbeaacb (0x796e bytes) 2024-09-14T05:30:20.691 INFO:teuthology.orchestra.run.smithi071.stdout:5593 writesame 0x8d164b1 thru 0x8d244cf data_size 0x99(0xe01f bytes) 2024-09-14T05:30:20.691 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:20.691 INFO:teuthology.orchestra.run.smithi071.stdout:5595 read 0x2caa772 thru 0x2cb973c (0xefcb bytes) 2024-09-14T05:30:20.691 INFO:teuthology.orchestra.run.smithi071.stdout:5596 read 0x60fcc18 thru 0x6109a60 (0xce49 bytes) 2024-09-14T05:30:20.692 INFO:teuthology.orchestra.run.smithi071.stdout:5597 read 0x49bc5cb thru 0x49c64d3 (0x9f09 bytes) 2024-09-14T05:30:20.692 INFO:teuthology.orchestra.run.smithi071.stdout:5598 read 0x5efcd4e thru 0x5f0455f (0x7812 bytes) 2024-09-14T05:30:20.692 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:20.692 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:20.701 INFO:teuthology.orchestra.run.smithi071.stdout:5599 writesame 0xc693e48 thru 0xc69deac data_size 0x1(0xa065 bytes) 2024-09-14T05:30:20.701 INFO:teuthology.orchestra.run.smithi071.stdout:5600 read 0x5b84692 thru 0x5b851e9 (0xb58 bytes) 2024-09-14T05:30:20.704 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:30:20.715 INFO:teuthology.orchestra.run.smithi071.stdout:5601 trunc from 0xc69dead to 0x8eccf3 2024-09-14T05:30:20.731 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:20.731+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:20.832 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:20.832+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:20.834 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:30:20.854 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:30:20.854 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:30:20.877 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-09-14T05:30:20.447857+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1158313292,"num_objects":2036,"num_object_clones":537,"num_object_copies":4072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2036,"num_whiteouts":1035,"num_read":152184,"num_read_kb":480253,"num_write":15101,"num_write_kb":240024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12880,"ondisk_log_size":12880,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":622636,"kb_used_data":288232,"kb_used_omap":12,"kb_used_meta":333939,"kb_avail":754352084,"statfs":{"total":773094113280,"available":772456534016,"internally_reserved":0,"allocated":295149568,"data_stored":313090540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":341954096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":84283328,"num_objects":132,"num_object_clones":50,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":28,"num_read":9959,"num_read_kb":31326,"num_write":2306,"num_write_kb":16727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001065"},"pg_stats":[{"pgid":"3.1f","version":"69'521","reported_seq":9946,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.102698+0000","last_change":"2024-09-14T05:29:26.019843+0000","last_active":"2024-09-14T05:30:16.102698+0000","last_peered":"2024-09-14T05:30:16.102698+0000","last_clean":"2024-09-14T05:30:16.102698+0000","last_became_active":"2024-09-14T05:29:24.018892+0000","last_became_peered":"2024-09-14T05:29:24.018892+0000","last_unstale":"2024-09-14T05:30:16.102698+0000","last_undegraded":"2024-09-14T05:30:16.102698+0000","last_fullsized":"2024-09-14T05:30:16.102698+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:36:15.231299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011130109999999999,"stat_sum":{"num_bytes":48187430,"num_objects":68,"num_object_clones":17,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":32,"num_read":11414,"num_read_kb":34973,"num_write":638,"num_write_kb":29869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"69'272","reported_seq":1893,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.272969+0000","last_change":"2024-09-14T05:29:26.019068+0000","last_active":"2024-09-14T05:30:16.272969+0000","last_peered":"2024-09-14T05:30:16.272969+0000","last_clean":"2024-09-14T05:30:16.272969+0000","last_became_active":"2024-09-14T05:29:24.014468+0000","last_became_peered":"2024-09-14T05:29:24.014468+0000","last_unstale":"2024-09-14T05:30:16.272969+0000","last_undegraded":"2024-09-14T05:30:16.272969+0000","last_fullsized":"2024-09-14T05:30:16.272969+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:22:52.170938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000399031,"stat_sum":{"num_bytes":65629102,"num_objects":65,"num_object_clones":18,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":28,"num_read":1789,"num_read_kb":23789,"num_write":287,"num_write_kb":20241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"69'225","reported_seq":8631,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.299176+0000","last_change":"2024-09-14T05:29:26.165031+0000","last_active":"2024-09-14T05:30:16.299176+0000","last_peered":"2024-09-14T05:30:16.299176+0000","last_clean":"2024-09-14T05:30:16.299176+0000","last_became_active":"2024-09-14T05:29:24.014251+0000","last_became_peered":"2024-09-14T05:29:24.014251+0000","last_unstale":"2024-09-14T05:30:16.299176+0000","last_undegraded":"2024-09-14T05:30:16.299176+0000","last_fullsized":"2024-09-14T05:30:16.299176+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:50.690781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010261580000000001,"stat_sum":{"num_bytes":60517535,"num_objects":66,"num_object_clones":23,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":33,"num_read":8352,"num_read_kb":20599,"num_write":212,"num_write_kb":5331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"70'367","reported_seq":9430,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.327828+0000","last_change":"2024-09-14T05:29:26.014243+0000","last_active":"2024-09-14T05:30:16.327828+0000","last_peered":"2024-09-14T05:30:16.327828+0000","last_clean":"2024-09-14T05:30:16.327828+0000","last_became_active":"2024-09-14T05:29:24.012427+0000","last_became_peered":"2024-09-14T05:29:24.012427+0000","last_unstale":"2024-09-14T05:30:16.327828+0000","last_undegraded":"2024-09-14T05:30:16.327828+0000","last_fullsized":"2024-09-14T05:30:16.327828+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:56:53.752030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045797999999999998,"stat_sum":{"num_bytes":64349949,"num_objects":78,"num_object_clones":22,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":40,"num_read":10016,"num_read_kb":33389,"num_write":356,"num_write_kb":4364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"69'410","reported_seq":2852,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.299398+0000","last_change":"2024-09-14T05:29:26.014462+0000","last_active":"2024-09-14T05:30:16.299398+0000","last_peered":"2024-09-14T05:30:16.299398+0000","last_clean":"2024-09-14T05:30:16.299398+0000","last_became_active":"2024-09-14T05:29:24.012079+0000","last_became_peered":"2024-09-14T05:29:24.012079+0000","last_unstale":"2024-09-14T05:30:16.299398+0000","last_undegraded":"2024-09-14T05:30:16.299398+0000","last_fullsized":"2024-09-14T05:30:16.299398+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:48.636176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070589999999999997,"stat_sum":{"num_bytes":38412609,"num_objects":65,"num_object_clones":17,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":35,"num_read":3755,"num_read_kb":24557,"num_write":451,"num_write_kb":21605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"69'727","reported_seq":4249,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.348206+0000","last_change":"2024-09-14T05:29:26.014453+0000","last_active":"2024-09-14T05:30:16.348206+0000","last_peered":"2024-09-14T05:30:16.348206+0000","last_clean":"2024-09-14T05:30:16.348206+0000","last_became_active":"2024-09-14T05:29:24.012461+0000","last_became_peered":"2024-09-14T05:29:24.012461+0000","last_unstale":"2024-09-14T05:30:16.348206+0000","last_undegraded":"2024-09-14T05:30:16.348206+0000","last_fullsized":"2024-09-14T05:30:16.348206+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:45:56.024446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061290100000000001,"stat_sum":{"num_bytes":26977174,"num_objects":71,"num_object_clones":18,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":36,"num_read":7214,"num_read_kb":30553,"num_write":1050,"num_write_kb":6100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"69'245","reported_seq":1569,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.299495+0000","last_change":"2024-09-14T05:29:26.014685+0000","last_active":"2024-09-14T05:30:16.299495+0000","last_peered":"2024-09-14T05:30:16.299495+0000","last_clean":"2024-09-14T05:30:16.299495+0000","last_became_active":"2024-09-14T05:29:24.018753+0000","last_became_peered":"2024-09-14T05:29:24.018753+0000","last_unstale":"2024-09-14T05:30:16.299495+0000","last_undegraded":"2024-09-14T05:30:16.299495+0000","last_fullsized":"2024-09-14T05:30:16.299495+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:54:45.857412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089943399999999995,"stat_sum":{"num_bytes":41249325,"num_objects":72,"num_object_clones":19,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":31,"num_read":1371,"num_read_kb":12605,"num_write":231,"num_write_kb":8813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"69'461","reported_seq":2124,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.240591+0000","last_change":"2024-09-14T05:29:26.019928+0000","last_active":"2024-09-14T05:30:16.240591+0000","last_peered":"2024-09-14T05:30:16.240591+0000","last_clean":"2024-09-14T05:30:16.240591+0000","last_became_active":"2024-09-14T05:29:24.011176+0000","last_became_peered":"2024-09-14T05:29:24.011176+0000","last_unstale":"2024-09-14T05:30:16.240591+0000","last_undegraded":"2024-09-14T05:30:16.240591+0000","last_fullsized":"2024-09-14T05:30:16.240591+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:06.592202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011097850000000001,"stat_sum":{"num_bytes":15460319,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":27,"num_read":4011,"num_read_kb":6833,"num_write":510,"num_write_kb":2929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"69'256","reported_seq":1801,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.191418+0000","last_change":"2024-09-14T05:29:26.019132+0000","last_active":"2024-09-14T05:30:16.191418+0000","last_peered":"2024-09-14T05:30:16.191418+0000","last_clean":"2024-09-14T05:30:16.191418+0000","last_became_active":"2024-09-14T05:29:24.014005+0000","last_became_peered":"2024-09-14T05:29:24.014005+0000","last_unstale":"2024-09-14T05:30:16.191418+0000","last_undegraded":"2024-09-14T05:30:16.191418+0000","last_fullsized":"2024-09-14T05:30:16.191418+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:51:18.120579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045225300000000001,"stat_sum":{"num_bytes":7413873,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":22,"num_read":2050,"num_read_kb":4565,"num_write":247,"num_write_kb":3195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"69'248","reported_seq":9375,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.309366+0000","last_change":"2024-09-14T05:29:26.028571+0000","last_active":"2024-09-14T05:30:16.309366+0000","last_peered":"2024-09-14T05:30:16.309366+0000","last_clean":"2024-09-14T05:30:16.309366+0000","last_became_active":"2024-09-14T05:29:24.013434+0000","last_became_peered":"2024-09-14T05:29:24.013434+0000","last_unstale":"2024-09-14T05:30:16.309366+0000","last_undegraded":"2024-09-14T05:30:16.309366+0000","last_fullsized":"2024-09-14T05:30:16.309366+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:16:21.510234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038301200000000001,"stat_sum":{"num_bytes":60233783,"num_objects":68,"num_object_clones":20,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":37,"num_read":9078,"num_read_kb":21386,"num_write":250,"num_write_kb":7575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"69'314","reported_seq":2225,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-14T05:30:17.467720+0000","last_change":"2024-09-14T05:29:26.020364+0000","last_active":"2024-09-14T05:30:17.467720+0000","last_peered":"2024-09-14T05:30:17.467720+0000","last_clean":"2024-09-14T05:30:17.467720+0000","last_became_active":"2024-09-14T05:29:24.014457+0000","last_became_peered":"2024-09-14T05:29:24.014457+0000","last_unstale":"2024-09-14T05:30:17.467720+0000","last_undegraded":"2024-09-14T05:30:17.467720+0000","last_fullsized":"2024-09-14T05:30:17.467720+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:34:56.200824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010888930000000001,"stat_sum":{"num_bytes":73089410,"num_objects":66,"num_object_clones":21,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":30,"num_read":2391,"num_read_kb":9067,"num_write":336,"num_write_kb":11850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"70'686","reported_seq":1898,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.302231+0000","last_change":"2024-09-14T05:29:26.164291+0000","last_active":"2024-09-14T05:30:16.302231+0000","last_peered":"2024-09-14T05:30:16.302231+0000","last_clean":"2024-09-14T05:30:16.302231+0000","last_became_active":"2024-09-14T05:29:24.013266+0000","last_became_peered":"2024-09-14T05:29:24.013266+0000","last_unstale":"2024-09-14T05:30:16.302231+0000","last_undegraded":"2024-09-14T05:30:16.302231+0000","last_fullsized":"2024-09-14T05:30:16.302231+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:05:02.453637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060129699999999999,"stat_sum":{"num_bytes":16426289,"num_objects":56,"num_object_clones":10,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":35,"num_read":3985,"num_read_kb":4291,"num_write":948,"num_write_kb":3740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":116,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.300325+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:30:16.300325+0000","last_peered":"2024-09-14T05:30:16.300325+0000","last_clean":"2024-09-14T05:30:16.300325+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:30:16.300325+0000","last_undegraded":"2024-09-14T05:30:16.300325+0000","last_fullsized":"2024-09-14T05:30:16.300325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"69'214","reported_seq":1516,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.299462+0000","last_change":"2024-09-14T05:29:26.014725+0000","last_active":"2024-09-14T05:30:16.299462+0000","last_peered":"2024-09-14T05:30:16.299462+0000","last_clean":"2024-09-14T05:30:16.299462+0000","last_became_active":"2024-09-14T05:29:24.012099+0000","last_became_peered":"2024-09-14T05:29:24.012099+0000","last_unstale":"2024-09-14T05:30:16.299462+0000","last_undegraded":"2024-09-14T05:30:16.299462+0000","last_fullsized":"2024-09-14T05:30:16.299462+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:22.073189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000794599,"stat_sum":{"num_bytes":8539390,"num_objects":52,"num_object_clones":15,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":27,"num_read":1664,"num_read_kb":2130,"num_write":200,"num_write_kb":2392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":114,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.299423+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:30:16.299423+0000","last_peered":"2024-09-14T05:30:16.299423+0000","last_clean":"2024-09-14T05:30:16.299423+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:30:16.299423+0000","last_undegraded":"2024-09-14T05:30:16.299423+0000","last_fullsized":"2024-09-14T05:30:16.299423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"70'627","reported_seq":2276,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.345301+0000","last_change":"2024-09-14T05:29:26.015082+0000","last_active":"2024-09-14T05:30:16.345301+0000","last_peered":"2024-09-14T05:30:16.345301+0000","last_clean":"2024-09-14T05:30:16.345301+0000","last_became_active":"2024-09-14T05:29:24.011481+0000","last_became_peered":"2024-09-14T05:29:24.011481+0000","last_unstale":"2024-09-14T05:30:16.345301+0000","last_undegraded":"2024-09-14T05:30:16.345301+0000","last_fullsized":"2024-09-14T05:30:16.345301+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:48:08.419430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011470110000000001,"stat_sum":{"num_bytes":44456816,"num_objects":61,"num_object_clones":13,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":29,"num_read":4489,"num_read_kb":4353,"num_write":822,"num_write_kb":2302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":116,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.300171+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:30:16.300171+0000","last_peered":"2024-09-14T05:30:16.300171+0000","last_clean":"2024-09-14T05:30:16.300171+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:30:16.300171+0000","last_undegraded":"2024-09-14T05:30:16.300171+0000","last_fullsized":"2024-09-14T05:30:16.300171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"69'439","reported_seq":1836,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.103946+0000","last_change":"2024-09-14T05:29:40.095950+0000","last_active":"2024-09-14T05:30:16.103946+0000","last_peered":"2024-09-14T05:30:16.103946+0000","last_clean":"2024-09-14T05:30:16.103946+0000","last_became_active":"2024-09-14T05:29:40.095441+0000","last_became_peered":"2024-09-14T05:29:40.095441+0000","last_unstale":"2024-09-14T05:30:16.103946+0000","last_undegraded":"2024-09-14T05:30:16.103946+0000","last_fullsized":"2024-09-14T05:30:16.103946+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:09.905119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053582499999999997,"stat_sum":{"num_bytes":24748212,"num_objects":64,"num_object_clones":16,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":36,"num_read":2748,"num_read_kb":10813,"num_write":523,"num_write_kb":9121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T05:30:14.325044+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:30:14.325044+0000","last_peered":"2024-09-14T05:30:14.325044+0000","last_clean":"2024-09-14T05:30:14.325044+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:30:14.325044+0000","last_undegraded":"2024-09-14T05:30:14.325044+0000","last_fullsized":"2024-09-14T05:30:14.325044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"69'234","reported_seq":1503,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.309227+0000","last_change":"2024-09-14T05:29:26.028547+0000","last_active":"2024-09-14T05:30:16.309227+0000","last_peered":"2024-09-14T05:30:16.309227+0000","last_clean":"2024-09-14T05:30:16.309227+0000","last_became_active":"2024-09-14T05:29:24.016940+0000","last_became_peered":"2024-09-14T05:29:24.016940+0000","last_unstale":"2024-09-14T05:30:16.309227+0000","last_undegraded":"2024-09-14T05:30:16.309227+0000","last_fullsized":"2024-09-14T05:30:16.309227+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:12:15.302398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042396399999999998,"stat_sum":{"num_bytes":11629336,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":27,"num_read":1511,"num_read_kb":4295,"num_write":245,"num_write_kb":2990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"71'316","reported_seq":1607,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-14T05:30:17.388362+0000","last_change":"2024-09-14T05:29:26.019829+0000","last_active":"2024-09-14T05:30:17.388362+0000","last_peered":"2024-09-14T05:30:17.388362+0000","last_clean":"2024-09-14T05:30:17.388362+0000","last_became_active":"2024-09-14T05:29:24.015905+0000","last_became_peered":"2024-09-14T05:29:24.015905+0000","last_unstale":"2024-09-14T05:30:17.388362+0000","last_undegraded":"2024-09-14T05:30:17.388362+0000","last_fullsized":"2024-09-14T05:30:17.388362+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:56:23.780961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049652099999999998,"stat_sum":{"num_bytes":12433295,"num_objects":72,"num_object_clones":21,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":42,"num_read":1615,"num_read_kb":5833,"num_write":366,"num_write_kb":4829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":118,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.299288+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:30:16.299288+0000","last_peered":"2024-09-14T05:30:16.299288+0000","last_clean":"2024-09-14T05:30:16.299288+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:30:16.299288+0000","last_undegraded":"2024-09-14T05:30:16.299288+0000","last_fullsized":"2024-09-14T05:30:16.299288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"69'336","reported_seq":10298,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T05:30:15.183513+0000","last_change":"2024-09-14T05:29:26.019261+0000","last_active":"2024-09-14T05:30:15.183513+0000","last_peered":"2024-09-14T05:30:15.183513+0000","last_clean":"2024-09-14T05:30:15.183513+0000","last_became_active":"2024-09-14T05:29:24.011764+0000","last_became_peered":"2024-09-14T05:29:24.011764+0000","last_unstale":"2024-09-14T05:30:15.183513+0000","last_undegraded":"2024-09-14T05:30:15.183513+0000","last_fullsized":"2024-09-14T05:30:15.183513+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:51:08.135809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055502299999999997,"stat_sum":{"num_bytes":45068264,"num_objects":66,"num_object_clones":17,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":33,"num_read":11244,"num_read_kb":38521,"num_write":310,"num_write_kb":4748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":116,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.302218+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:30:16.302218+0000","last_peered":"2024-09-14T05:30:16.302218+0000","last_clean":"2024-09-14T05:30:16.302218+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:30:16.302218+0000","last_undegraded":"2024-09-14T05:30:16.302218+0000","last_fullsized":"2024-09-14T05:30:16.302218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"69'585","reported_seq":1513,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.309195+0000","last_change":"2024-09-14T05:29:40.095721+0000","last_active":"2024-09-14T05:30:16.309195+0000","last_peered":"2024-09-14T05:30:16.309195+0000","last_clean":"2024-09-14T05:30:16.309195+0000","last_became_active":"2024-09-14T05:29:40.095255+0000","last_became_peered":"2024-09-14T05:29:40.095255+0000","last_unstale":"2024-09-14T05:30:16.309195+0000","last_undegraded":"2024-09-14T05:30:16.309195+0000","last_fullsized":"2024-09-14T05:30:16.309195+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:36:43.795277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077857800000000004,"stat_sum":{"num_bytes":50785003,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":25,"num_read":2908,"num_read_kb":6054,"num_write":839,"num_write_kb":9901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":116,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.298811+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:30:16.298811+0000","last_peered":"2024-09-14T05:30:16.298811+0000","last_clean":"2024-09-14T05:30:16.298811+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:30:16.298811+0000","last_undegraded":"2024-09-14T05:30:16.298811+0000","last_fullsized":"2024-09-14T05:30:16.298811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"69'450","reported_seq":2230,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.299579+0000","last_change":"2024-09-14T05:29:26.015124+0000","last_active":"2024-09-14T05:30:16.299579+0000","last_peered":"2024-09-14T05:30:16.299579+0000","last_clean":"2024-09-14T05:30:16.299579+0000","last_became_active":"2024-09-14T05:29:24.015792+0000","last_became_peered":"2024-09-14T05:29:24.015792+0000","last_unstale":"2024-09-14T05:30:16.299579+0000","last_undegraded":"2024-09-14T05:30:16.299579+0000","last_fullsized":"2024-09-14T05:30:16.299579+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:55:58.100179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001049964,"stat_sum":{"num_bytes":16760257,"num_objects":68,"num_object_clones":16,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":40,"num_read":3285,"num_read_kb":9181,"num_write":525,"num_write_kb":3681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":117,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.299305+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:30:16.299305+0000","last_peered":"2024-09-14T05:30:16.299305+0000","last_clean":"2024-09-14T05:30:16.299305+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:30:16.299305+0000","last_undegraded":"2024-09-14T05:30:16.299305+0000","last_fullsized":"2024-09-14T05:30:16.299305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":207,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.305712+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:30:16.305712+0000","last_peered":"2024-09-14T05:30:16.305712+0000","last_clean":"2024-09-14T05:30:16.305712+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:30:16.305712+0000","last_undegraded":"2024-09-14T05:30:16.305712+0000","last_fullsized":"2024-09-14T05:30:16.305712+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"70'299","reported_seq":1692,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.353601+0000","last_change":"2024-09-14T05:29:26.014430+0000","last_active":"2024-09-14T05:30:16.353601+0000","last_peered":"2024-09-14T05:30:16.353601+0000","last_clean":"2024-09-14T05:30:16.353601+0000","last_became_active":"2024-09-14T05:29:24.012222+0000","last_became_peered":"2024-09-14T05:29:24.012222+0000","last_unstale":"2024-09-14T05:30:16.353601+0000","last_undegraded":"2024-09-14T05:30:16.353601+0000","last_fullsized":"2024-09-14T05:30:16.353601+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:35:24.025128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076580099999999998,"stat_sum":{"num_bytes":28754938,"num_objects":59,"num_object_clones":15,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":29,"num_read":2278,"num_read_kb":5556,"num_write":270,"num_write_kb":3747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"70'481","reported_seq":1593,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.338428+0000","last_change":"2024-09-14T05:29:26.164542+0000","last_active":"2024-09-14T05:30:16.338428+0000","last_peered":"2024-09-14T05:30:16.338428+0000","last_clean":"2024-09-14T05:30:16.338428+0000","last_became_active":"2024-09-14T05:29:24.012041+0000","last_became_peered":"2024-09-14T05:29:24.012041+0000","last_unstale":"2024-09-14T05:30:16.338428+0000","last_undegraded":"2024-09-14T05:30:16.338428+0000","last_fullsized":"2024-09-14T05:30:16.338428+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:24:03.024555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078334300000000004,"stat_sum":{"num_bytes":21030850,"num_objects":51,"num_object_clones":12,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":26,"num_read":2414,"num_read_kb":4819,"num_write":649,"num_write_kb":3256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"70'487","reported_seq":1993,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.298997+0000","last_change":"2024-09-14T05:29:26.165164+0000","last_active":"2024-09-14T05:30:16.298997+0000","last_peered":"2024-09-14T05:30:16.298997+0000","last_clean":"2024-09-14T05:30:16.298997+0000","last_became_active":"2024-09-14T05:29:24.011403+0000","last_became_peered":"2024-09-14T05:29:24.011403+0000","last_unstale":"2024-09-14T05:30:16.298997+0000","last_undegraded":"2024-09-14T05:30:16.298997+0000","last_fullsized":"2024-09-14T05:30:16.298997+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:47:04.030517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083114500000000004,"stat_sum":{"num_bytes":19112233,"num_objects":66,"num_object_clones":18,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":30,"num_read":3303,"num_read_kb":7453,"num_write":589,"num_write_kb":4836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"69'477","reported_seq":9482,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.302299+0000","last_change":"2024-09-14T05:29:26.163783+0000","last_active":"2024-09-14T05:30:16.302299+0000","last_peered":"2024-09-14T05:30:16.302299+0000","last_clean":"2024-09-14T05:30:16.302299+0000","last_became_active":"2024-09-14T05:29:24.006836+0000","last_became_peered":"2024-09-14T05:29:24.006836+0000","last_unstale":"2024-09-14T05:30:16.302299+0000","last_undegraded":"2024-09-14T05:30:16.302299+0000","last_fullsized":"2024-09-14T05:30:16.302299+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:15:43.292440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031188400000000001,"stat_sum":{"num_bytes":45603650,"num_objects":71,"num_object_clones":13,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":37,"num_read":11244,"num_read_kb":29098,"num_write":573,"num_write_kb":12983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"70'258","reported_seq":2532,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-14T05:30:17.341721+0000","last_change":"2024-09-14T05:29:26.020113+0000","last_active":"2024-09-14T05:30:17.341721+0000","last_peered":"2024-09-14T05:30:17.341721+0000","last_clean":"2024-09-14T05:30:17.341721+0000","last_became_active":"2024-09-14T05:29:24.012574+0000","last_became_peered":"2024-09-14T05:29:24.012574+0000","last_unstale":"2024-09-14T05:30:17.341721+0000","last_undegraded":"2024-09-14T05:30:17.341721+0000","last_fullsized":"2024-09-14T05:30:17.341721+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:31:02.175073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072680599999999998,"stat_sum":{"num_bytes":44989371,"num_objects":69,"num_object_clones":24,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":33,"num_read":2256,"num_read_kb":9773,"num_write":231,"num_write_kb":6429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"69'845","reported_seq":10044,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.299851+0000","last_change":"2024-09-14T05:29:26.164736+0000","last_active":"2024-09-14T05:30:16.299851+0000","last_peered":"2024-09-14T05:30:16.299851+0000","last_clean":"2024-09-14T05:30:16.299851+0000","last_became_active":"2024-09-14T05:29:24.012401+0000","last_became_peered":"2024-09-14T05:29:24.012401+0000","last_unstale":"2024-09-14T05:30:16.299851+0000","last_undegraded":"2024-09-14T05:30:16.299851+0000","last_fullsized":"2024-09-14T05:30:16.299851+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":845,"log_dups_size":0,"ondisk_log_size":845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:23:07.661241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085260200000000005,"stat_sum":{"num_bytes":44405850,"num_objects":78,"num_object_clones":20,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":42,"num_read":12871,"num_read_kb":20489,"num_write":1131,"num_write_kb":5144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"69'253","reported_seq":1674,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.309409+0000","last_change":"2024-09-14T05:29:26.028811+0000","last_active":"2024-09-14T05:30:16.309409+0000","last_peered":"2024-09-14T05:30:16.309409+0000","last_clean":"2024-09-14T05:30:16.309409+0000","last_became_active":"2024-09-14T05:29:24.015315+0000","last_became_peered":"2024-09-14T05:29:24.015315+0000","last_unstale":"2024-09-14T05:30:16.309409+0000","last_undegraded":"2024-09-14T05:30:16.309409+0000","last_fullsized":"2024-09-14T05:30:16.309409+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:08:41.504667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046485199999999997,"stat_sum":{"num_bytes":25912399,"num_objects":68,"num_object_clones":22,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":41,"num_read":1360,"num_read_kb":2557,"num_write":228,"num_write_kb":3504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"69'338","reported_seq":10806,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-14T05:30:17.341641+0000","last_change":"2024-09-14T05:29:26.020027+0000","last_active":"2024-09-14T05:30:17.341641+0000","last_peered":"2024-09-14T05:30:17.341641+0000","last_clean":"2024-09-14T05:30:17.341641+0000","last_became_active":"2024-09-14T05:29:24.018854+0000","last_became_peered":"2024-09-14T05:29:24.018854+0000","last_unstale":"2024-09-14T05:30:17.341641+0000","last_undegraded":"2024-09-14T05:30:17.341641+0000","last_fullsized":"2024-09-14T05:30:17.341641+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:20:20.550534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063672000000000002,"stat_sum":{"num_bytes":35786346,"num_objects":87,"num_object_clones":24,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":46,"num_read":11264,"num_read_kb":29152,"num_write":311,"num_write_kb":11046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"69'271","reported_seq":1696,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.305766+0000","last_change":"2024-09-14T05:29:26.014200+0000","last_active":"2024-09-14T05:30:16.305766+0000","last_peered":"2024-09-14T05:30:16.305766+0000","last_clean":"2024-09-14T05:30:16.305766+0000","last_became_active":"2024-09-14T05:29:24.010897+0000","last_became_peered":"2024-09-14T05:29:24.010897+0000","last_unstale":"2024-09-14T05:30:16.305766+0000","last_undegraded":"2024-09-14T05:30:16.305766+0000","last_fullsized":"2024-09-14T05:30:16.305766+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:01:04.578925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051531799999999996,"stat_sum":{"num_bytes":42298008,"num_objects":57,"num_object_clones":16,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":28,"num_read":1434,"num_read_kb":12678,"num_write":260,"num_write_kb":7593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"69'379","reported_seq":2578,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.298937+0000","last_change":"2024-09-14T05:29:26.164788+0000","last_active":"2024-09-14T05:30:16.298937+0000","last_peered":"2024-09-14T05:30:16.298937+0000","last_clean":"2024-09-14T05:30:16.298937+0000","last_became_active":"2024-09-14T05:29:24.013792+0000","last_became_peered":"2024-09-14T05:29:24.013792+0000","last_unstale":"2024-09-14T05:30:16.298937+0000","last_undegraded":"2024-09-14T05:30:16.298937+0000","last_fullsized":"2024-09-14T05:30:16.298937+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:18:34.204007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090671299999999996,"stat_sum":{"num_bytes":29675747,"num_objects":57,"num_object_clones":13,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":29,"num_read":3895,"num_read_kb":18576,"num_write":377,"num_write_kb":2669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"70'252","reported_seq":1515,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.299227+0000","last_change":"2024-09-14T05:29:40.095745+0000","last_active":"2024-09-14T05:30:16.299227+0000","last_peered":"2024-09-14T05:30:16.299227+0000","last_clean":"2024-09-14T05:30:16.299227+0000","last_became_active":"2024-09-14T05:29:40.095331+0000","last_became_peered":"2024-09-14T05:29:40.095331+0000","last_unstale":"2024-09-14T05:30:16.299227+0000","last_undegraded":"2024-09-14T05:30:16.299227+0000","last_fullsized":"2024-09-14T05:30:16.299227+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:14:20.934233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000774981,"stat_sum":{"num_bytes":31198800,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":23,"num_read":1760,"num_read_kb":13641,"num_write":237,"num_write_kb":6359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"69'532","reported_seq":1860,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T05:30:16.299167+0000","last_change":"2024-09-14T05:29:26.164295+0000","last_active":"2024-09-14T05:30:16.299167+0000","last_peered":"2024-09-14T05:30:16.299167+0000","last_clean":"2024-09-14T05:30:16.299167+0000","last_became_active":"2024-09-14T05:29:24.014371+0000","last_became_peered":"2024-09-14T05:29:24.014371+0000","last_unstale":"2024-09-14T05:30:16.299167+0000","last_undegraded":"2024-09-14T05:30:16.299167+0000","last_fullsized":"2024-09-14T05:30:16.299167+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:36:21.561951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050570599999999997,"stat_sum":{"num_bytes":56587342,"num_objects":57,"num_object_clones":14,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":26,"num_read":3149,"num_read_kb":18619,"num_write":796,"num_write_kb":5252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1157722905,"num_objects":2032,"num_object_clones":537,"num_object_copies":4064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2032,"num_whiteouts":1035,"num_read":152118,"num_read_kb":480198,"num_write":14998,"num_write_kb":238394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":287584256,"data_stored":309105558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12805,"ondisk_log_size":12805,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476754,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90368,"kb_used_data":50864,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94281472,"statfs":{"total":96636764160,"available":96544227328,"internally_reserved":0,"allocated":52084736,"data_stored":53754563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59168,"kb_used_data":23820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94312672,"statfs":{"total":96636764160,"available":96576176128,"internally_reserved":0,"allocated":24391680,"data_stored":25386885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70144,"kb_used_data":30624,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94301696,"statfs":{"total":96636764160,"available":96564936704,"internally_reserved":0,"allocated":31358976,"data_stored":34386193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87208,"kb_used_data":47704,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94284632,"statfs":{"total":96636764160,"available":96547463168,"internally_reserved":0,"allocated":48848896,"data_stored":52395348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71836,"kb_used_data":32080,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94300004,"statfs":{"total":96636764160,"available":96563204096,"internally_reserved":0,"allocated":32849920,"data_stored":33200571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89292,"kb_used_data":27580,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94282548,"statfs":{"total":96636764160,"available":96545329152,"internally_reserved":0,"allocated":28241920,"data_stored":31467454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66704,"kb_used_data":27200,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94305136,"statfs":{"total":96636764160,"available":96568459264,"internally_reserved":0,"allocated":27852800,"data_stored":29739972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87916,"kb_used_data":48360,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94283924,"statfs":{"total":96636764160,"available":96546738176,"internally_reserved":0,"allocated":49520640,"data_stored":52759554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":51261440,"data_stored":53459281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":48082944,"data_stored":51768271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27090944,"data_stored":29404715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":27402240,"data_stored":31058655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":32079872,"data_stored":32862373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":47513600,"data_stored":51497546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":30572544,"data_stored":34034702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":23580672,"data_stored":25020015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:30:21.103 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:21.103 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:21.104 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.104 INFO:teuthology.orchestra.run.smithi071.stdout:5604 read 0x5144e2 thru 0x519209 (0x4d28 bytes) 2024-09-14T05:30:21.104 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.104 INFO:teuthology.orchestra.run.smithi071.stdout:5608 read 0x274b4 thru 0x3598b (0xe4d8 bytes) 2024-09-14T05:30:21.105 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:30:21.106 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.106 INFO:teuthology.orchestra.run.smithi071.stdout:5610 write 0xb6fecd4 thru 0xb70ad45 (0xc072 bytes) 2024-09-14T05:30:21.106 INFO:teuthology.orchestra.run.smithi071.stdout:5613 writesame 0xc232f4b thru 0xc236115 data_size 0x7(0x31cb bytes) 2024-09-14T05:30:21.106 INFO:teuthology.orchestra.run.smithi071.stdout:5614 write 0xcead413 thru 0xceae3d8 (0xfc6 bytes) 2024-09-14T05:30:21.106 INFO:teuthology.orchestra.run.smithi071.stdout:5616 write 0xa0f32aa thru 0xa0ff5a5 (0xc2fc bytes) 2024-09-14T05:30:21.107 INFO:teuthology.orchestra.run.smithi071.stdout:5617 writesame 0x6b29b00 thru 0x6b2cefe data_size 0x3(0x33ff bytes) 2024-09-14T05:30:21.107 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.107 INFO:teuthology.orchestra.run.smithi071.stdout:5619 trunc from 0xceae3d9 to 0x3d32ab6 2024-09-14T05:30:21.107 INFO:teuthology.orchestra.run.smithi071.stdout:5620 read 0x1517f64 thru 0x151fb9c (0x7c39 bytes) 2024-09-14T05:30:21.107 INFO:teuthology.orchestra.run.smithi071.stdout:5623 read 0x28be7e2 thru 0x28c5bf3 (0x7412 bytes) 2024-09-14T05:30:21.107 INFO:teuthology.orchestra.run.smithi071.stdout:5626 write 0x7cf739 thru 0x7d092f (0x11f7 bytes) 2024-09-14T05:30:21.107 INFO:teuthology.orchestra.run.smithi071.stdout:5627 punch from 0x10e4bc8 to 0x10e68d1, (0x1d09 bytes) 2024-09-14T05:30:21.107 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:21.107 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:21.108 INFO:teuthology.orchestra.run.smithi071.stdout:5628 write 0xdde4b07 thru 0xddede5e (0x9358 bytes) 2024-09-14T05:30:21.108 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.108 INFO:teuthology.orchestra.run.smithi071.stdout:5631 read 0xd8c9275 thru 0xd8d7357 (0xe0e3 bytes) 2024-09-14T05:30:21.108 INFO:teuthology.orchestra.run.smithi071.stdout:5632 read 0x6a9dc7d thru 0x6aac579 (0xe8fd bytes) 2024-09-14T05:30:21.108 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:21.109 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:21.109 INFO:teuthology.orchestra.run.smithi071.stdout:5634 trunc from 0xddede5f to 0x645bd16 2024-09-14T05:30:21.109 INFO:teuthology.orchestra.run.smithi071.stdout:5635 trunc from 0x645bd16 to 0x5960a02 2024-09-14T05:30:21.109 INFO:teuthology.orchestra.run.smithi071.stdout:5636 punch from 0xb2d856 to 0xb36da7, (0x9551 bytes) 2024-09-14T05:30:21.110 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.110 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:21.110 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:21.110 INFO:teuthology.orchestra.run.smithi071.stdout:5639 trunc from 0x5960a02 to 0xbf053f5 2024-09-14T05:30:21.110 INFO:teuthology.orchestra.run.smithi071.stdout:5640 trunc from 0xbf053f5 to 0x76dff26 2024-09-14T05:30:21.111 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:21.111 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:21.111 INFO:teuthology.orchestra.run.smithi071.stdout:5641 write 0xd49f209 thru 0xd4a8f8f (0x9d87 bytes) 2024-09-14T05:30:21.111 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:21.112 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:21.112 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:21.103+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:21.113 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.113 INFO:teuthology.orchestra.run.smithi071.stdout:5643 write 0x202c48d thru 0x2030f01 (0x4a75 bytes) 2024-09-14T05:30:21.113 INFO:teuthology.orchestra.run.smithi071.stdout:5644 write 0x142d3f6 thru 0x1439a24 (0xc62f bytes) 2024-09-14T05:30:21.113 INFO:teuthology.orchestra.run.smithi071.stdout:5645 write 0xe023529 thru 0xe023605 (0xdd bytes) 2024-09-14T05:30:21.113 INFO:teuthology.orchestra.run.smithi071.stdout:5646 punch from 0x966c17f to 0x967ae33, (0xecb4 bytes) 2024-09-14T05:30:21.113 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.114 INFO:teuthology.orchestra.run.smithi071.stdout:5650 punch from 0x8944bc2 to 0x894c6e6, (0x7b24 bytes) 2024-09-14T05:30:21.114 INFO:teuthology.orchestra.run.smithi071.stdout:5651 writesame 0x7670be0 thru 0x7672b57 data_size 0x35(0x1f78 bytes) 2024-09-14T05:30:21.114 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:21.114 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:21.114 INFO:teuthology.orchestra.run.smithi071.stdout:5653 trunc from 0xe023606 to 0x6c76f8e 2024-09-14T05:30:21.115 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:21.115 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:21.116 INFO:teuthology.orchestra.run.smithi071.stdout:5655 read 0x67bcb2d thru 0x67c62db (0x97af bytes) 2024-09-14T05:30:21.116 INFO:teuthology.orchestra.run.smithi071.stdout:5656 read 0x1caa688 thru 0x1cac2b6 (0x1c2f bytes) 2024-09-14T05:30:21.116 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.116 INFO:teuthology.orchestra.run.smithi071.stdout:5661 write 0x15771c7 thru 0x157a1db (0x3015 bytes) 2024-09-14T05:30:21.116 INFO:teuthology.orchestra.run.smithi071.stdout:5662 punch from 0x1c49d60 to 0x1c4b8ce, (0x1b6e bytes) 2024-09-14T05:30:21.116 INFO:teuthology.orchestra.run.smithi071.stdout:5663 trunc from 0x6c76f8e to 0x73c24ae 2024-09-14T05:30:21.116 INFO:teuthology.orchestra.run.smithi071.stdout:5664 read 0xd50cb thru 0xd8008 (0x2f3e bytes) 2024-09-14T05:30:21.116 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:21.117 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:21.119 INFO:teuthology.orchestra.run.smithi071.stdout:5665 write 0xe47eb88 thru 0xe48c455 (0xd8ce bytes) 2024-09-14T05:30:21.123 INFO:teuthology.orchestra.run.smithi071.stdout:5666 punch from 0x56451bd to 0x5654b48, (0xf98b bytes) 2024-09-14T05:30:21.123 INFO:teuthology.orchestra.run.smithi071.stdout:5667 read 0xa583a36 thru 0xa593734 (0xfcff bytes) 2024-09-14T05:30:21.123 INFO:teuthology.orchestra.run.smithi071.stdout:5668 read 0xda7e3c0 thru 0xda8e37a (0xffbb bytes) 2024-09-14T05:30:21.123 INFO:teuthology.orchestra.run.smithi071.stdout:5669 write 0xa5cde7b thru 0xa5d61bc (0x8342 bytes) 2024-09-14T05:30:21.127 INFO:teuthology.orchestra.run.smithi071.stdout:5671 write 0xbb21847 thru 0xbb2f704 (0xdebe bytes) 2024-09-14T05:30:21.130 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.130 INFO:teuthology.orchestra.run.smithi071.stdout:5676 read 0x94a9c37 thru 0x94b50e7 (0xb4b1 bytes) 2024-09-14T05:30:21.130 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.144 INFO:teuthology.orchestra.run.smithi071.stdout:5682 trunc from 0xe48c456 to 0x3d61b4c 2024-09-14T05:30:21.152 INFO:teuthology.orchestra.run.smithi071.stdout:5683 writesame 0x39aeee0 thru 0x39b3fb6 data_size 0x1(0x50d7 bytes) 2024-09-14T05:30:21.177 INFO:teuthology.orchestra.run.smithi071.stdout:5684 writesame 0xa03763d thru 0xa0426ab data_size 0x1f(0xb06f bytes) 2024-09-14T05:30:21.177 INFO:teuthology.orchestra.run.smithi071.stdout:5686 read 0x1395d3b thru 0x13a0bd2 (0xae98 bytes) 2024-09-14T05:30:21.177 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.177 INFO:teuthology.orchestra.run.smithi071.stdout:5688 punch from 0x8d34dfd to 0x8d37c8c, (0x2e8f bytes) 2024-09-14T05:30:21.177 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.177 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.178 INFO:teuthology.orchestra.run.smithi071.stdout:5691 trunc from 0xa0426ac to 0x8c4c3a0 2024-09-14T05:30:21.184 INFO:teuthology.orchestra.run.smithi071.stdout:5692 punch from 0x3e73f39 to 0x3e75cf0, (0x1db7 bytes) 2024-09-14T05:30:21.184 INFO:teuthology.orchestra.run.smithi071.stdout:5694 write 0x7865041 thru 0x786d551 (0x8511 bytes) 2024-09-14T05:30:21.187 INFO:teuthology.orchestra.run.smithi071.stdout:5695 read 0x16db83 thru 0x17c857 (0xecd5 bytes) 2024-09-14T05:30:21.188 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.188 INFO:teuthology.orchestra.run.smithi071.stdout:5697 read 0x3b977d1 thru 0x3ba58fe (0xe12e bytes) 2024-09-14T05:30:21.188 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.188 INFO:teuthology.orchestra.run.smithi071.stdout:5701 write 0x2c2910c thru 0x2c2a059 (0xf4e bytes) 2024-09-14T05:30:21.190 INFO:teuthology.orchestra.run.smithi071.stdout:5702 write 0x21c14d0 thru 0x21c31e6 (0x1d17 bytes) 2024-09-14T05:30:21.204 INFO:teuthology.orchestra.run.smithi071.stdout:5705 writesame 0xd41dcba thru 0xd41e490 data_size 0x3(0x7d7 bytes) 2024-09-14T05:30:21.205 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.205 INFO:teuthology.orchestra.run.smithi071.stdout:5707 read 0xca2602b thru 0xca2c577 (0x654d bytes) 2024-09-14T05:30:21.205 INFO:teuthology.orchestra.run.smithi071.stdout:5708 punch from 0x120880f to 0x120de90, (0x5681 bytes) 2024-09-14T05:30:21.205 INFO:teuthology.orchestra.run.smithi071.stdout:5709 write 0xc0139b8 thru 0xc02148e (0xdad7 bytes) 2024-09-14T05:30:21.205 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:21.203+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:21.215 INFO:teuthology.orchestra.run.smithi071.stdout:5710 trunc from 0xd41e491 to 0x7366a48 2024-09-14T05:30:21.224 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:21.224 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:21.224 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:21.224 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:21.226 INFO:teuthology.orchestra.run.smithi071.stdout:5711 read 0x7139567 thru 0x7148556 (0xeff0 bytes) 2024-09-14T05:30:21.226 INFO:teuthology.orchestra.run.smithi071.stdout:5713 read 0x5648b68 thru 0x564fc6a (0x7103 bytes) 2024-09-14T05:30:21.226 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.226 INFO:teuthology.orchestra.run.smithi071.stdout:5717 read 0x2c46dcd thru 0x2c4cec0 (0x60f4 bytes) 2024-09-14T05:30:21.226 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.226 INFO:teuthology.orchestra.run.smithi071.stdout:5719 write 0x1d4a3c3 thru 0x1d559a6 (0xb5e4 bytes) 2024-09-14T05:30:21.233 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:30:21.236 INFO:teuthology.orchestra.run.smithi071.stdout:5720 trunc from 0x7366a48 to 0x121d33b 2024-09-14T05:30:21.250 INFO:teuthology.orchestra.run.smithi071.stdout:5723 punch from 0x855c33 to 0x85d724, (0x7af1 bytes) 2024-09-14T05:30:21.250 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.250 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.264 INFO:teuthology.orchestra.run.smithi071.stdout:5731 writesame 0x88a0968 thru 0x88ab0cd data_size 0x7(0xa766 bytes) 2024-09-14T05:30:21.264 INFO:teuthology.orchestra.run.smithi071.stdout:5732 punch from 0x1bc1b33 to 0x1bce535, (0xca02 bytes) 2024-09-14T05:30:21.265 INFO:teuthology.orchestra.run.smithi071.stdout:5734 write 0x425694 thru 0x42af03 (0x5870 bytes) 2024-09-14T05:30:21.266 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.266 INFO:teuthology.orchestra.run.smithi071.stdout:5738 writesame 0x63ebb68 thru 0x63f5657 data_size 0x86(0x9af0 bytes) 2024-09-14T05:30:21.279 INFO:teuthology.orchestra.run.smithi071.stdout:5739 trunc from 0x88ab0ce to 0x50b259a 2024-09-14T05:30:21.289 INFO:teuthology.orchestra.run.smithi071.stdout:5741 read 0x18c35c thru 0x18d2c7 (0xf6c bytes) 2024-09-14T05:30:21.290 INFO:teuthology.orchestra.run.smithi071.stdout:5742 read 0x3cbf067 thru 0x3cc8d05 (0x9c9f bytes) 2024-09-14T05:30:21.302 INFO:teuthology.orchestra.run.smithi071.stdout:5743 writesame 0xac6e35c thru 0xac75a7d data_size 0x11(0x7722 bytes) 2024-09-14T05:30:21.305 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:21.304+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:21.312 INFO:teuthology.orchestra.run.smithi071.stdout:5745 write 0xe30f430 thru 0xe31b2a7 (0xbe78 bytes) 2024-09-14T05:30:21.337 INFO:teuthology.orchestra.run.smithi071.stdout:5746 trunc from 0xe31b2a8 to 0x15ea6f6 2024-09-14T05:30:21.340 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:21.340 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:21.340 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:21.340 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:21.340 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:21.348 INFO:teuthology.orchestra.run.smithi071.stdout:5747 read 0xd891a0 thru 0xd97a35 (0xe896 bytes) 2024-09-14T05:30:21.349 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:30:21.350 INFO:teuthology.orchestra.run.smithi071.stdout:5750 punch from 0x1061ff5 to 0x1066042, (0x404d bytes) 2024-09-14T05:30:21.350 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.350 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.350 INFO:teuthology.orchestra.run.smithi071.stdout:5755 read 0xa4b108 thru 0xa52f4f (0x7e48 bytes) 2024-09-14T05:30:21.350 INFO:teuthology.orchestra.run.smithi071.stdout:5756 punch from 0x4c5da3 to 0x4d061a, (0xa877 bytes) 2024-09-14T05:30:21.350 INFO:teuthology.orchestra.run.smithi071.stdout:5757 flatten 2024-09-14T05:30:21.355 INFO:teuthology.orchestra.run.smithi071.stdout:5758 read 0x14896c2 thru 0x14913c6 (0x7d05 bytes) 2024-09-14T05:30:21.355 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.355 INFO:teuthology.orchestra.run.smithi071.stdout:5761 read 0x14ae12b thru 0x14bcf83 (0xee59 bytes) 2024-09-14T05:30:21.355 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.355 INFO:teuthology.orchestra.run.smithi071.stdout:5765 read 0xf9c08c thru 0xf9d505 (0x147a bytes) 2024-09-14T05:30:21.378 INFO:teuthology.orchestra.run.smithi071.stdout:5766 writesame 0xddabe52 thru 0xddadc50 data_size 0x1(0x1dff bytes) 2024-09-14T05:30:21.378 INFO:teuthology.orchestra.run.smithi071.stdout:5767 punch from 0x7565215 to 0x756dd7f, (0x8b6a bytes) 2024-09-14T05:30:21.379 INFO:teuthology.orchestra.run.smithi071.stdout:5768 read 0x8ab6029 thru 0x8ac1241 (0xb219 bytes) 2024-09-14T05:30:21.379 INFO:teuthology.orchestra.run.smithi071.stdout:5769 read 0x316145 thru 0x31d588 (0x7444 bytes) 2024-09-14T05:30:21.379 INFO:teuthology.orchestra.run.smithi071.stdout:5771 writesame 0x2135175 thru 0x21351c4 data_size 0x5(0x50 bytes) 2024-09-14T05:30:21.382 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.382 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.388 INFO:teuthology.orchestra.run.smithi071.stdout:5776 trunc from 0xddadc51 to 0x8a73712 2024-09-14T05:30:21.394 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.394 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.394 INFO:teuthology.orchestra.run.smithi071.stdout:5780 write 0x7712ada thru 0x7712b5a (0x81 bytes) 2024-09-14T05:30:21.396 INFO:teuthology.orchestra.run.smithi071.stdout:5781 writesame 0x2809886 thru 0x280ac1a data_size 0x3(0x1395 bytes) 2024-09-14T05:30:21.399 INFO:teuthology.orchestra.run.smithi071.stdout:5783 punch from 0x2a2b228 to 0x2a32a9b, (0x7873 bytes) 2024-09-14T05:30:21.404 INFO:teuthology.orchestra.run.smithi071.stdout:5785 trunc from 0x8a73712 to 0x4d90f6b 2024-09-14T05:30:21.414 INFO:teuthology.orchestra.run.smithi071.stdout:5786 write 0x1f81351 thru 0x1f88a7a (0x772a bytes) 2024-09-14T05:30:21.423 INFO:teuthology.orchestra.run.smithi071.stdout:5787 write 0x5908d1e thru 0x59132d7 (0xa5ba bytes) 2024-09-14T05:30:21.426 INFO:teuthology.orchestra.run.smithi071.stdout:5788 read 0x2708a70 thru 0x2710939 (0x7eca bytes) 2024-09-14T05:30:21.426 INFO:teuthology.orchestra.run.smithi071.stdout:5789 punch from 0x546c970 to 0x546d034, (0x6c4 bytes) 2024-09-14T05:30:21.433 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:21.433+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:21.437 INFO:teuthology.orchestra.run.smithi071.stdout:5790 write 0x94c18ad thru 0x94c8bbc (0x7310 bytes) 2024-09-14T05:30:21.440 INFO:teuthology.orchestra.run.smithi071.stdout:5791 read 0x255e494 thru 0x25652f6 (0x6e63 bytes) 2024-09-14T05:30:21.440 INFO:teuthology.orchestra.run.smithi071.stdout:5793 read 0x71f3426 thru 0x71fdda2 (0xa97d bytes) 2024-09-14T05:30:21.440 INFO:teuthology.orchestra.run.smithi071.stdout:5794 read 0x7117bd5 thru 0x712186e (0x9c9a bytes) 2024-09-14T05:30:21.440 INFO:teuthology.orchestra.run.smithi071.stdout:5796 read 0x833d492 thru 0x834378c (0x62fb bytes) 2024-09-14T05:30:21.447 INFO:teuthology.orchestra.run.smithi071.stdout:5797 write 0xa564669 thru 0xa56eb9e (0xa536 bytes) 2024-09-14T05:30:21.452 INFO:teuthology.orchestra.run.smithi071.stdout:5799 write 0x9c5bb7d thru 0x9c65842 (0x9cc6 bytes) 2024-09-14T05:30:21.458 INFO:teuthology.orchestra.run.smithi071.stdout:5802 writesame 0x632b3f2 thru 0x6339eed data_size 0x36(0xeafc bytes) 2024-09-14T05:30:21.464 INFO:teuthology.orchestra.run.smithi071.stdout:5803 trunc from 0xa56eb9f to 0x89e80f2 2024-09-14T05:30:21.474 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.474 INFO:teuthology.orchestra.run.smithi071.stdout:5809 write 0x1f0ba5 thru 0x1f37e6 (0x2c42 bytes) 2024-09-14T05:30:21.475 INFO:teuthology.orchestra.run.smithi071.stdout:5810 read 0xd680de thru 0xd762f0 (0xe213 bytes) 2024-09-14T05:30:21.480 INFO:teuthology.orchestra.run.smithi071.stdout:5811 trunc from 0x89e80f2 to 0xc35d319 2024-09-14T05:30:21.486 INFO:teuthology.orchestra.run.smithi071.stdout:5812 writesame 0xb7e3572 thru 0xb7e571e data_size 0x1(0x21ad bytes) 2024-09-14T05:30:21.499 INFO:teuthology.orchestra.run.smithi071.stdout:5813 write 0xde75d63 thru 0xde80488 (0xa726 bytes) 2024-09-14T05:30:21.500 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:21.500 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:21.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:21.500 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:21.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.501 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3523 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6062080~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.501 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:21.501 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.984452367000003, 2024-09-14T05:30:21.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096911030999999995, 2024-09-14T05:30:21.501 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.501 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.501 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.501 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.501 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.501 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.501 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3523 2024-09-14T05:30:21.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.502 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:21.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:21.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514359+0000", 2024-09-14T05:30:21.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9299999999999999e-07 2024-09-14T05:30:21.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514361+0000", 2024-09-14T05:30:21.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:21.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514366+0000", 2024-09-14T05:30:21.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1660000000000002e-06 2024-09-14T05:30:21.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514371+0000", 2024-09-14T05:30:21.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.648e-06 2024-09-14T05:30:21.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.610954+0000", 2024-09-14T05:30:21.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096582170999999994 2024-09-14T05:30:21.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611002+0000", 2024-09-14T05:30:21.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7923999999999997e-05 2024-09-14T05:30:21.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611269+0000", 2024-09-14T05:30:21.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026789499999999998 2024-09-14T05:30:21.505 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.505 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.505 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.505 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3524 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6553600~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.505 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:21.506 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.984433174999999, 2024-09-14T05:30:21.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097263347999999999, 2024-09-14T05:30:21.506 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.506 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.506 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.506 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.506 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.506 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.506 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3524 2024-09-14T05:30:21.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.506 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:21.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:21.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:21.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:21.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514380+0000", 2024-09-14T05:30:21.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6500000000000001e-06 2024-09-14T05:30:21.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514386+0000", 2024-09-14T05:30:21.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7360000000000004e-06 2024-09-14T05:30:21.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514391+0000", 2024-09-14T05:30:21.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2830000000000003e-06 2024-09-14T05:30:21.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611295+0000", 2024-09-14T05:30:21.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096904814000000006 2024-09-14T05:30:21.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611343+0000", 2024-09-14T05:30:21.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7506999999999998e-05 2024-09-14T05:30:21.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611641+0000", 2024-09-14T05:30:21.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029798800000000001 2024-09-14T05:30:21.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.510 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.510 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3525 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7045120~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.510 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:21.510 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.984411287999997, 2024-09-14T05:30:21.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097644422999999994, 2024-09-14T05:30:21.510 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.510 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.510 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.510 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.511 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3525 2024-09-14T05:30:21.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.511 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:21.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:21.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514400+0000", 2024-09-14T05:30:21.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.77e-07 2024-09-14T05:30:21.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514402+0000", 2024-09-14T05:30:21.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6530000000000001e-06 2024-09-14T05:30:21.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514411+0000", 2024-09-14T05:30:21.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7429999999999997e-06 2024-09-14T05:30:21.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514417+0000", 2024-09-14T05:30:21.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2249999999999997e-06 2024-09-14T05:30:21.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611668+0000", 2024-09-14T05:30:21.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097250681000000005 2024-09-14T05:30:21.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611716+0000", 2024-09-14T05:30:21.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7336000000000002e-05 2024-09-14T05:30:21.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612044+0000", 2024-09-14T05:30:21.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032840799999999998 2024-09-14T05:30:21.514 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.514 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.514 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.514 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3526 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7536640~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.514 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:21.514 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.984386804000003, 2024-09-14T05:30:21.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097943884999999994, 2024-09-14T05:30:21.515 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.515 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.515 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.515 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3526 2024-09-14T05:30:21.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.515 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:21.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:21.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:21.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0400000000000002e-07 2024-09-14T05:30:21.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514426+0000", 2024-09-14T05:30:21.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:21.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514432+0000", 2024-09-14T05:30:21.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.105e-06 2024-09-14T05:30:21.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514438+0000", 2024-09-14T05:30:21.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5149999999999997e-06 2024-09-14T05:30:21.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612072+0000", 2024-09-14T05:30:21.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097633947999999998 2024-09-14T05:30:21.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612120+0000", 2024-09-14T05:30:21.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8458999999999998e-05 2024-09-14T05:30:21.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612368+0000", 2024-09-14T05:30:21.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024761999999999999 2024-09-14T05:30:21.518 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.519 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.519 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.519 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3527 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 8028160~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.519 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:21.519 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.984366090999998, 2024-09-14T05:30:21.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098244955999999994, 2024-09-14T05:30:21.519 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.519 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.519 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.519 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3527 2024-09-14T05:30:21.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.520 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:21.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:21.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519842+0000", 2024-09-14T05:30:21.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053975230000000004 2024-09-14T05:30:21.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519851+0000", 2024-09-14T05:30:21.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5599999999999994e-06 2024-09-14T05:30:21.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519858+0000", 2024-09-14T05:30:21.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2409999999999998e-06 2024-09-14T05:30:21.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519863+0000", 2024-09-14T05:30:21.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:30:21.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612395+0000", 2024-09-14T05:30:21.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092532552000000004 2024-09-14T05:30:21.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612442+0000", 2024-09-14T05:30:21.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6486000000000002e-05 2024-09-14T05:30:21.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612690+0000", 2024-09-14T05:30:21.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024791599999999999 2024-09-14T05:30:21.523 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.523 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.523 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.523 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3603 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5177344~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.523 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:21.523 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.978813645999999, 2024-09-14T05:30:21.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096796436, 2024-09-14T05:30:21.524 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.524 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.524 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.524 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.524 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.524 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.524 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3603 2024-09-14T05:30:21.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.524 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:21.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:21.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:21.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1899999999999999e-07 2024-09-14T05:30:21.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519998+0000", 2024-09-14T05:30:21.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5000000000000001e-07 2024-09-14T05:30:21.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520002+0000", 2024-09-14T05:30:21.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7299999999999999e-06 2024-09-14T05:30:21.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520004+0000", 2024-09-14T05:30:21.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.429e-06 2024-09-14T05:30:21.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616493+0000", 2024-09-14T05:30:21.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096488345000000003 2024-09-14T05:30:21.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616544+0000", 2024-09-14T05:30:21.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1724000000000001e-05 2024-09-14T05:30:21.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616794+0000", 2024-09-14T05:30:21.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000249139 2024-09-14T05:30:21.527 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.527 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.527 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.528 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3604 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5668864~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.528 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:21.528 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.978799008000003, 2024-09-14T05:30:21.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108222999999994, 2024-09-14T05:30:21.528 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.528 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.528 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.528 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.528 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.528 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.528 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3604 2024-09-14T05:30:21.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.529 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:21.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:21.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:21.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7000000000000001e-07 2024-09-14T05:30:21.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520013+0000", 2024-09-14T05:30:21.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.389e-06 2024-09-14T05:30:21.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520018+0000", 2024-09-14T05:30:21.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3019999999999996e-06 2024-09-14T05:30:21.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520026+0000", 2024-09-14T05:30:21.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6170000000000007e-06 2024-09-14T05:30:21.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616816+0000", 2024-09-14T05:30:21.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096789766999999999 2024-09-14T05:30:21.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616871+0000", 2024-09-14T05:30:21.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4811000000000001e-05 2024-09-14T05:30:21.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617120+0000", 2024-09-14T05:30:21.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024906700000000001 2024-09-14T05:30:21.532 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.532 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.532 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.532 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3605 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6160384~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.532 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:21.532 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.978780548000003, 2024-09-14T05:30:21.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097423945999999997, 2024-09-14T05:30:21.532 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.532 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.532 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.533 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3605 2024-09-14T05:30:21.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.533 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:21.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:21.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:21.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3999999999999998e-07 2024-09-14T05:30:21.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520032+0000", 2024-09-14T05:30:21.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.083e-06 2024-09-14T05:30:21.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520036+0000", 2024-09-14T05:30:21.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.019e-06 2024-09-14T05:30:21.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520038+0000", 2024-09-14T05:30:21.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4849999999999999e-06 2024-09-14T05:30:21.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617147+0000", 2024-09-14T05:30:21.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108665999999996 2024-09-14T05:30:21.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617199+0000", 2024-09-14T05:30:21.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2558000000000001e-05 2024-09-14T05:30:21.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617454+0000", 2024-09-14T05:30:21.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025489499999999999 2024-09-14T05:30:21.536 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.536 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.536 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.537 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3606 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6651904~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.537 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:21.537 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.978769825999997, 2024-09-14T05:30:21.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097745705000000002, 2024-09-14T05:30:21.537 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.537 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.537 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.537 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.537 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.537 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.537 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3606 2024-09-14T05:30:21.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.537 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:21.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:21.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:21.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-14T05:30:21.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520042+0000", 2024-09-14T05:30:21.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4799999999999997e-07 2024-09-14T05:30:21.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520046+0000", 2024-09-14T05:30:21.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8149999999999999e-06 2024-09-14T05:30:21.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520049+0000", 2024-09-14T05:30:21.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.576e-06 2024-09-14T05:30:21.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617476+0000", 2024-09-14T05:30:21.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097427558999999997 2024-09-14T05:30:21.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617522+0000", 2024-09-14T05:30:21.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5927999999999998e-05 2024-09-14T05:30:21.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617787+0000", 2024-09-14T05:30:21.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000264663 2024-09-14T05:30:21.541 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.541 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.541 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.541 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3607 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7143424~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.541 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:21.541 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.978759299000004, 2024-09-14T05:30:21.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097871125000000003, 2024-09-14T05:30:21.541 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.541 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.541 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.541 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.541 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.542 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.542 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3607 2024-09-14T05:30:21.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.542 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:21.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:21.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520052+0000", 2024-09-14T05:30:21.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.03e-07 2024-09-14T05:30:21.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520053+0000", 2024-09-14T05:30:21.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.85e-07 2024-09-14T05:30:21.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520056+0000", 2024-09-14T05:30:21.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9140000000000001e-06 2024-09-14T05:30:21.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520059+0000", 2024-09-14T05:30:21.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4600000000000002e-06 2024-09-14T05:30:21.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617809+0000", 2024-09-14T05:30:21.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097750216000000001 2024-09-14T05:30:21.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617864+0000", 2024-09-14T05:30:21.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5211999999999999e-05 2024-09-14T05:30:21.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617923+0000", 2024-09-14T05:30:21.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8235000000000001e-05 2024-09-14T05:30:21.545 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.545 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.545 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.545 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3608 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7634944~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.545 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:21.545 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.978748991000003, 2024-09-14T05:30:21.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098199212999999994, 2024-09-14T05:30:21.546 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.546 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.546 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.546 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.546 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.546 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.546 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3608 2024-09-14T05:30:21.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.546 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:21.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:21.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525953+0000", 2024-09-14T05:30:21.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058911850000000002 2024-09-14T05:30:21.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525957+0000", 2024-09-14T05:30:21.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2379999999999999e-06 2024-09-14T05:30:21.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525967+0000", 2024-09-14T05:30:21.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5389999999999996e-06 2024-09-14T05:30:21.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525972+0000", 2024-09-14T05:30:21.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.541e-06 2024-09-14T05:30:21.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617945+0000", 2024-09-14T05:30:21.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091972597000000003 2024-09-14T05:30:21.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617992+0000", 2024-09-14T05:30:21.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7397e-05 2024-09-14T05:30:21.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.618261+0000", 2024-09-14T05:30:21.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026871600000000001 2024-09-14T05:30:21.549 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.550 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.550 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.550 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3640 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6586368~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.550 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:21.550 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.972641148999998, 2024-09-14T05:30:21.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097068580000000002, 2024-09-14T05:30:21.550 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.550 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.550 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.551 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3640 2024-09-14T05:30:21.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.551 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:21.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:21.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:21.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.23e-07 2024-09-14T05:30:21.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526171+0000", 2024-09-14T05:30:21.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:30:21.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526175+0000", 2024-09-14T05:30:21.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8369999999999999e-06 2024-09-14T05:30:21.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526177+0000", 2024-09-14T05:30:21.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4959999999999999e-06 2024-09-14T05:30:21.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622847+0000", 2024-09-14T05:30:21.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096669790000000005 2024-09-14T05:30:21.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622902+0000", 2024-09-14T05:30:21.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5157000000000001e-05 2024-09-14T05:30:21.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623238+0000", 2024-09-14T05:30:21.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033603399999999997 2024-09-14T05:30:21.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.554 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.554 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3641 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7077888~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.554 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:21.554 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.972630639000002, 2024-09-14T05:30:21.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097371141999999994, 2024-09-14T05:30:21.554 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.555 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.555 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.555 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3641 2024-09-14T05:30:21.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.555 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:21.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:21.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531851+0000", 2024-09-14T05:30:21.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005671265 2024-09-14T05:30:21.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531855+0000", 2024-09-14T05:30:21.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0849999999999999e-06 2024-09-14T05:30:21.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531867+0000", 2024-09-14T05:30:21.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1028e-05 2024-09-14T05:30:21.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531872+0000", 2024-09-14T05:30:21.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000003e-06 2024-09-14T05:30:21.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623251+0000", 2024-09-14T05:30:21.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091378980999999998 2024-09-14T05:30:21.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623295+0000", 2024-09-14T05:30:21.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3881999999999998e-05 2024-09-14T05:30:21.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623551+0000", 2024-09-14T05:30:21.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025636099999999998 2024-09-14T05:30:21.558 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.558 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.558 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.559 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3721 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.559 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:21.559 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.966772364000001, 2024-09-14T05:30:21.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096806472000000005, 2024-09-14T05:30:21.559 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.560 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.560 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.560 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.560 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.560 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.560 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3721 2024-09-14T05:30:21.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.561 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:21.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:21.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532039+0000", 2024-09-14T05:30:21.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.05e-07 2024-09-14T05:30:21.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532040+0000", 2024-09-14T05:30:21.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9099999999999991e-07 2024-09-14T05:30:21.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532043+0000", 2024-09-14T05:30:21.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6610000000000002e-06 2024-09-14T05:30:21.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532046+0000", 2024-09-14T05:30:21.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4080000000000001e-06 2024-09-14T05:30:21.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628545+0000", 2024-09-14T05:30:21.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096499615999999996 2024-09-14T05:30:21.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628592+0000", 2024-09-14T05:30:21.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6907000000000003e-05 2024-09-14T05:30:21.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628845+0000", 2024-09-14T05:30:21.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025268399999999998 2024-09-14T05:30:21.564 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.564 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.564 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.564 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3722 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6193152~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.564 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:21.564 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.966762283999998, 2024-09-14T05:30:21.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097146587000000006, 2024-09-14T05:30:21.565 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.565 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.565 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.565 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.565 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.565 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.565 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3722 2024-09-14T05:30:21.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.565 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:21.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:21.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538191+0000", 2024-09-14T05:30:21.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061429969999999999 2024-09-14T05:30:21.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538200+0000", 2024-09-14T05:30:21.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7069999999999998e-06 2024-09-14T05:30:21.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538212+0000", 2024-09-14T05:30:21.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1504e-05 2024-09-14T05:30:21.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538219+0000", 2024-09-14T05:30:21.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7500000000000003e-06 2024-09-14T05:30:21.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628869+0000", 2024-09-14T05:30:21.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090649224 2024-09-14T05:30:21.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628919+0000", 2024-09-14T05:30:21.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0144e-05 2024-09-14T05:30:21.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.629195+0000", 2024-09-14T05:30:21.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027626099999999998 2024-09-14T05:30:21.568 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.568 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.568 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.569 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3758 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7110656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.569 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:21.569 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.960168068000002, 2024-09-14T05:30:21.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096977544999999998, 2024-09-14T05:30:21.569 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.569 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.569 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.569 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.569 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.569 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.569 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3758 2024-09-14T05:30:21.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.569 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:21.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:21.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:21.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:21.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538645+0000", 2024-09-14T05:30:21.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.843e-06 2024-09-14T05:30:21.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538651+0000", 2024-09-14T05:30:21.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6169999999999998e-06 2024-09-14T05:30:21.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538655+0000", 2024-09-14T05:30:21.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5399999999999997e-06 2024-09-14T05:30:21.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635225+0000", 2024-09-14T05:30:21.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096570149999999993 2024-09-14T05:30:21.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635272+0000", 2024-09-14T05:30:21.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7199e-05 2024-09-14T05:30:21.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635620+0000", 2024-09-14T05:30:21.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034782600000000003 2024-09-14T05:30:21.573 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.573 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.573 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.573 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3759 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.573 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:21.573 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.960150157000001, 2024-09-14T05:30:21.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097335331999999997, 2024-09-14T05:30:21.573 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.573 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.574 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.574 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.574 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.574 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.574 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3759 2024-09-14T05:30:21.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.574 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:21.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:21.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544013+0000", 2024-09-14T05:30:21.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053520829999999997 2024-09-14T05:30:21.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544016+0000", 2024-09-14T05:30:21.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6660000000000001e-06 2024-09-14T05:30:21.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544026+0000", 2024-09-14T05:30:21.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7089999999999997e-06 2024-09-14T05:30:21.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544043+0000", 2024-09-14T05:30:21.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833999999999998e-05 2024-09-14T05:30:21.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635646+0000", 2024-09-14T05:30:21.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091603544999999995 2024-09-14T05:30:21.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635696+0000", 2024-09-14T05:30:21.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9307000000000001e-05 2024-09-14T05:30:21.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635996+0000", 2024-09-14T05:30:21.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030018800000000001 2024-09-14T05:30:21.577 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.577 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.577 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.578 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3838 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6225920~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.578 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:21.578 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.954488476999998, 2024-09-14T05:30:21.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096894235999999995, 2024-09-14T05:30:21.578 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.578 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.578 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.578 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3838 2024-09-14T05:30:21.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.578 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:21.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:21.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544323+0000", 2024-09-14T05:30:21.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-14T05:30:21.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544325+0000", 2024-09-14T05:30:21.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.75e-06 2024-09-14T05:30:21.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544331+0000", 2024-09-14T05:30:21.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5019999999999998e-06 2024-09-14T05:30:21.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544335+0000", 2024-09-14T05:30:21.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2989999999999998e-06 2024-09-14T05:30:21.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640869+0000", 2024-09-14T05:30:21.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096533351000000003 2024-09-14T05:30:21.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640919+0000", 2024-09-14T05:30:21.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999999e-05 2024-09-14T05:30:21.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641217+0000", 2024-09-14T05:30:21.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029717600000000002 2024-09-14T05:30:21.582 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.582 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.582 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.582 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3839 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.582 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:21.582 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.954469539000002, 2024-09-14T05:30:21.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097240339999999995, 2024-09-14T05:30:21.582 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.582 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.583 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.583 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3839 2024-09-14T05:30:21.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.583 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:21.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:21.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549834+0000", 2024-09-14T05:30:21.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054927819999999999 2024-09-14T05:30:21.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549838+0000", 2024-09-14T05:30:21.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8689999999999997e-06 2024-09-14T05:30:21.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549848+0000", 2024-09-14T05:30:21.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0154e-05 2024-09-14T05:30:21.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549853+0000", 2024-09-14T05:30:21.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9740000000000001e-06 2024-09-14T05:30:21.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641245+0000", 2024-09-14T05:30:21.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091392245999999996 2024-09-14T05:30:21.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641295+0000", 2024-09-14T05:30:21.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0018999999999997e-05 2024-09-14T05:30:21.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641582+0000", 2024-09-14T05:30:21.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028629600000000001 2024-09-14T05:30:21.586 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.586 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.586 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.586 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3868 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.587 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:21.587 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.948806961000003, 2024-09-14T05:30:21.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096859555999999999, 2024-09-14T05:30:21.587 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.587 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.587 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.587 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.587 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.587 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.587 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3868 2024-09-14T05:30:21.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.587 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:21.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:21.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556173+0000", 2024-09-14T05:30:21.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006169038 2024-09-14T05:30:21.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556177+0000", 2024-09-14T05:30:21.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0010000000000003e-06 2024-09-14T05:30:21.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556188+0000", 2024-09-14T05:30:21.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0675e-05 2024-09-14T05:30:21.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556199+0000", 2024-09-14T05:30:21.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.155e-05 2024-09-14T05:30:21.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646511+0000", 2024-09-14T05:30:21.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090312187000000002 2024-09-14T05:30:21.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646563+0000", 2024-09-14T05:30:21.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2219000000000003e-05 2024-09-14T05:30:21.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646863+0000", 2024-09-14T05:30:21.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029988600000000002 2024-09-14T05:30:21.590 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.591 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.591 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.591 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.591 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.591 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:21.593 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:30:21.595 INFO:teuthology.orchestra.run.smithi071.stdout:5815 punch from 0x6183c2 to 0x6249ea, (0xc628 bytes) 2024-09-14T05:30:21.595 INFO:teuthology.orchestra.run.smithi071.stdout:5816 writesame 0x1f3b12e thru 0x1f48719 data_size 0x2(0xd5ec bytes) 2024-09-14T05:30:21.595 INFO:teuthology.orchestra.run.smithi071.stdout:5818 read 0xcf181af thru 0xcf2258b (0xa3dd bytes) 2024-09-14T05:30:21.595 INFO:teuthology.orchestra.run.smithi071.stdout:5820 write 0x14cfe04 thru 0x14d10b9 (0x12b6 bytes) 2024-09-14T05:30:21.595 INFO:teuthology.orchestra.run.smithi071.stdout:5821 trunc from 0xde80489 to 0x94384a9 2024-09-14T05:30:21.595 INFO:teuthology.orchestra.run.smithi071.stdout:5822 punch from 0x4b4397e to 0x4b4d669, (0x9ceb bytes) 2024-09-14T05:30:21.595 INFO:teuthology.orchestra.run.smithi071.stdout:5823 writesame 0xcc0c46f thru 0xcc0d32d data_size 0x5(0xebf bytes) 2024-09-14T05:30:21.595 INFO:teuthology.orchestra.run.smithi071.stdout:5825 write 0xa3a8927 thru 0xa3b1432 (0x8b0c bytes) 2024-09-14T05:30:21.595 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.595 INFO:teuthology.orchestra.run.smithi071.stdout:5827 read 0x3d42da thru 0x3ddc21 (0x9948 bytes) 2024-09-14T05:30:21.596 INFO:teuthology.orchestra.run.smithi071.stdout:5828 write 0xa422221 thru 0xa42e580 (0xc360 bytes) 2024-09-14T05:30:21.596 INFO:teuthology.orchestra.run.smithi071.stdout:5829 punch from 0x205d90a to 0x206030b, (0x2a01 bytes) 2024-09-14T05:30:21.596 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.596 INFO:teuthology.orchestra.run.smithi071.stdout:5831 read 0x1fb5c9e thru 0x1fc3d24 (0xe087 bytes) 2024-09-14T05:30:21.596 INFO:teuthology.orchestra.run.smithi071.stdout:5832 read 0x5ea2250 thru 0x5eae81a (0xc5cb bytes) 2024-09-14T05:30:21.596 INFO:teuthology.orchestra.run.smithi071.stdout:5833 trunc from 0xcc0d32e to 0xe2ae08f 2024-09-14T05:30:21.596 INFO:teuthology.orchestra.run.smithi071.stdout:5834 write 0x59b970e thru 0x59c3d3d (0xa630 bytes) 2024-09-14T05:30:21.596 INFO:teuthology.orchestra.run.smithi071.stdout:5835 writesame 0xebc3f19 thru 0xebcaa6f data_size 0x1(0x6b57 bytes) 2024-09-14T05:30:21.596 INFO:teuthology.orchestra.run.smithi071.stdout:5836 trunc from 0xebcaa70 to 0xcd52971 2024-09-14T05:30:21.596 INFO:teuthology.orchestra.run.smithi071.stdout:5837 read 0x914fba4 thru 0x915d2e2 (0xd73f bytes) 2024-09-14T05:30:21.596 INFO:teuthology.orchestra.run.smithi071.stdout:5838 punch from 0x2bef8dc to 0x2bfcf72, (0xd696 bytes) 2024-09-14T05:30:21.596 INFO:teuthology.orchestra.run.smithi071.stdout:5839 punch from 0x133cd53 to 0x1348ec3, (0xc170 bytes) 2024-09-14T05:30:21.596 INFO:teuthology.orchestra.run.smithi071.stdout:5840 write 0xca53eeb thru 0xca5f86e (0xb984 bytes) 2024-09-14T05:30:21.596 INFO:teuthology.orchestra.run.smithi071.stdout:5841 read 0xa94910c thru 0xa94d378 (0x426d bytes) 2024-09-14T05:30:21.597 INFO:teuthology.orchestra.run.smithi071.stdout:5842 trunc from 0xcd52971 to 0xb5e1f89 2024-09-14T05:30:21.597 INFO:teuthology.orchestra.run.smithi071.stdout:5843 punch from 0x9736431 to 0x973645b, (0x2a bytes) 2024-09-14T05:30:21.597 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.597 INFO:teuthology.orchestra.run.smithi071.stdout:5846 punch from 0xa03970 to 0xa0eac4, (0xb154 bytes) 2024-09-14T05:30:21.597 INFO:teuthology.orchestra.run.smithi071.stdout:5847 punch from 0x19b692a to 0x19c0c6d, (0xa343 bytes) 2024-09-14T05:30:21.597 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:21.591+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:21.597 INFO:teuthology.orchestra.run.smithi071.stdout:5848 write 0xe318d02 thru 0xe32767b (0xe97a bytes) 2024-09-14T05:30:21.602 INFO:teuthology.orchestra.run.smithi071.stdout:5849 punch from 0x16b6380 to 0x16ba436, (0x40b6 bytes) 2024-09-14T05:30:21.619 INFO:teuthology.orchestra.run.smithi071.stdout:5850 trunc from 0xe32767c to 0x705710 2024-09-14T05:30:21.643 INFO:teuthology.orchestra.run.smithi071.stdout:5851 write 0x2750484 thru 0x2759df2 (0x996f bytes) 2024-09-14T05:30:21.662 INFO:teuthology.orchestra.run.smithi071.stdout:5852 writesame 0xc56d794 thru 0xc57260e data_size 0x25(0x4e7b bytes) 2024-09-14T05:30:21.662 INFO:teuthology.orchestra.run.smithi071.stdout:5853 punch from 0x49f9084 to 0x49fac43, (0x1bbf bytes) 2024-09-14T05:30:21.669 INFO:teuthology.orchestra.run.smithi071.stdout:5855 write 0xd7a17b9 thru 0xd7b033e (0xeb86 bytes) 2024-09-14T05:30:21.672 INFO:teuthology.orchestra.run.smithi071.stdout:5856 read 0x3b66536 thru 0x3b7058a (0xa055 bytes) 2024-09-14T05:30:21.673 INFO:teuthology.orchestra.run.smithi071.stdout:5861 trunc from 0xd7b033f to 0xe4d0ce4 2024-09-14T05:30:21.679 INFO:teuthology.orchestra.run.smithi071.stdout:5862 punch from 0x3882e8e to 0x3892785, (0xf8f7 bytes) 2024-09-14T05:30:21.679 INFO:teuthology.orchestra.run.smithi071.stdout:5863 write 0xaa7208f thru 0xaa76a34 (0x49a6 bytes) 2024-09-14T05:30:21.682 INFO:teuthology.orchestra.run.smithi071.stdout:5865 write 0x34e2660 thru 0x34e9492 (0x6e33 bytes) 2024-09-14T05:30:21.693 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:21.692+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:21.698 INFO:teuthology.orchestra.run.smithi071.stdout:5867 trunc from 0xe4d0ce4 to 0x3528d46 2024-09-14T05:30:21.709 INFO:teuthology.orchestra.run.smithi071.stdout:5868 read 0x1948bf5 thru 0x194aff9 (0x2405 bytes) 2024-09-14T05:30:21.709 INFO:teuthology.orchestra.run.smithi071.stdout:5869 read 0x96dd94 thru 0x97617c (0x83e9 bytes) 2024-09-14T05:30:21.709 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.709 INFO:teuthology.orchestra.run.smithi071.stdout:5872 read 0x96109d thru 0x9635fa (0x255e bytes) 2024-09-14T05:30:21.717 INFO:teuthology.orchestra.run.smithi071.stdout:5873 write 0x59ecf35 thru 0x59f520e (0x82da bytes) 2024-09-14T05:30:21.723 INFO:teuthology.orchestra.run.smithi071.stdout:5874 read 0x4b37d0d thru 0x4b4162f (0x9923 bytes) 2024-09-14T05:30:21.723 INFO:teuthology.orchestra.run.smithi071.stdout:5875 read 0x2380817 thru 0x238afc8 (0xa7b2 bytes) 2024-09-14T05:30:21.723 INFO:teuthology.orchestra.run.smithi071.stdout:5876 write 0x2e22982 thru 0x2e25a5d (0x30dc bytes) 2024-09-14T05:30:21.726 INFO:teuthology.orchestra.run.smithi071.stdout:5877 writesame 0x4fbd7af thru 0x4fc0eb4 data_size 0x1(0x3706 bytes) 2024-09-14T05:30:21.734 INFO:teuthology.orchestra.run.smithi071.stdout:5878 writesame 0x4195136 thru 0x419dd1e data_size 0x1(0x8be9 bytes) 2024-09-14T05:30:21.739 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:21.739 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:21.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:135116 3.0 3:06afecc8:::rbd_data.112dd6b36f9a.000000000000001a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T05:30:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:21.736594+0000", 2024-09-14T05:30:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0029360440000000001, 2024-09-14T05:30:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002954436, 2024-09-14T05:30:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 135116 2024-09-14T05:30:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:21.736594+0000", 2024-09-14T05:30:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:21.736594+0000", 2024-09-14T05:30:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:21.736595+0000", 2024-09-14T05:30:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2240000000000001e-06 2024-09-14T05:30:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:21.736601+0000", 2024-09-14T05:30:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5300000000000004e-06 2024-09-14T05:30:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:21.736614+0000", 2024-09-14T05:30:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2756e-05 2024-09-14T05:30:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:21.736620+0000", 2024-09-14T05:30:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8259999999999998e-06 2024-09-14T05:30:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:21.736675+0000", 2024-09-14T05:30:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5078999999999999e-05 2024-09-14T05:30:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:21.736751+0000", 2024-09-14T05:30:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6658999999999999e-05 2024-09-14T05:30:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.744 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.744 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-14T05:30:21.744 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 1 2024-09-14T05:30:21.744 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:21.751 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:30:21.766 INFO:teuthology.orchestra.run.smithi071.stdout:5880 trunc from 0x59f520f to 0xe4f619e 2024-09-14T05:30:21.771 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.784 INFO:teuthology.orchestra.run.smithi071.stdout:5883 trunc from 0xe4f619e to 0x32e82b3 2024-09-14T05:30:21.793 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:21.793+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:21.796 INFO:teuthology.orchestra.run.smithi071.stdout:5884 read 0x62b9a3 thru 0x639d64 (0xe3c2 bytes) 2024-09-14T05:30:21.797 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.797 INFO:teuthology.orchestra.run.smithi071.stdout:5886 read 0x1683ef5 thru 0x168c011 (0x811d bytes) 2024-09-14T05:30:21.797 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.797 INFO:teuthology.orchestra.run.smithi071.stdout:5889 read 0x15c84e7 thru 0x15ce741 (0x625b bytes) 2024-09-14T05:30:21.797 INFO:teuthology.orchestra.run.smithi071.stdout:5891 read 0xedd530 thru 0xee1692 (0x4163 bytes) 2024-09-14T05:30:21.812 INFO:teuthology.orchestra.run.smithi071.stdout:5892 write 0xae21190 thru 0xae2c8da (0xb74b bytes) 2024-09-14T05:30:21.815 INFO:teuthology.orchestra.run.smithi071.stdout:5894 read 0x58c7717 thru 0x58cfd66 (0x8650 bytes) 2024-09-14T05:30:21.815 INFO:teuthology.orchestra.run.smithi071.stdout:5895 read 0x5a58300 thru 0x5a59a52 (0x1753 bytes) 2024-09-14T05:30:21.815 INFO:teuthology.orchestra.run.smithi071.stdout:5896 write 0x82e04e9 thru 0x82ed372 (0xce8a bytes) 2024-09-14T05:30:21.819 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.819 INFO:teuthology.orchestra.run.smithi071.stdout:5898 write 0x76a61d6 thru 0x76ae4e5 (0x8310 bytes) 2024-09-14T05:30:21.822 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.822 INFO:teuthology.orchestra.run.smithi071.stdout:5900 read 0x67a7646 thru 0x67a7c54 (0x60f bytes) 2024-09-14T05:30:21.824 INFO:teuthology.orchestra.run.smithi071.stdout:5901 trunc from 0xae2c8db to 0xce0d7f5 2024-09-14T05:30:21.829 INFO:teuthology.orchestra.run.smithi071.stdout:5902 write 0x64671ee thru 0x647033a (0x914d bytes) 2024-09-14T05:30:21.834 INFO:teuthology.orchestra.run.smithi071.stdout:5904 punch from 0x6bb62be to 0x6bbf127, (0x8e69 bytes) 2024-09-14T05:30:21.834 INFO:teuthology.orchestra.run.smithi071.stdout:5905 write 0x49363e5 thru 0x493aa5f (0x467b bytes) 2024-09-14T05:30:21.837 INFO:teuthology.orchestra.run.smithi071.stdout:5906 write 0x74307b6 thru 0x743d5d2 (0xce1d bytes) 2024-09-14T05:30:21.841 INFO:teuthology.orchestra.run.smithi071.stdout:5907 read 0x1066a28 thru 0x107173f (0xad18 bytes) 2024-09-14T05:30:21.842 INFO:teuthology.orchestra.run.smithi071.stdout:5909 writesame 0x91e359f thru 0x91e3ad2 data_size 0x4(0x534 bytes) 2024-09-14T05:30:21.845 INFO:teuthology.orchestra.run.smithi071.stdout:5910 write 0x99a8b83 thru 0x99a9dcd (0x124b bytes) 2024-09-14T05:30:21.848 INFO:teuthology.orchestra.run.smithi071.stdout:5914 read 0xafb945d thru 0xafbe17b (0x4d1f bytes) 2024-09-14T05:30:21.848 INFO:teuthology.orchestra.run.smithi071.stdout:5915 write 0x821b4ae thru 0x8227bc4 (0xc717 bytes) 2024-09-14T05:30:21.855 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.856 INFO:teuthology.orchestra.run.smithi071.stdout:5917 read 0x85e201a thru 0x85ec9fc (0xa9e3 bytes) 2024-09-14T05:30:21.856 INFO:teuthology.orchestra.run.smithi071.stdout:5918 writesame 0x16189d4 thru 0x16193d5 data_size 0x15(0xa02 bytes) 2024-09-14T05:30:21.858 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.860 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:21.860 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:21.860 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:21.860 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:21.860 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:21.864 INFO:teuthology.orchestra.run.smithi071.stdout:5920 write 0xdbeee9a thru 0xdbf9ae5 (0xac4c bytes) 2024-09-14T05:30:21.869 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:30:21.872 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.881 INFO:teuthology.orchestra.run.smithi071.stdout:5922 trunc from 0xdbf9ae6 to 0x57a9319 2024-09-14T05:30:21.894 INFO:teuthology.orchestra.run.smithi071.stdout:5927 punch from 0x40675a0 to 0x406dc6d, (0x66cd bytes) 2024-09-14T05:30:21.906 INFO:teuthology.orchestra.run.smithi071.stdout:5928 writesame 0xb611b86 thru 0xb61e6ff data_size 0x5(0xcb7a bytes) 2024-09-14T05:30:21.906 INFO:teuthology.orchestra.run.smithi071.stdout:5929 read 0x3cf2d68 thru 0x3cfe2f7 (0xb590 bytes) 2024-09-14T05:30:21.908 INFO:teuthology.orchestra.run.smithi071.stdout:5931 trunc from 0xb61e700 to 0xd2da415 2024-09-14T05:30:21.912 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:21.912+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:21.913 INFO:teuthology.orchestra.run.smithi071.stdout:5932 read 0x80f0a3a thru 0x80f488f (0x3e56 bytes) 2024-09-14T05:30:21.913 INFO:teuthology.orchestra.run.smithi071.stdout:5933 punch from 0x86e26c6 to 0x86e899e, (0x62d8 bytes) 2024-09-14T05:30:21.913 INFO:teuthology.orchestra.run.smithi071.stdout:5935 writesame 0x8fd3eb3 thru 0x8fd9fab data_size 0x19(0x60f9 bytes) 2024-09-14T05:30:21.916 INFO:teuthology.orchestra.run.smithi071.stdout:5936 punch from 0x2355102 to 0x235c7ee, (0x76ec bytes) 2024-09-14T05:30:21.917 INFO:teuthology.orchestra.run.smithi071.stdout:5938 writesame 0xa8a0a2 thru 0xa98d73 data_size 0x1(0xecd2 bytes) 2024-09-14T05:30:21.934 INFO:teuthology.orchestra.run.smithi071.stdout:5940 write 0x82a82b thru 0x833ab9 (0x928f bytes) 2024-09-14T05:30:21.936 INFO:teuthology.orchestra.run.smithi071.stdout:5942 read 0xd70757 thru 0xd77f42 (0x77ec bytes) 2024-09-14T05:30:21.937 INFO:teuthology.orchestra.run.smithi071.stdout:5944 punch from 0x2f476b2 to 0x2f5219c, (0xaaea bytes) 2024-09-14T05:30:21.937 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.937 INFO:teuthology.orchestra.run.smithi071.stdout:5946 punch from 0x7ccceb6 to 0x7cdac4b, (0xdd95 bytes) 2024-09-14T05:30:21.937 INFO:teuthology.orchestra.run.smithi071.stdout:5948 read 0x895c785 thru 0x8964a6c (0x82e8 bytes) 2024-09-14T05:30:21.937 INFO:teuthology.orchestra.run.smithi071.stdout:5950 writesame 0xca63c2c thru 0xca6c4f4 data_size 0x13(0x88c9 bytes) 2024-09-14T05:30:21.940 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.940 INFO:teuthology.orchestra.run.smithi071.stdout:5957 writesame 0x4909bff thru 0x491946c data_size 0x1(0xf86e bytes) 2024-09-14T05:30:21.958 INFO:teuthology.orchestra.run.smithi071.stdout:5958 writesame 0xebed4 thru 0xf1313 data_size 0x20(0x5440 bytes) 2024-09-14T05:30:21.960 INFO:teuthology.orchestra.run.smithi071.stdout:5959 write 0xbeffb82 thru 0xbf08b45 (0x8fc4 bytes) 2024-09-14T05:30:21.965 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.965 INFO:teuthology.orchestra.run.smithi071.stdout:5963 write 0x2d82397 thru 0x2d82c38 (0x8a2 bytes) 2024-09-14T05:30:21.968 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:21.969 INFO:teuthology.orchestra.run.smithi071.stdout:5965 punch from 0xbb80ceb to 0xbb84151, (0x3466 bytes) 2024-09-14T05:30:21.969 INFO:teuthology.orchestra.run.smithi071.stdout:5966 read 0x39eab18 thru 0x39f7d60 (0xd249 bytes) 2024-09-14T05:30:21.978 INFO:teuthology.orchestra.run.smithi071.stdout:5967 trunc from 0xd2da415 to 0x57c8e5c 2024-09-14T05:30:21.980 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:21.980 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:21.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:21.980 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:21.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.980 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1989 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 16326080~54496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:21.980 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:21.980 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.417056033999998, 2024-09-14T05:30:21.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096664322999999996, 2024-09-14T05:30:21.980 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.981 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.981 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:21.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.981 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1989 2024-09-14T05:30:21.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.981 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:21.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:21.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561455+0000", 2024-09-14T05:30:21.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1699999999999997e-07 2024-09-14T05:30:21.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590703+0000", 2024-09-14T05:30:21.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029248225999999999 2024-09-14T05:30:21.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590732+0000", 2024-09-14T05:30:21.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8679000000000001e-05 2024-09-14T05:30:21.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590759+0000", 2024-09-14T05:30:21.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7319999999999999e-05 2024-09-14T05:30:21.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590835+0000", 2024-09-14T05:30:21.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5543999999999994e-05 2024-09-14T05:30:21.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590956+0000", 2024-09-14T05:30:21.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012168 2024-09-14T05:30:21.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 1", 2024-09-14T05:30:21.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.591261+0000", 2024-09-14T05:30:21.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030484799999999998 2024-09-14T05:30:21.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:21.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.637682+0000", 2024-09-14T05:30:21.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046420550999999997 2024-09-14T05:30:21.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:21.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.657912+0000", 2024-09-14T05:30:21.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020230011999999999 2024-09-14T05:30:21.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:21.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658007+0000", 2024-09-14T05:30:21.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5814000000000001e-05 2024-09-14T05:30:21.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658118+0000", 2024-09-14T05:30:21.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110832 2024-09-14T05:30:21.986 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.986 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.986 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.986 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2033 3.1e 3:7e47f6eb:::rbd_data.112d1f36d097.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30853661~63531] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:21.986 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:21.986 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.305781367999998, 2024-09-14T05:30:21.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065167630000000004, 2024-09-14T05:30:21.986 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.986 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.986 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:21.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.987 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.987 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.987 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2033 2024-09-14T05:30:21.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.987 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:21.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:21.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672730+0000", 2024-09-14T05:30:21.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.393e-06 2024-09-14T05:30:21.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691289+0000", 2024-09-14T05:30:21.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018558991 2024-09-14T05:30:21.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691316+0000", 2024-09-14T05:30:21.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7498999999999999e-05 2024-09-14T05:30:21.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691331+0000", 2024-09-14T05:30:21.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4652e-05 2024-09-14T05:30:21.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691397+0000", 2024-09-14T05:30:21.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6112000000000007e-05 2024-09-14T05:30:21.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691509+0000", 2024-09-14T05:30:21.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111599 2024-09-14T05:30:21.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:21.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.692253+0000", 2024-09-14T05:30:21.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074456200000000004 2024-09-14T05:30:21.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:21.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.719140+0000", 2024-09-14T05:30:21.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026886542999999999 2024-09-14T05:30:21.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:21.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737789+0000", 2024-09-14T05:30:21.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018649203999999999 2024-09-14T05:30:21.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:21.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737846+0000", 2024-09-14T05:30:21.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6501e-05 2024-09-14T05:30:21.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737896+0000", 2024-09-14T05:30:21.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0574000000000001e-05 2024-09-14T05:30:21.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.992 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.992 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3008 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13729792~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.992 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:21.993 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.674857498000001, 2024-09-14T05:30:21.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061007651000000003, 2024-09-14T05:30:21.993 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.993 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.993 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.993 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3008 2024-09-14T05:30:21.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.993 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:21.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:21.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303653+0000", 2024-09-14T05:30:21.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0699999999999999e-07 2024-09-14T05:30:21.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303654+0000", 2024-09-14T05:30:21.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-07 2024-09-14T05:30:21.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303657+0000", 2024-09-14T05:30:21.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2890000000000002e-06 2024-09-14T05:30:21.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:21.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303661+0000", 2024-09-14T05:30:21.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3290000000000004e-06 2024-09-14T05:30:21.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:21.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364233+0000", 2024-09-14T05:30:21.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060572134 2024-09-14T05:30:21.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:21.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364315+0000", 2024-09-14T05:30:21.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1549000000000001e-05 2024-09-14T05:30:21.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:21.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364660+0000", 2024-09-14T05:30:21.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034525300000000001 2024-09-14T05:30:21.996 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.997 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:21.997 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:21.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.997 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3009 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14221312~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:21.997 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:21.997 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.674843408000001, 2024-09-14T05:30:21.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061445063000000001, 2024-09-14T05:30:21.997 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:21.997 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:21.997 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:21.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:21.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:21.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:21.997 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3009 2024-09-14T05:30:21.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.998 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:21.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:21.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:21.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:21.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:21.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:21.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:21.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:21.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:21.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:21.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303669+0000", 2024-09-14T05:30:21.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7150000000000001e-06 2024-09-14T05:30:21.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:21.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:21.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303675+0000", 2024-09-14T05:30:21.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5339999999999996e-06 2024-09-14T05:30:21.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:21.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303681+0000", 2024-09-14T05:30:22.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3639999999999998e-06 2024-09-14T05:30:22.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364705+0000", 2024-09-14T05:30:22.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061024123 2024-09-14T05:30:22.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364786+0000", 2024-09-14T05:30:22.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1378999999999995e-05 2024-09-14T05:30:22.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365112+0000", 2024-09-14T05:30:22.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032558700000000001 2024-09-14T05:30:22.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.001 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.001 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3010 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14712832~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.001 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:22.001 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.674822544000001, 2024-09-14T05:30:22.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061834630000000002, 2024-09-14T05:30:22.001 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.002 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.002 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.002 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3010 2024-09-14T05:30:22.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.002 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:22.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:22.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303688+0000", 2024-09-14T05:30:22.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:30:22.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303690+0000", 2024-09-14T05:30:22.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9460000000000001e-06 2024-09-14T05:30:22.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303696+0000", 2024-09-14T05:30:22.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2230000000000004e-06 2024-09-14T05:30:22.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303700+0000", 2024-09-14T05:30:22.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7290000000000002e-06 2024-09-14T05:30:22.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365163+0000", 2024-09-14T05:30:22.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061463093000000003 2024-09-14T05:30:22.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365214+0000", 2024-09-14T05:30:22.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0977999999999999e-05 2024-09-14T05:30:22.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365522+0000", 2024-09-14T05:30:22.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000308227 2024-09-14T05:30:22.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.006 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.006 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3241 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.006 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:22.006 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.649148122, 2024-09-14T05:30:22.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061041892, 2024-09-14T05:30:22.006 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.006 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.006 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.007 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3241 2024-09-14T05:30:22.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.007 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:22.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:22.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329496+0000", 2024-09-14T05:30:22.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013421799999999999 2024-09-14T05:30:22.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329499+0000", 2024-09-14T05:30:22.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8459999999999999e-06 2024-09-14T05:30:22.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329505+0000", 2024-09-14T05:30:22.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4320000000000004e-06 2024-09-14T05:30:22.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329509+0000", 2024-09-14T05:30:22.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0339999999999998e-06 2024-09-14T05:30:22.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.389947+0000", 2024-09-14T05:30:22.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060438093999999998 2024-09-14T05:30:22.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390038+0000", 2024-09-14T05:30:22.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0253999999999993e-05 2024-09-14T05:30:22.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390404+0000", 2024-09-14T05:30:22.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036601400000000001 2024-09-14T05:30:22.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.010 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.010 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3242 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.010 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:22.010 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.648993214000001, 2024-09-14T05:30:22.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061282426000000001, 2024-09-14T05:30:22.011 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.011 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.011 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.011 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3242 2024-09-14T05:30:22.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.011 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:22.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:22.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:22.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9900000000000001e-07 2024-09-14T05:30:22.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329518+0000", 2024-09-14T05:30:22.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.328e-06 2024-09-14T05:30:22.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329522+0000", 2024-09-14T05:30:22.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.563e-06 2024-09-14T05:30:22.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329525+0000", 2024-09-14T05:30:22.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7379999999999999e-06 2024-09-14T05:30:22.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390437+0000", 2024-09-14T05:30:22.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060912463 2024-09-14T05:30:22.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390498+0000", 2024-09-14T05:30:22.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0615999999999997e-05 2024-09-14T05:30:22.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390799+0000", 2024-09-14T05:30:22.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030131899999999998 2024-09-14T05:30:22.014 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.014 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.014 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.015 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3243 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.015 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:22.015 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.648981427999999, 2024-09-14T05:30:22.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061728980000000003, 2024-09-14T05:30:22.015 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.015 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.015 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.015 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3243 2024-09-14T05:30:22.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.015 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:22.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:22.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329692+0000", 2024-09-14T05:30:22.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000163948 2024-09-14T05:30:22.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329696+0000", 2024-09-14T05:30:22.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5540000000000002e-06 2024-09-14T05:30:22.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329706+0000", 2024-09-14T05:30:22.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7960000000000007e-06 2024-09-14T05:30:22.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329712+0000", 2024-09-14T05:30:22.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6109999999999999e-06 2024-09-14T05:30:22.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390861+0000", 2024-09-14T05:30:22.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061148767 2024-09-14T05:30:22.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390942+0000", 2024-09-14T05:30:22.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1266000000000004e-05 2024-09-14T05:30:22.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391258+0000", 2024-09-14T05:30:22.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031503800000000002 2024-09-14T05:30:22.019 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.019 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.019 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.019 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3244 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14778368~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.019 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:22.019 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.648786379999997, 2024-09-14T05:30:22.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062050594000000001, 2024-09-14T05:30:22.019 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.019 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.019 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.020 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.020 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.020 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3244 2024-09-14T05:30:22.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.020 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:22.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:22.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329916+0000", 2024-09-14T05:30:22.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019290799999999999 2024-09-14T05:30:22.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329919+0000", 2024-09-14T05:30:22.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9019999999999999e-06 2024-09-14T05:30:22.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329926+0000", 2024-09-14T05:30:22.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9e-06 2024-09-14T05:30:22.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329932+0000", 2024-09-14T05:30:22.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.088e-06 2024-09-14T05:30:22.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391299+0000", 2024-09-14T05:30:22.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061366136000000002 2024-09-14T05:30:22.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391391+0000", 2024-09-14T05:30:22.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2204e-05 2024-09-14T05:30:22.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391774+0000", 2024-09-14T05:30:22.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038345600000000001 2024-09-14T05:30:22.023 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.023 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.023 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.023 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3921 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 5308416~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.023 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:22.023 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.577242570000003, 2024-09-14T05:30:22.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060842694000000003, 2024-09-14T05:30:22.024 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.024 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.024 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.024 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.024 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.024 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.024 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3921 2024-09-14T05:30:22.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.024 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:22.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:22.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401705+0000", 2024-09-14T05:30:22.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768499999999999 2024-09-14T05:30:22.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401709+0000", 2024-09-14T05:30:22.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0319999999999997e-06 2024-09-14T05:30:22.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401720+0000", 2024-09-14T05:30:22.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1005000000000001e-05 2024-09-14T05:30:22.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401727+0000", 2024-09-14T05:30:22.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1879999999999996e-06 2024-09-14T05:30:22.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461748+0000", 2024-09-14T05:30:22.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060021199999999997 2024-09-14T05:30:22.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461793+0000", 2024-09-14T05:30:22.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4530999999999998e-05 2024-09-14T05:30:22.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462110+0000", 2024-09-14T05:30:22.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031705299999999997 2024-09-14T05:30:22.027 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.027 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.027 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.028 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3923 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 6291456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.028 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:22.028 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.576596541999997, 2024-09-14T05:30:22.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060891273000000003, 2024-09-14T05:30:22.028 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.028 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.028 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.028 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.028 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.028 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.028 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3923 2024-09-14T05:30:22.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.029 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:22.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:22.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407054+0000", 2024-09-14T05:30:22.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005140787 2024-09-14T05:30:22.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407059+0000", 2024-09-14T05:30:22.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.04e-06 2024-09-14T05:30:22.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407069+0000", 2024-09-14T05:30:22.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4429999999999992e-06 2024-09-14T05:30:22.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407076+0000", 2024-09-14T05:30:22.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1679999999999998e-06 2024-09-14T05:30:22.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462514+0000", 2024-09-14T05:30:22.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055438551000000003 2024-09-14T05:30:22.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462562+0000", 2024-09-14T05:30:22.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7649999999999999e-05 2024-09-14T05:30:22.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462805+0000", 2024-09-14T05:30:22.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024263400000000001 2024-09-14T05:30:22.032 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.032 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.032 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.032 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4052 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 12550144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.032 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:22.032 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.561024533000001, 2024-09-14T05:30:22.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060872904999999998, 2024-09-14T05:30:22.032 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.033 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.033 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.033 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.033 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.033 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.033 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4052 2024-09-14T05:30:22.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.033 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:22.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:22.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417486+0000", 2024-09-14T05:30:22.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-14T05:30:22.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417487+0000", 2024-09-14T05:30:22.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4499999999999995e-07 2024-09-14T05:30:22.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417489+0000", 2024-09-14T05:30:22.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8899999999999999e-06 2024-09-14T05:30:22.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417492+0000", 2024-09-14T05:30:22.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4389999999999999e-06 2024-09-14T05:30:22.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478027+0000", 2024-09-14T05:30:22.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060534959999999999 2024-09-14T05:30:22.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478079+0000", 2024-09-14T05:30:22.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2104999999999997e-05 2024-09-14T05:30:22.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478358+0000", 2024-09-14T05:30:22.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000279351 2024-09-14T05:30:22.036 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.036 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.036 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.037 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4053 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13041664~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.037 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:22.037 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.561015148999999, 2024-09-14T05:30:22.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061258866000000002, 2024-09-14T05:30:22.037 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.037 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.037 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.037 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.037 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.037 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.037 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4053 2024-09-14T05:30:22.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.037 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:22.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:22.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:22.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.17e-07 2024-09-14T05:30:22.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417496+0000", 2024-09-14T05:30:22.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0890000000000001e-06 2024-09-14T05:30:22.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417499+0000", 2024-09-14T05:30:22.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8629999999999999e-06 2024-09-14T05:30:22.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417502+0000", 2024-09-14T05:30:22.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6290000000000002e-06 2024-09-14T05:30:22.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478387+0000", 2024-09-14T05:30:22.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060885433000000003 2024-09-14T05:30:22.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478435+0000", 2024-09-14T05:30:22.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8167999999999997e-05 2024-09-14T05:30:22.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478754+0000", 2024-09-14T05:30:22.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031846700000000001 2024-09-14T05:30:22.041 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.041 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.041 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.041 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4054 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13533184~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.041 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:22.042 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.561005264999999, 2024-09-14T05:30:22.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061604694000000002, 2024-09-14T05:30:22.042 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.042 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.042 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.042 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.042 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.042 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.042 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4054 2024-09-14T05:30:22.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.042 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:22.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:22.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:22.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.04e-07 2024-09-14T05:30:22.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417506+0000", 2024-09-14T05:30:22.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8299999999999995e-07 2024-09-14T05:30:22.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417509+0000", 2024-09-14T05:30:22.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.88e-06 2024-09-14T05:30:22.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417511+0000", 2024-09-14T05:30:22.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5950000000000001e-06 2024-09-14T05:30:22.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478784+0000", 2024-09-14T05:30:22.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061272971000000002 2024-09-14T05:30:22.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478841+0000", 2024-09-14T05:30:22.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-14T05:30:22.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479109+0000", 2024-09-14T05:30:22.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026794600000000003 2024-09-14T05:30:22.046 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.046 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.046 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.046 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4055 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14024704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.046 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:22.046 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.560995562999999, 2024-09-14T05:30:22.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061932299000000003, 2024-09-14T05:30:22.046 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.046 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.046 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.047 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.047 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.047 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.047 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4055 2024-09-14T05:30:22.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.047 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:22.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:22.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417515+0000", 2024-09-14T05:30:22.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-14T05:30:22.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417516+0000", 2024-09-14T05:30:22.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7199999999999997e-07 2024-09-14T05:30:22.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417519+0000", 2024-09-14T05:30:22.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.937e-06 2024-09-14T05:30:22.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417522+0000", 2024-09-14T05:30:22.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1420000000000002e-06 2024-09-14T05:30:22.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479140+0000", 2024-09-14T05:30:22.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061617933999999999 2024-09-14T05:30:22.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479188+0000", 2024-09-14T05:30:22.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8717000000000001e-05 2024-09-14T05:30:22.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479447+0000", 2024-09-14T05:30:22.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025838800000000002 2024-09-14T05:30:22.050 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.050 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.050 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.051 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4056 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14516224~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.051 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:22.051 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.560985389999999, 2024-09-14T05:30:22.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062263185999999998, 2024-09-14T05:30:22.051 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.051 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.051 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.051 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.051 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.051 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.051 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4056 2024-09-14T05:30:22.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.051 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:22.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:22.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:22.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8700000000000002e-07 2024-09-14T05:30:22.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417528+0000", 2024-09-14T05:30:22.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.579e-06 2024-09-14T05:30:22.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417533+0000", 2024-09-14T05:30:22.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2400000000000001e-06 2024-09-14T05:30:22.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417536+0000", 2024-09-14T05:30:22.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0309999999999999e-06 2024-09-14T05:30:22.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479477+0000", 2024-09-14T05:30:22.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061940967 2024-09-14T05:30:22.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479523+0000", 2024-09-14T05:30:22.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6335000000000003e-05 2024-09-14T05:30:22.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479788+0000", 2024-09-14T05:30:22.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026474700000000002 2024-09-14T05:30:22.055 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.055 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.055 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.055 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4057 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15007744~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.055 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:22.055 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.560970929, 2024-09-14T05:30:22.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062589928000000003, 2024-09-14T05:30:22.055 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.055 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.056 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.056 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.056 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.056 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.056 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4057 2024-09-14T05:30:22.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.056 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:22.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:22.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422838+0000", 2024-09-14T05:30:22.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052990349999999997 2024-09-14T05:30:22.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422854+0000", 2024-09-14T05:30:22.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5523999999999999e-05 2024-09-14T05:30:22.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422864+0000", 2024-09-14T05:30:22.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0084e-05 2024-09-14T05:30:22.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422872+0000", 2024-09-14T05:30:22.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-14T05:30:22.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479813+0000", 2024-09-14T05:30:22.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056941389000000002 2024-09-14T05:30:22.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479868+0000", 2024-09-14T05:30:22.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4973999999999999e-05 2024-09-14T05:30:22.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.480129+0000", 2024-09-14T05:30:22.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026084200000000001 2024-09-14T05:30:22.060 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.060 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.060 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.061 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4171 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14057472~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.061 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:22.061 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.548973369000002, 2024-09-14T05:30:22.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061184929999999998, 2024-09-14T05:30:22.061 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.061 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.061 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.061 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.061 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.061 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.061 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4171 2024-09-14T05:30:22.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.061 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:22.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:22.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:22.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.39e-07 2024-09-14T05:30:22.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429541+0000", 2024-09-14T05:30:22.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:30:22.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429547+0000", 2024-09-14T05:30:22.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3280000000000002e-06 2024-09-14T05:30:22.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429552+0000", 2024-09-14T05:30:22.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4859999999999997e-06 2024-09-14T05:30:22.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490379+0000", 2024-09-14T05:30:22.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060826681 2024-09-14T05:30:22.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490434+0000", 2024-09-14T05:30:22.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4577999999999998e-05 2024-09-14T05:30:22.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490722+0000", 2024-09-14T05:30:22.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028781400000000001 2024-09-14T05:30:22.065 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.065 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.065 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.065 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4172 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14548992~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.066 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:22.066 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.548951361, 2024-09-14T05:30:22.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061498256000000001, 2024-09-14T05:30:22.066 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.066 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.066 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.066 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.066 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.066 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.066 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4172 2024-09-14T05:30:22.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.066 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:22.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:22.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:22.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.72e-07 2024-09-14T05:30:22.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429561+0000", 2024-09-14T05:30:22.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.717e-06 2024-09-14T05:30:22.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429566+0000", 2024-09-14T05:30:22.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6529999999999999e-06 2024-09-14T05:30:22.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429571+0000", 2024-09-14T05:30:22.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5979999999999999e-06 2024-09-14T05:30:22.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490751+0000", 2024-09-14T05:30:22.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061180336000000002 2024-09-14T05:30:22.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490798+0000", 2024-09-14T05:30:22.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6875000000000001e-05 2024-09-14T05:30:22.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491057+0000", 2024-09-14T05:30:22.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025870500000000002 2024-09-14T05:30:22.070 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.070 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.070 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.070 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4173 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15040512~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.070 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:22.070 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.548933734999999, 2024-09-14T05:30:22.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061894820000000003, 2024-09-14T05:30:22.070 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.070 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.070 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.070 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.070 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.070 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.071 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4173 2024-09-14T05:30:22.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.071 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:22.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:22.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429577+0000", 2024-09-14T05:30:22.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-14T05:30:22.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429578+0000", 2024-09-14T05:30:22.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6980000000000001e-06 2024-09-14T05:30:22.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429585+0000", 2024-09-14T05:30:22.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.725e-06 2024-09-14T05:30:22.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429590+0000", 2024-09-14T05:30:22.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.938e-06 2024-09-14T05:30:22.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491091+0000", 2024-09-14T05:30:22.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061500773000000002 2024-09-14T05:30:22.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491136+0000", 2024-09-14T05:30:22.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5284000000000002e-05 2024-09-14T05:30:22.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491471+0000", 2024-09-14T05:30:22.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033485699999999998 2024-09-14T05:30:22.074 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.074 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.074 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.074 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.074 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.074 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:22.076 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:30:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:5969 write 0x68a7aea thru 0x68a9b6a (0x2081 bytes) 2024-09-14T05:30:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:5971 trunc from 0x68a9b6b to 0x4354c1c 2024-09-14T05:30:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:5974 write 0x776f951 thru 0x7778b01 (0x91b1 bytes) 2024-09-14T05:30:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:5975 read 0x743940e thru 0x7446a71 (0xd664 bytes) 2024-09-14T05:30:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:5979 writesame 0x48d3d29 thru 0x48e3268 data_size 0xc0(0xf540 bytes) 2024-09-14T05:30:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:5980 punch from 0x696e8d7 to 0x697b2bb, (0xc9e4 bytes) 2024-09-14T05:30:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:5981 writesame 0x5fa2433 thru 0x5fa608b data_size 0x1(0x3c59 bytes) 2024-09-14T05:30:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:5982 write 0x5144878 thru 0x5150cf4 (0xc47d bytes) 2024-09-14T05:30:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:5983 punch from 0x1c04153 to 0x1c10c36, (0xcae3 bytes) 2024-09-14T05:30:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:5984 write 0xbe80f1b thru 0xbe8d9a8 (0xca8e bytes) 2024-09-14T05:30:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:5985 write 0x3e0dfaf thru 0x3e0fd73 (0x1dc5 bytes) 2024-09-14T05:30:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:5986 read 0x902f121 thru 0x90322c7 (0x31a7 bytes) 2024-09-14T05:30:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:5987 write 0xdd27514 thru 0xdd35b42 (0xe62f bytes) 2024-09-14T05:30:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:5988 writesame 0xaf5c9a2 thru 0xaf63046 data_size 0x13(0x66a5 bytes) 2024-09-14T05:30:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:compare and write disabled 2024-09-14T05:30:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:5996 punch from 0x2df7551 to 0x2dfa549, (0x2ff8 bytes) 2024-09-14T05:30:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:5998 read 0x3027211 thru 0x3036c69 (0xfa59 bytes) 2024-09-14T05:30:22.079 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:22.075+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:22.080 INFO:teuthology.orchestra.run.smithi071.stdout:5999 punch from 0x2634d74 to 0x263bb7c, (0x6e08 bytes) 2024-09-14T05:30:22.080 INFO:teuthology.orchestra.run.smithi071.stdout:6000 read 0x934f3d thru 0x9374fa (0x25be bytes) 2024-09-14T05:30:22.103 INFO:teuthology.orchestra.run.smithi071.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-14T05:30:22.176 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:22.175+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:22.224 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:22.224 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:22.225 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:22.226 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:22.233 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:30:22.277 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:22.276+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:22.340 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T05:30:22.377 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:22.376+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:22.386 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:22.386 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:22.386 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:22.386 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:22.387 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:22.395 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:30:22.479 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:22.479+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:22.543 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:22.543 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:22.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:22.544 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:22.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.544 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1205 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:22.544 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:22.544 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.343830122999996, 2024-09-14T05:30:22.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0086364279999999998, 2024-09-14T05:30:22.544 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.544 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.544 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.544 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.544 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.544 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.544 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1205 2024-09-14T05:30:22.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.545 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:22.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:22.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197891+0000", 2024-09-14T05:30:22.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.364e-06 2024-09-14T05:30:22.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197896+0000", 2024-09-14T05:30:22.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3000000000000001e-06 2024-09-14T05:30:22.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197910+0000", 2024-09-14T05:30:22.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3339000000000001e-05 2024-09-14T05:30:22.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197919+0000", 2024-09-14T05:30:22.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8869999999999998e-06 2024-09-14T05:30:22.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197974+0000", 2024-09-14T05:30:22.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5720000000000002e-05 2024-09-14T05:30:22.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.198094+0000", 2024-09-14T05:30:22.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119739 2024-09-14T05:30:22.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206526+0000", 2024-09-14T05:30:22.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084320790000000003 2024-09-14T05:30:22.549 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.549 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.549 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.549 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1220 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:22.549 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:22.549 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.335073758999997, 2024-09-14T05:30:22.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012822337, 2024-09-14T05:30:22.550 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.550 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.550 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:22.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.550 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1220 2024-09-14T05:30:22.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.550 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:22.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:22.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206647+0000", 2024-09-14T05:30:22.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4600000000000003e-07 2024-09-14T05:30:22.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206652+0000", 2024-09-14T05:30:22.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6619999999999996e-06 2024-09-14T05:30:22.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206662+0000", 2024-09-14T05:30:22.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.995e-06 2024-09-14T05:30:22.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206667+0000", 2024-09-14T05:30:22.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-14T05:30:22.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206706+0000", 2024-09-14T05:30:22.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9523000000000001e-05 2024-09-14T05:30:22.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206755+0000", 2024-09-14T05:30:22.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000002e-05 2024-09-14T05:30:22.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:22.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206877+0000", 2024-09-14T05:30:22.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121938 2024-09-14T05:30:22.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:22.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.207310+0000", 2024-09-14T05:30:22.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000432452 2024-09-14T05:30:22.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:22.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219364+0000", 2024-09-14T05:30:22.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012054481000000001 2024-09-14T05:30:22.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:22.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219425+0000", 2024-09-14T05:30:22.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0618000000000001e-05 2024-09-14T05:30:22.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219468+0000", 2024-09-14T05:30:22.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3739000000000003e-05 2024-09-14T05:30:22.555 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.556 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.556 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.557 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1243 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:22.557 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:22.557 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.322108532000001, 2024-09-14T05:30:22.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078660580000000004, 2024-09-14T05:30:22.557 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.558 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.558 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.558 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.558 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.558 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.558 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1243 2024-09-14T05:30:22.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.558 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:22.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:22.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219612+0000", 2024-09-14T05:30:22.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.09e-07 2024-09-14T05:30:22.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219616+0000", 2024-09-14T05:30:22.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.216e-06 2024-09-14T05:30:22.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219630+0000", 2024-09-14T05:30:22.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3064999999999999e-05 2024-09-14T05:30:22.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219637+0000", 2024-09-14T05:30:22.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7409999999999992e-06 2024-09-14T05:30:22.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219697+0000", 2024-09-14T05:30:22.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.02e-05 2024-09-14T05:30:22.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219833+0000", 2024-09-14T05:30:22.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013595600000000001 2024-09-14T05:30:22.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227477+0000", 2024-09-14T05:30:22.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076439710000000003 2024-09-14T05:30:22.562 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.562 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.562 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.562 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1264 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:22.562 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:22.562 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.314097738999997, 2024-09-14T05:30:22.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080867990000000004, 2024-09-14T05:30:22.562 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.562 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.563 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:22.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.563 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1264 2024-09-14T05:30:22.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.563 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:22.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:22.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227623+0000", 2024-09-14T05:30:22.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:30:22.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227627+0000", 2024-09-14T05:30:22.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.194e-06 2024-09-14T05:30:22.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227639+0000", 2024-09-14T05:30:22.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1296e-05 2024-09-14T05:30:22.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227648+0000", 2024-09-14T05:30:22.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2280000000000004e-06 2024-09-14T05:30:22.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227689+0000", 2024-09-14T05:30:22.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1329999999999999e-05 2024-09-14T05:30:22.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227741+0000", 2024-09-14T05:30:22.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1641000000000002e-05 2024-09-14T05:30:22.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:30:22.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227914+0000", 2024-09-14T05:30:22.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000173215 2024-09-14T05:30:22.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:22.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228474+0000", 2024-09-14T05:30:22.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055980199999999998 2024-09-14T05:30:22.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:22.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235608+0000", 2024-09-14T05:30:22.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0071343830000000002 2024-09-14T05:30:22.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:22.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235669+0000", 2024-09-14T05:30:22.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1068999999999994e-05 2024-09-14T05:30:22.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235709+0000", 2024-09-14T05:30:22.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9671e-05 2024-09-14T05:30:22.568 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.568 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.569 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.569 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1520 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:22.569 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:30:22.569 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.219090119999997, 2024-09-14T05:30:22.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061963629999999999, 2024-09-14T05:30:22.569 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.569 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.569 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:22.569 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.569 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.570 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.570 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1520 2024-09-14T05:30:22.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.570 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:30:22.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:30:22.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322631+0000", 2024-09-14T05:30:22.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.54e-07 2024-09-14T05:30:22.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322635+0000", 2024-09-14T05:30:22.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0450000000000002e-06 2024-09-14T05:30:22.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322646+0000", 2024-09-14T05:30:22.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1919999999999999e-05 2024-09-14T05:30:22.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322654+0000", 2024-09-14T05:30:22.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3139999999999998e-06 2024-09-14T05:30:22.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322704+0000", 2024-09-14T05:30:22.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0420000000000002e-05 2024-09-14T05:30:22.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322794+0000", 2024-09-14T05:30:22.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0037000000000005e-05 2024-09-14T05:30:22.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328826+0000", 2024-09-14T05:30:22.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060318730000000001 2024-09-14T05:30:22.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.574 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.575 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1526 3.4 3:2695cbe4:::rbd_data.112dd25fb4c.0000000000000119:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:22.575 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:22.575 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.215557175999997, 2024-09-14T05:30:22.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0079537359999999994, 2024-09-14T05:30:22.575 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.575 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.575 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:22.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.575 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1526 2024-09-14T05:30:22.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.576 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:22.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:22.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:22.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3600000000000003e-07 2024-09-14T05:30:22.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326168+0000", 2024-09-14T05:30:22.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3460000000000004e-06 2024-09-14T05:30:22.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326178+0000", 2024-09-14T05:30:22.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0291e-05 2024-09-14T05:30:22.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326183+0000", 2024-09-14T05:30:22.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-14T05:30:22.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326230+0000", 2024-09-14T05:30:22.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6650999999999998e-05 2024-09-14T05:30:22.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326295+0000", 2024-09-14T05:30:22.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5158000000000002e-05 2024-09-14T05:30:22.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:22.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326408+0000", 2024-09-14T05:30:22.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113039 2024-09-14T05:30:22.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:22.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.331648+0000", 2024-09-14T05:30:22.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052404929999999997 2024-09-14T05:30:22.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:22.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334011+0000", 2024-09-14T05:30:22.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002362669 2024-09-14T05:30:22.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:22.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334064+0000", 2024-09-14T05:30:22.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3198999999999997e-05 2024-09-14T05:30:22.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334116+0000", 2024-09-14T05:30:22.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2133999999999999e-05 2024-09-14T05:30:22.582 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.582 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.582 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.582 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1529 3.1 3:809746f7:::rbd_data.112dd25fb4c.0000000000000118:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:22.582 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:30:22.582 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.213792626, 2024-09-14T05:30:22.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061981060000000001, 2024-09-14T05:30:22.583 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.583 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.583 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:22.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.583 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1529 2024-09-14T05:30:22.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.583 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:30:22.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:30:22.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327928+0000", 2024-09-14T05:30:22.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0100000000000004e-07 2024-09-14T05:30:22.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327933+0000", 2024-09-14T05:30:22.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4689999999999999e-06 2024-09-14T05:30:22.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327944+0000", 2024-09-14T05:30:22.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1623999999999999e-05 2024-09-14T05:30:22.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327951+0000", 2024-09-14T05:30:22.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9759999999999998e-06 2024-09-14T05:30:22.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328003+0000", 2024-09-14T05:30:22.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1521999999999999e-05 2024-09-14T05:30:22.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328087+0000", 2024-09-14T05:30:22.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3988999999999998e-05 2024-09-14T05:30:22.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:22.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328304+0000", 2024-09-14T05:30:22.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000217254 2024-09-14T05:30:22.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:22.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.333363+0000", 2024-09-14T05:30:22.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050586499999999996 2024-09-14T05:30:22.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:22.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334026+0000", 2024-09-14T05:30:22.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066303499999999997 2024-09-14T05:30:22.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:22.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334080+0000", 2024-09-14T05:30:22.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4261000000000001e-05 2024-09-14T05:30:22.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334125+0000", 2024-09-14T05:30:22.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5525000000000002e-05 2024-09-14T05:30:22.589 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.589 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.589 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.589 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1531 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:22.589 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:22.589 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.21273901, 2024-09-14T05:30:22.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082114309999999999, 2024-09-14T05:30:22.590 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.590 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.590 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:22.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.590 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1531 2024-09-14T05:30:22.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.590 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:22.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:22.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328982+0000", 2024-09-14T05:30:22.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:30:22.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328985+0000", 2024-09-14T05:30:22.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7900000000000001e-06 2024-09-14T05:30:22.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328996+0000", 2024-09-14T05:30:22.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.028e-05 2024-09-14T05:30:22.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329001+0000", 2024-09-14T05:30:22.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0610000000000003e-06 2024-09-14T05:30:22.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329035+0000", 2024-09-14T05:30:22.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4366999999999998e-05 2024-09-14T05:30:22.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329082+0000", 2024-09-14T05:30:22.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6646e-05 2024-09-14T05:30:22.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:22.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329195+0000", 2024-09-14T05:30:22.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113279 2024-09-14T05:30:22.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:22.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334293+0000", 2024-09-14T05:30:22.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050980399999999999 2024-09-14T05:30:22.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:22.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337101+0000", 2024-09-14T05:30:22.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0028076170000000001 2024-09-14T05:30:22.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:22.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337154+0000", 2024-09-14T05:30:22.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3641999999999999e-05 2024-09-14T05:30:22.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337192+0000", 2024-09-14T05:30:22.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7932000000000002e-05 2024-09-14T05:30:22.595 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.595 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.595 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.595 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:2131 3.b e30/24 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head v 30'36, mlcod=30'34)", 2024-09-14T05:30:22.596 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:22.596 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.749391654999997, 2024-09-14T05:30:22.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031871709999999998, 2024-09-14T05:30:22.596 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.596 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.596 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:22.596 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:22.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:22.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792331+0000", 2024-09-14T05:30:22.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6900000000000001e-06 2024-09-14T05:30:22.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797319+0000", 2024-09-14T05:30:22.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049876349999999998 2024-09-14T05:30:22.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797349+0000", 2024-09-14T05:30:22.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0046e-05 2024-09-14T05:30:22.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797359+0000", 2024-09-14T05:30:22.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0037999999999999e-05 2024-09-14T05:30:22.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797445+0000", 2024-09-14T05:30:22.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6132999999999997e-05 2024-09-14T05:30:22.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797472+0000", 2024-09-14T05:30:22.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7478999999999999e-05 2024-09-14T05:30:22.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:22.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824161+0000", 2024-09-14T05:30:22.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026688714999999998 2024-09-14T05:30:22.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824200+0000", 2024-09-14T05:30:22.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8974000000000003e-05 2024-09-14T05:30:22.600 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.600 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.600 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.600 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4259 3.4 3:25f21dc9:::rbd_data.112d3d611e54.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 294912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.600 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:22.600 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.150219041, 2024-09-14T05:30:22.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010043553, 2024-09-14T05:30:22.600 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.600 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.600 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:22.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.601 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4259 2024-09-14T05:30:22.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.601 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:22.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:22.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391503+0000", 2024-09-14T05:30:22.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9099999999999999e-06 2024-09-14T05:30:22.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391821+0000", 2024-09-14T05:30:22.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031871800000000001 2024-09-14T05:30:22.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391844+0000", 2024-09-14T05:30:22.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2045e-05 2024-09-14T05:30:22.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391853+0000", 2024-09-14T05:30:22.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7249999999999992e-06 2024-09-14T05:30:22.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391917+0000", 2024-09-14T05:30:22.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3458000000000001e-05 2024-09-14T05:30:22.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392037+0000", 2024-09-14T05:30:22.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120449 2024-09-14T05:30:22.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:22.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392305+0000", 2024-09-14T05:30:22.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026739299999999999 2024-09-14T05:30:22.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:22.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.393609+0000", 2024-09-14T05:30:22.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013042450000000001 2024-09-14T05:30:22.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:22.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401430+0000", 2024-09-14T05:30:22.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078211670000000004 2024-09-14T05:30:22.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:22.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401499+0000", 2024-09-14T05:30:22.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9374000000000004e-05 2024-09-14T05:30:22.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401544+0000", 2024-09-14T05:30:22.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5068999999999998e-05 2024-09-14T05:30:22.607 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.607 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.607 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.607 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4452 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.607 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:30:22.607 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.659406433000001, 2024-09-14T05:30:22.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066061380000000001, 2024-09-14T05:30:22.607 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.607 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.608 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:22.608 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.608 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.608 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.608 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4452 2024-09-14T05:30:22.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.608 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:30:22.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:30:22.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882314+0000", 2024-09-14T05:30:22.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5899999999999998e-07 2024-09-14T05:30:22.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882315+0000", 2024-09-14T05:30:22.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6169999999999999e-06 2024-09-14T05:30:22.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882319+0000", 2024-09-14T05:30:22.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3629999999999998e-06 2024-09-14T05:30:22.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882324+0000", 2024-09-14T05:30:22.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1370000000000001e-06 2024-09-14T05:30:22.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883369+0000", 2024-09-14T05:30:22.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010448320000000001 2024-09-14T05:30:22.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883421+0000", 2024-09-14T05:30:22.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2401000000000002e-05 2024-09-14T05:30:22.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:22.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883638+0000", 2024-09-14T05:30:22.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021725600000000001 2024-09-14T05:30:22.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:22.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.884464+0000", 2024-09-14T05:30:22.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00082569800000000004 2024-09-14T05:30:22.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:22.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888860+0000", 2024-09-14T05:30:22.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0043963370000000002 2024-09-14T05:30:22.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:22.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888896+0000", 2024-09-14T05:30:22.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6022999999999997e-05 2024-09-14T05:30:22.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888920+0000", 2024-09-14T05:30:22.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3215000000000001e-05 2024-09-14T05:30:22.613 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.614 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.614 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4467 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.614 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:22.614 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.641037982999997, 2024-09-14T05:30:22.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0065700189999999999, 2024-09-14T05:30:22.614 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.614 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.614 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:22.614 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.615 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4467 2024-09-14T05:30:22.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.615 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:22.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:22.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:22.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-14T05:30:22.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900684+0000", 2024-09-14T05:30:22.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.629e-06 2024-09-14T05:30:22.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900688+0000", 2024-09-14T05:30:22.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8469999999999998e-06 2024-09-14T05:30:22.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900691+0000", 2024-09-14T05:30:22.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-14T05:30:22.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.901775+0000", 2024-09-14T05:30:22.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001084177 2024-09-14T05:30:22.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.901862+0000", 2024-09-14T05:30:22.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7167000000000001e-05 2024-09-14T05:30:22.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:22.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.902223+0000", 2024-09-14T05:30:22.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036068800000000001 2024-09-14T05:30:22.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:22.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.902843+0000", 2024-09-14T05:30:22.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000620489 2024-09-14T05:30:22.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:22.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907167+0000", 2024-09-14T05:30:22.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0043242649999999999 2024-09-14T05:30:22.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:22.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907217+0000", 2024-09-14T05:30:22.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9648000000000003e-05 2024-09-14T05:30:22.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907252+0000", 2024-09-14T05:30:22.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4780000000000002e-05 2024-09-14T05:30:22.620 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.620 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.621 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.621 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4554 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:22.621 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:22.621 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.55731823, 2024-09-14T05:30:22.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069998999999999999, 2024-09-14T05:30:22.621 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.621 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.621 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:22.621 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.621 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.622 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.622 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4554 2024-09-14T05:30:22.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.622 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:22.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:22.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:22.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9299999999999999e-07 2024-09-14T05:30:22.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984404+0000", 2024-09-14T05:30:22.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.855e-06 2024-09-14T05:30:22.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984407+0000", 2024-09-14T05:30:22.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-14T05:30:22.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984410+0000", 2024-09-14T05:30:22.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8159999999999998e-06 2024-09-14T05:30:22.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985511+0000", 2024-09-14T05:30:22.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00110053 2024-09-14T05:30:22.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985561+0000", 2024-09-14T05:30:22.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0389000000000002e-05 2024-09-14T05:30:22.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:22.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985772+0000", 2024-09-14T05:30:22.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000210571 2024-09-14T05:30:22.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:22.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.987075+0000", 2024-09-14T05:30:22.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001303128 2024-09-14T05:30:22.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:22.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991303+0000", 2024-09-14T05:30:22.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042283310000000001 2024-09-14T05:30:22.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:22.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991359+0000", 2024-09-14T05:30:22.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6335999999999997e-05 2024-09-14T05:30:22.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991402+0000", 2024-09-14T05:30:22.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2074999999999999e-05 2024-09-14T05:30:22.627 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.627 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.627 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.627 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4711 3.4 3:226df663:::rbd_data.112d3d611e54.000000000000002d:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:22.628 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:22.628 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.374647074000002, 2024-09-14T05:30:22.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011304193000000001, 2024-09-14T05:30:22.628 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.628 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.628 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:22.628 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.628 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.628 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.628 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4711 2024-09-14T05:30:22.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.629 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:22.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:22.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167074+0000", 2024-09-14T05:30:22.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0999999999999997e-07 2024-09-14T05:30:22.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167078+0000", 2024-09-14T05:30:22.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.878e-06 2024-09-14T05:30:22.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167087+0000", 2024-09-14T05:30:22.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6940000000000007e-06 2024-09-14T05:30:22.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167093+0000", 2024-09-14T05:30:22.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7740000000000001e-06 2024-09-14T05:30:22.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167128+0000", 2024-09-14T05:30:22.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5231999999999997e-05 2024-09-14T05:30:22.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167207+0000", 2024-09-14T05:30:22.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9036999999999996e-05 2024-09-14T05:30:22.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:22.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.169140+0000", 2024-09-14T05:30:22.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001932897 2024-09-14T05:30:22.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:22.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178258+0000", 2024-09-14T05:30:22.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091180819999999996 2024-09-14T05:30:22.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:22.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178314+0000", 2024-09-14T05:30:22.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5571e-05 2024-09-14T05:30:22.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:22.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178348+0000", 2024-09-14T05:30:22.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3747000000000001e-05 2024-09-14T05:30:22.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178377+0000", 2024-09-14T05:30:22.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9471e-05 2024-09-14T05:30:22.633 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.633 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.634 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.634 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4721 3.1 3:86302a6e:::rbd_data.112d3d611e54.0000000000000011:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:22.634 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:30:22.634 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.342661587999999, 2024-09-14T05:30:22.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066831750000000004, 2024-09-14T05:30:22.634 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.634 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.634 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:22.634 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.634 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.634 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.635 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4721 2024-09-14T05:30:22.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.635 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:30:22.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:30:22.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199059+0000", 2024-09-14T05:30:22.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.06e-07 2024-09-14T05:30:22.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199062+0000", 2024-09-14T05:30:22.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2279999999999998e-06 2024-09-14T05:30:22.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199072+0000", 2024-09-14T05:30:22.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5999999999999996e-06 2024-09-14T05:30:22.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199078+0000", 2024-09-14T05:30:22.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9109999999999998e-06 2024-09-14T05:30:22.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199123+0000", 2024-09-14T05:30:22.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5278000000000003e-05 2024-09-14T05:30:22.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199202+0000", 2024-09-14T05:30:22.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8521000000000003e-05 2024-09-14T05:30:22.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:22.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199271+0000", 2024-09-14T05:30:22.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9857999999999994e-05 2024-09-14T05:30:22.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:22.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199941+0000", 2024-09-14T05:30:22.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066984599999999998 2024-09-14T05:30:22.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:22.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205653+0000", 2024-09-14T05:30:22.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057117319999999997 2024-09-14T05:30:22.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:22.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205701+0000", 2024-09-14T05:30:22.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8254000000000003e-05 2024-09-14T05:30:22.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205741+0000", 2024-09-14T05:30:22.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0241000000000003e-05 2024-09-14T05:30:22.639 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.639 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.639 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.640 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4819 3.12 3:4e8efced:::rbd_data.112d3d611e54.000000000000000e:head [stat,truncate 1114112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:22.640 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:30:22.640 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.257347428999999, 2024-09-14T05:30:22.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062422770000000001, 2024-09-14T05:30:22.640 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.640 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.640 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:22.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:22.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:22.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:22.640 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4819 2024-09-14T05:30:22.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.641 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:30:22.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:30:22.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284373+0000", 2024-09-14T05:30:22.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3399999999999998e-07 2024-09-14T05:30:22.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284377+0000", 2024-09-14T05:30:22.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8539999999999999e-06 2024-09-14T05:30:22.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284388+0000", 2024-09-14T05:30:22.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1328999999999999e-05 2024-09-14T05:30:22.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284395+0000", 2024-09-14T05:30:22.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9759999999999998e-06 2024-09-14T05:30:22.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284723+0000", 2024-09-14T05:30:22.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032775799999999999 2024-09-14T05:30:22.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284801+0000", 2024-09-14T05:30:22.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7562e-05 2024-09-14T05:30:22.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:30:22.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284933+0000", 2024-09-14T05:30:22.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000131989 2024-09-14T05:30:22.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:22.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.287450+0000", 2024-09-14T05:30:22.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025177699999999999 2024-09-14T05:30:22.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:22.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290531+0000", 2024-09-14T05:30:22.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0030809959999999999 2024-09-14T05:30:22.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:22.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290588+0000", 2024-09-14T05:30:22.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6177000000000001e-05 2024-09-14T05:30:22.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290615+0000", 2024-09-14T05:30:22.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7132e-05 2024-09-14T05:30:22.646 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.646 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.647 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.647 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:7067 3.5 e35/24 3:a65d0cff:::rbd_data.112df7f2fe0a.0000000000000028:head v 35'89, mlcod=34'86)", 2024-09-14T05:30:22.647 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:22.647 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.398954158000002, 2024-09-14T05:30:22.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0064513080000000002, 2024-09-14T05:30:22.647 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.647 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.647 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:22.647 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:22.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:22.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:22.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:22.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142771+0000", 2024-09-14T05:30:22.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8949999999999997e-06 2024-09-14T05:30:22.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142779+0000", 2024-09-14T05:30:22.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8699999999999992e-06 2024-09-14T05:30:22.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142785+0000", 2024-09-14T05:30:22.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4899999999999997e-06 2024-09-14T05:30:22.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142843+0000", 2024-09-14T05:30:22.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7377000000000003e-05 2024-09-14T05:30:22.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142871+0000", 2024-09-14T05:30:22.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8305000000000001e-05 2024-09-14T05:30:22.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:22.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149177+0000", 2024-09-14T05:30:22.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063058639999999996 2024-09-14T05:30:22.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149217+0000", 2024-09-14T05:30:22.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0080999999999998e-05 2024-09-14T05:30:22.652 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.652 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.652 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.653 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:72582 3.b e47/24 3:d5072567:::rbd_data.112d4a90444c.0000000000000119:head v 47'386, mlcod=46'383)", 2024-09-14T05:30:22.653 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:22.653 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.016693220999997, 2024-09-14T05:30:22.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0067720940000000002, 2024-09-14T05:30:22.653 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.653 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.653 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:22.653 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:22.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:22.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:22.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1100000000000002e-07 2024-09-14T05:30:22.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525031+0000", 2024-09-14T05:30:22.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0740000000000003e-06 2024-09-14T05:30:22.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525040+0000", 2024-09-14T05:30:22.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5329999999999994e-06 2024-09-14T05:30:22.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525045+0000", 2024-09-14T05:30:22.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:30:22.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525090+0000", 2024-09-14T05:30:22.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4743000000000002e-05 2024-09-14T05:30:22.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525102+0000", 2024-09-14T05:30:22.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2067e-05 2024-09-14T05:30:22.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:22.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531757+0000", 2024-09-14T05:30:22.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066550380000000003 2024-09-14T05:30:22.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531799+0000", 2024-09-14T05:30:22.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1640000000000001e-05 2024-09-14T05:30:22.657 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.657 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.657 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.657 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:79671 3.15 e47/24 3:ab572765:::rbd_data.112d956b542f.0000000000000013:head v 47'120, mlcod=47'118)", 2024-09-14T05:30:22.657 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:30:22.657 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.390713003000002, 2024-09-14T05:30:22.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066700609999999997, 2024-09-14T05:30:22.658 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.658 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.658 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:22.658 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:30:22.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:30:22.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151008+0000", 2024-09-14T05:30:22.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:30:22.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151264+0000", 2024-09-14T05:30:22.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025656099999999999 2024-09-14T05:30:22.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151278+0000", 2024-09-14T05:30:22.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3288999999999999e-05 2024-09-14T05:30:22.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151287+0000", 2024-09-14T05:30:22.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7909999999999993e-06 2024-09-14T05:30:22.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151361+0000", 2024-09-14T05:30:22.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3979000000000004e-05 2024-09-14T05:30:22.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151383+0000", 2024-09-14T05:30:22.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2039999999999999e-05 2024-09-14T05:30:22.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:22.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.157542+0000", 2024-09-14T05:30:22.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006158915 2024-09-14T05:30:22.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.157677+0000", 2024-09-14T05:30:22.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013544300000000001 2024-09-14T05:30:22.662 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.662 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.662 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.662 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:123378 3.4 e71/39 3:24969376:::rbd_data.112d471e22e6.0000000000000009:head v 71'440, mlcod=69'438)", 2024-09-14T05:30:22.662 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:17.791627+0000", 2024-09-14T05:30:22.663 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.7500932909999998, 2024-09-14T05:30:22.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066686189999999998, 2024-09-14T05:30:22.663 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:22.663 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:22.663 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:22.663 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:22.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:22.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.791627+0000", 2024-09-14T05:30:22.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:22.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.791627+0000", 2024-09-14T05:30:22.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:22.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:22.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.791627+0000", 2024-09-14T05:30:22.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.65e-07 2024-09-14T05:30:22.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:22.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792806+0000", 2024-09-14T05:30:22.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001179193 2024-09-14T05:30:22.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:22.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792832+0000", 2024-09-14T05:30:22.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5514999999999999e-05 2024-09-14T05:30:22.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:22.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792842+0000", 2024-09-14T05:30:22.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0481e-05 2024-09-14T05:30:22.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:22.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792919+0000", 2024-09-14T05:30:22.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6720999999999999e-05 2024-09-14T05:30:22.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:22.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792945+0000", 2024-09-14T05:30:22.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6367999999999999e-05 2024-09-14T05:30:22.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:22.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.798239+0000", 2024-09-14T05:30:22.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052942509999999998 2024-09-14T05:30:22.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:22.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:22.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:22.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.798295+0000", 2024-09-14T05:30:22.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5825000000000002e-05 2024-09-14T05:30:22.667 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.667 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.667 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.667 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:22.667 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:22.667 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:22.669 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:30:22.669 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:22.668+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:22.769 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:22.769+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:22.812 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:22.812 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:22.812 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:22.812 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:22.820 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:30:22.871 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:22.870+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:22.923 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:22.923 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:22.923 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:22.923 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:22.923 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:22.931 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:30:22.975 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:22.975+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:23.036 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:23.036 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:23.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:23.036 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:23.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.036 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2131 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:23.036 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:23.036 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.259727517999998, 2024-09-14T05:30:23.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049526593000000001, 2024-09-14T05:30:23.036 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.037 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.037 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:23.037 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.037 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.037 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.037 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2131 2024-09-14T05:30:23.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.037 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:23.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:23.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774867+0000", 2024-09-14T05:30:23.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4430000000000002e-06 2024-09-14T05:30:23.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790357+0000", 2024-09-14T05:30:23.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015489466 2024-09-14T05:30:23.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790383+0000", 2024-09-14T05:30:23.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6247000000000001e-05 2024-09-14T05:30:23.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790394+0000", 2024-09-14T05:30:23.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0728e-05 2024-09-14T05:30:23.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790471+0000", 2024-09-14T05:30:23.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7539000000000004e-05 2024-09-14T05:30:23.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790639+0000", 2024-09-14T05:30:23.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016787700000000001 2024-09-14T05:30:23.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-14T05:30:23.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790998+0000", 2024-09-14T05:30:23.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035931299999999999 2024-09-14T05:30:23.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:23.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.823529+0000", 2024-09-14T05:30:23.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032530798 2024-09-14T05:30:23.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:23.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824324+0000", 2024-09-14T05:30:23.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00079442399999999995 2024-09-14T05:30:23.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:23.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824362+0000", 2024-09-14T05:30:23.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8398000000000001e-05 2024-09-14T05:30:23.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824391+0000", 2024-09-14T05:30:23.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.936e-05 2024-09-14T05:30:23.042 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.042 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.042 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.042 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3892 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7602176~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.043 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:23.043 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.678149519999998, 2024-09-14T05:30:23.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047082951999999997, 2024-09-14T05:30:23.043 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.043 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.043 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.043 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.043 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.043 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.043 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3892 2024-09-14T05:30:23.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.044 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:23.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:23.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:23.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0199999999999998e-07 2024-09-14T05:30:23.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356454+0000", 2024-09-14T05:30:23.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0414999999999999e-05 2024-09-14T05:30:23.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356463+0000", 2024-09-14T05:30:23.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6739999999999992e-06 2024-09-14T05:30:23.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356468+0000", 2024-09-14T05:30:23.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5680000000000001e-06 2024-09-14T05:30:23.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403143+0000", 2024-09-14T05:30:23.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046675051000000002 2024-09-14T05:30:23.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403206+0000", 2024-09-14T05:30:23.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3127999999999996e-05 2024-09-14T05:30:23.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403526+0000", 2024-09-14T05:30:23.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031981400000000002 2024-09-14T05:30:23.048 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.048 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.048 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.048 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3893 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8093696~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.048 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:23.048 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.678104683000001, 2024-09-14T05:30:23.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047516305000000002, 2024-09-14T05:30:23.049 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.049 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.049 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.049 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3893 2024-09-14T05:30:23.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.049 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:23.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:23.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6899999999999997e-07 2024-09-14T05:30:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356491+0000", 2024-09-14T05:30:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8109999999999999e-06 2024-09-14T05:30:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356498+0000", 2024-09-14T05:30:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5320000000000003e-06 2024-09-14T05:30:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356502+0000", 2024-09-14T05:30:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000002e-06 2024-09-14T05:30:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403579+0000", 2024-09-14T05:30:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047076706000000003 2024-09-14T05:30:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403664+0000", 2024-09-14T05:30:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5240000000000004e-05 2024-09-14T05:30:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404004+0000", 2024-09-14T05:30:23.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033974500000000002 2024-09-14T05:30:23.053 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.053 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.053 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.053 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3993 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 262144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.053 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:23.053 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.678082812, 2024-09-14T05:30:23.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048088476999999998, 2024-09-14T05:30:23.054 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.054 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.054 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.054 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.054 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.054 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.054 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3993 2024-09-14T05:30:23.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.054 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:23.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:23.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:23.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.46e-07 2024-09-14T05:30:23.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356512+0000", 2024-09-14T05:30:23.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.827e-06 2024-09-14T05:30:23.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356518+0000", 2024-09-14T05:30:23.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7780000000000004e-06 2024-09-14T05:30:23.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356522+0000", 2024-09-14T05:30:23.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.865e-06 2024-09-14T05:30:23.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404036+0000", 2024-09-14T05:30:23.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047513725999999999 2024-09-14T05:30:23.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404135+0000", 2024-09-14T05:30:23.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9352999999999996e-05 2024-09-14T05:30:23.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404598+0000", 2024-09-14T05:30:23.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046258200000000001 2024-09-14T05:30:23.058 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.058 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.058 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.058 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3994 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.058 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:23.059 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.678040799999998, 2024-09-14T05:30:23.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048274625000000002, 2024-09-14T05:30:23.059 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.059 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.059 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.059 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.059 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.059 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.059 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3994 2024-09-14T05:30:23.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.059 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:23.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:23.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:23.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3799999999999997e-07 2024-09-14T05:30:23.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356554+0000", 2024-09-14T05:30:23.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2259999999999999e-06 2024-09-14T05:30:23.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356560+0000", 2024-09-14T05:30:23.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-14T05:30:23.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356566+0000", 2024-09-14T05:30:23.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3909999999999997e-06 2024-09-14T05:30:23.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404632+0000", 2024-09-14T05:30:23.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048066154 2024-09-14T05:30:23.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404719+0000", 2024-09-14T05:30:23.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7440000000000003e-05 2024-09-14T05:30:23.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404826+0000", 2024-09-14T05:30:23.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106813 2024-09-14T05:30:23.062 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.062 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3995 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.678015661000003, 2024-09-14T05:30:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048828209999999997, 2024-09-14T05:30:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3995 2024-09-14T05:30:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6599999999999996e-07 2024-09-14T05:30:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356579+0000", 2024-09-14T05:30:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0719999999999998e-06 2024-09-14T05:30:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356585+0000", 2024-09-14T05:30:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.959e-06 2024-09-14T05:30:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356590+0000", 2024-09-14T05:30:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.464e-06 2024-09-14T05:30:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404875+0000", 2024-09-14T05:30:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048285649999999999 2024-09-14T05:30:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404985+0000", 2024-09-14T05:30:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109417 2024-09-14T05:30:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405405+0000", 2024-09-14T05:30:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042008200000000001 2024-09-14T05:30:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3996 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.677706778000001, 2024-09-14T05:30:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049061595, 2024-09-14T05:30:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3996 2024-09-14T05:30:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.61e-07 2024-09-14T05:30:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356892+0000", 2024-09-14T05:30:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9050000000000002e-06 2024-09-14T05:30:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356901+0000", 2024-09-14T05:30:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4149999999999999e-06 2024-09-14T05:30:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356907+0000", 2024-09-14T05:30:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0140000000000004e-06 2024-09-14T05:30:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405448+0000", 2024-09-14T05:30:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048541647 2024-09-14T05:30:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405542+0000", 2024-09-14T05:30:23.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3639000000000003e-05 2024-09-14T05:30:23.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405947+0000", 2024-09-14T05:30:23.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040521399999999999 2024-09-14T05:30:23.071 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.071 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.071 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.071 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3997 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2228224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.677492796000003, 2024-09-14T05:30:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049308774999999999, 2024-09-14T05:30:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3997 2024-09-14T05:30:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:23.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:23.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6499999999999999e-07 2024-09-14T05:30:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357104+0000", 2024-09-14T05:30:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4989999999999999e-06 2024-09-14T05:30:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357112+0000", 2024-09-14T05:30:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4279999999999996e-06 2024-09-14T05:30:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357117+0000", 2024-09-14T05:30:23.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6190000000000002e-06 2024-09-14T05:30:23.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405989+0000", 2024-09-14T05:30:23.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048872080999999998 2024-09-14T05:30:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406074+0000", 2024-09-14T05:30:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4676e-05 2024-09-14T05:30:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406408+0000", 2024-09-14T05:30:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033480700000000001 2024-09-14T05:30:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.077 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3998 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.077 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:23.077 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.677171367, 2024-09-14T05:30:23.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049462314, 2024-09-14T05:30:23.077 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.077 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.077 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.078 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.078 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3998 2024-09-14T05:30:23.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.078 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:23.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:23.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357422+0000", 2024-09-14T05:30:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3099999999999996e-07 2024-09-14T05:30:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357429+0000", 2024-09-14T05:30:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2880000000000004e-06 2024-09-14T05:30:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357439+0000", 2024-09-14T05:30:23.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.302e-06 2024-09-14T05:30:23.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357445+0000", 2024-09-14T05:30:23.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4269999999999996e-06 2024-09-14T05:30:23.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406448+0000", 2024-09-14T05:30:23.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049003096000000003 2024-09-14T05:30:23.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406537+0000", 2024-09-14T05:30:23.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9135999999999995e-05 2024-09-14T05:30:23.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406883+0000", 2024-09-14T05:30:23.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000346134 2024-09-14T05:30:23.081 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.081 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.081 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.081 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3999 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.082 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:23.082 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.677101010000001, 2024-09-14T05:30:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049797556, 2024-09-14T05:30:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3999 2024-09-14T05:30:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:23.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357492+0000", 2024-09-14T05:30:23.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8299999999999996e-07 2024-09-14T05:30:23.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357495+0000", 2024-09-14T05:30:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0450000000000001e-06 2024-09-14T05:30:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357503+0000", 2024-09-14T05:30:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9689999999999994e-06 2024-09-14T05:30:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357510+0000", 2024-09-14T05:30:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-14T05:30:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406914+0000", 2024-09-14T05:30:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049404293000000002 2024-09-14T05:30:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406965+0000", 2024-09-14T05:30:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0470999999999998e-05 2024-09-14T05:30:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407289+0000", 2024-09-14T05:30:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032434899999999998 2024-09-14T05:30:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4000 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3702784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.676754664999997, 2024-09-14T05:30:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049936728999999999, 2024-09-14T05:30:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.088 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.088 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4000 2024-09-14T05:30:23.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.088 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:23.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:23.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357839+0000", 2024-09-14T05:30:23.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-14T05:30:23.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357842+0000", 2024-09-14T05:30:23.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4850000000000001e-06 2024-09-14T05:30:23.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357851+0000", 2024-09-14T05:30:23.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1819999999999995e-06 2024-09-14T05:30:23.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357859+0000", 2024-09-14T05:30:23.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8129999999999996e-06 2024-09-14T05:30:23.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407337+0000", 2024-09-14T05:30:23.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049478084999999998 2024-09-14T05:30:23.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407442+0000", 2024-09-14T05:30:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000105302 2024-09-14T05:30:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407774+0000", 2024-09-14T05:30:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033200100000000001 2024-09-14T05:30:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4001 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.676659309999998, 2024-09-14T05:30:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050304205999999997, 2024-09-14T05:30:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4001 2024-09-14T05:30:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:23.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:23.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357934+0000", 2024-09-14T05:30:23.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000004e-07 2024-09-14T05:30:23.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357937+0000", 2024-09-14T05:30:23.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0869999999999998e-06 2024-09-14T05:30:23.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357946+0000", 2024-09-14T05:30:23.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1439999999999992e-06 2024-09-14T05:30:23.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357952+0000", 2024-09-14T05:30:23.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.609e-06 2024-09-14T05:30:23.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407820+0000", 2024-09-14T05:30:23.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049868580000000003 2024-09-14T05:30:23.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407909+0000", 2024-09-14T05:30:23.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8887000000000005e-05 2024-09-14T05:30:23.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408237+0000", 2024-09-14T05:30:23.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032824599999999997 2024-09-14T05:30:23.095 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.096 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.096 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.096 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4002 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.096 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:23.096 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.676442715, 2024-09-14T05:30:23.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050539529999999999, 2024-09-14T05:30:23.096 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.096 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.096 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.096 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.096 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.097 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.097 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4002 2024-09-14T05:30:23.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.097 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:23.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:23.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:23.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5099999999999999e-07 2024-09-14T05:30:23.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358154+0000", 2024-09-14T05:30:23.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6799999999999999e-06 2024-09-14T05:30:23.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358165+0000", 2024-09-14T05:30:23.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0899e-05 2024-09-14T05:30:23.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358172+0000", 2024-09-14T05:30:23.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8689999999999998e-06 2024-09-14T05:30:23.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408281+0000", 2024-09-14T05:30:23.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050108699 2024-09-14T05:30:23.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408369+0000", 2024-09-14T05:30:23.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8542999999999996e-05 2024-09-14T05:30:23.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408689+0000", 2024-09-14T05:30:23.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032008900000000002 2024-09-14T05:30:23.100 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.100 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.100 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.100 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4003 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5177344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.100 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:23.101 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.676402879999998, 2024-09-14T05:30:23.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050868759, 2024-09-14T05:30:23.101 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.101 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.101 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.101 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.101 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.101 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.101 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4003 2024-09-14T05:30:23.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.101 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:23.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:23.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:23.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-07 2024-09-14T05:30:23.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358193+0000", 2024-09-14T05:30:23.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.588e-06 2024-09-14T05:30:23.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358204+0000", 2024-09-14T05:30:23.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1525999999999999e-05 2024-09-14T05:30:23.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358213+0000", 2024-09-14T05:30:23.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5010000000000004e-06 2024-09-14T05:30:23.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408719+0000", 2024-09-14T05:30:23.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050506708999999997 2024-09-14T05:30:23.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408766+0000", 2024-09-14T05:30:23.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6579999999999998e-05 2024-09-14T05:30:23.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409058+0000", 2024-09-14T05:30:23.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000292365 2024-09-14T05:30:23.105 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.105 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.105 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.105 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4004 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5668864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.105 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:23.105 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.675980037000002, 2024-09-14T05:30:23.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050839848, 2024-09-14T05:30:23.105 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.106 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.106 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.106 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.106 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.106 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.106 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4004 2024-09-14T05:30:23.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.106 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:23.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:23.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358613+0000", 2024-09-14T05:30:23.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:30:23.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358618+0000", 2024-09-14T05:30:23.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6009999999999997e-06 2024-09-14T05:30:23.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358630+0000", 2024-09-14T05:30:23.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2122e-05 2024-09-14T05:30:23.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358638+0000", 2024-09-14T05:30:23.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3969999999999994e-06 2024-09-14T05:30:23.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409088+0000", 2024-09-14T05:30:23.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050449565000000002 2024-09-14T05:30:23.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409153+0000", 2024-09-14T05:30:23.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4728999999999996e-05 2024-09-14T05:30:23.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409452+0000", 2024-09-14T05:30:23.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029958900000000001 2024-09-14T05:30:23.109 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.109 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.109 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.110 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4005 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6160384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.110 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:23.110 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.675763103000001, 2024-09-14T05:30:23.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051054752000000002, 2024-09-14T05:30:23.110 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.110 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.110 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.110 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.110 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.110 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.110 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4005 2024-09-14T05:30:23.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.111 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:23.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:23.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358830+0000", 2024-09-14T05:30:23.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-14T05:30:23.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358833+0000", 2024-09-14T05:30:23.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9529999999999999e-06 2024-09-14T05:30:23.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358843+0000", 2024-09-14T05:30:23.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8069999999999994e-06 2024-09-14T05:30:23.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358849+0000", 2024-09-14T05:30:23.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1040000000000003e-06 2024-09-14T05:30:23.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409487+0000", 2024-09-14T05:30:23.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050638626999999999 2024-09-14T05:30:23.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409535+0000", 2024-09-14T05:30:23.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7525000000000003e-05 2024-09-14T05:30:23.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409884+0000", 2024-09-14T05:30:23.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034914800000000003 2024-09-14T05:30:23.114 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.114 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.114 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.114 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4006 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.114 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:23.114 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.675725991, 2024-09-14T05:30:23.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051425323000000002, 2024-09-14T05:30:23.115 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.115 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.115 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.115 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.115 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.115 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.115 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4006 2024-09-14T05:30:23.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.115 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:23.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:23.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358867+0000", 2024-09-14T05:30:23.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2900000000000004e-07 2024-09-14T05:30:23.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358869+0000", 2024-09-14T05:30:23.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4210000000000002e-06 2024-09-14T05:30:23.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358876+0000", 2024-09-14T05:30:23.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4980000000000003e-06 2024-09-14T05:30:23.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358881+0000", 2024-09-14T05:30:23.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3090000000000003e-06 2024-09-14T05:30:23.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409915+0000", 2024-09-14T05:30:23.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051033348999999999 2024-09-14T05:30:23.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409964+0000", 2024-09-14T05:30:23.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9858000000000003e-05 2024-09-14T05:30:23.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410292+0000", 2024-09-14T05:30:23.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032735899999999999 2024-09-14T05:30:23.119 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.119 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.119 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.119 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4007 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.119 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:23.119 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.675703398000003, 2024-09-14T05:30:23.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05184076, 2024-09-14T05:30:23.120 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.120 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.120 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.120 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.120 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.120 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.120 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4007 2024-09-14T05:30:23.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.120 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:23.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:23.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:23.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:30:23.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358891+0000", 2024-09-14T05:30:23.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9819999999999998e-06 2024-09-14T05:30:23.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358897+0000", 2024-09-14T05:30:23.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9390000000000002e-06 2024-09-14T05:30:23.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358902+0000", 2024-09-14T05:30:23.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6770000000000003e-06 2024-09-14T05:30:23.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410335+0000", 2024-09-14T05:30:23.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051432991999999997 2024-09-14T05:30:23.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410416+0000", 2024-09-14T05:30:23.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1316999999999994e-05 2024-09-14T05:30:23.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410730+0000", 2024-09-14T05:30:23.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000313419 2024-09-14T05:30:23.123 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.123 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.124 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.124 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4008 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7634944~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.124 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:23.124 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.675679037000002, 2024-09-14T05:30:23.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052208082000000003, 2024-09-14T05:30:23.124 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.124 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.124 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.124 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.124 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.124 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.124 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4008 2024-09-14T05:30:23.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.125 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:23.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:23.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358914+0000", 2024-09-14T05:30:23.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6100000000000001e-07 2024-09-14T05:30:23.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358916+0000", 2024-09-14T05:30:23.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.902e-06 2024-09-14T05:30:23.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358923+0000", 2024-09-14T05:30:23.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4780000000000002e-06 2024-09-14T05:30:23.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358929+0000", 2024-09-14T05:30:23.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0970000000000002e-06 2024-09-14T05:30:23.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410761+0000", 2024-09-14T05:30:23.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051831436000000002 2024-09-14T05:30:23.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410829+0000", 2024-09-14T05:30:23.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7874999999999996e-05 2024-09-14T05:30:23.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411121+0000", 2024-09-14T05:30:23.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029283299999999999 2024-09-14T05:30:23.128 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.128 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.128 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.129 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4009 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8126464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:23.129 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:23.129 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.675531812000003, 2024-09-14T05:30:23.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052518169000000003, 2024-09-14T05:30:23.129 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:23.129 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:23.129 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:23.129 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:23.129 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:23.129 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:23.129 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4009 2024-09-14T05:30:23.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.129 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:23.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:23.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:23.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:23.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:23.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:23.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:23.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:23.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0200000000000002e-07 2024-09-14T05:30:23.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:23.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359066+0000", 2024-09-14T05:30:23.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7899999999999999e-06 2024-09-14T05:30:23.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:23.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359074+0000", 2024-09-14T05:30:23.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2579999999999995e-06 2024-09-14T05:30:23.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:23.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359082+0000", 2024-09-14T05:30:23.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.464e-06 2024-09-14T05:30:23.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:23.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411160+0000", 2024-09-14T05:30:23.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052078078 2024-09-14T05:30:23.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:23.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411244+0000", 2024-09-14T05:30:23.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4041000000000004e-05 2024-09-14T05:30:23.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:23.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:23.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:23.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411579+0000", 2024-09-14T05:30:23.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033493599999999998 2024-09-14T05:30:23.133 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.133 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.133 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.133 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:23.133 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:23.133 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:23.134 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:30:23.135 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:23.134+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:23.136 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:30:23.235 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:23.235+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:23.244 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:23.244 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:23.244 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:23.245 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:23.253 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:30:23.260 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:23.260 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:23.278 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:23.278 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:23.294 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:23.294 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:23.310 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:23.310 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:23.327 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:23.327 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:23.331 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-14T05:30:23.336 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:23.335+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:23.352 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:23.353 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:23.369 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:23.369 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:23.370 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:23.370 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:23.381 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:30:23.392 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:23.392 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:23.437 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:23.436+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:23.498 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:23.498 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:23.506 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:30:23.538 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:23.537+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:23.609 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:23.609 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:23.617 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:30:23.639 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:23.638+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:23.720 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:23.720 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:23.729 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:30:23.773 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:23.772+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:23.830 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:23.830 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:23.839 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:30:23.880 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:23.879+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:23.941 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:23.941+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:23.982 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:23.982 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:23.991 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:30:24.042 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:24.042+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:24.133 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:24.133 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:24.142 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:30:24.143 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:24.142+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:24.243 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:24.243+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:24.245 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:24.245 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:24.254 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:30:24.344 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:24.343+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:24.361 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:24.361 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:24.370 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:30:24.445 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:24.444+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:24.483 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:24.483 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:24.493 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:30:24.546 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:24.545+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:24.596 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:24.596 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:24.605 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:30:24.646 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:24.646+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:24.712 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:24.712 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:24.721 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:30:24.765 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:24.764+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:24.825 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:24.825 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:24.833 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:30:24.848 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:24.848+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:24.949 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:24.949+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:24.977 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:24.977 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:24.985 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:30:25.051 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:25.050+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:25.130 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:25.130 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:25.138 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:30:25.181 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:25.181+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:25.242 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:25.242 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:25.250 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:30:25.292 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:25.291+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:25.353 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:25.353+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:25.356 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T05:30:25.400 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:25.401 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:25.405 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:30:25.409 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:30:25.493 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:25.492+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:25.545 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:25.545 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:25.556 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:25.556 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:25.556 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:25.554+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:25.563 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:25.563 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:25.564 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:30:25.579 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:25.580 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:25.596 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:25.596 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:25.613 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:25.613 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:25.629 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:25.630 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:25.645 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:25.646 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:25.656 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:25.655+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:25.663 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:25.663 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:25.708 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:25.708 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:25.709 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:25.709 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:25.709 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:25.717 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:30:25.756 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:25.756+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:25.857 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:25.857+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:25.861 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:25.861 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:25.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:25.862 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:25.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.862 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1205 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:25.862 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:25.862 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.662361161, 2024-09-14T05:30:25.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0086364279999999998, 2024-09-14T05:30:25.862 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.862 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.862 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:25.862 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:25.862 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:25.862 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:25.862 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1205 2024-09-14T05:30:25.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.863 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:25.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:25.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197891+0000", 2024-09-14T05:30:25.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.364e-06 2024-09-14T05:30:25.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197896+0000", 2024-09-14T05:30:25.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3000000000000001e-06 2024-09-14T05:30:25.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197910+0000", 2024-09-14T05:30:25.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3339000000000001e-05 2024-09-14T05:30:25.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197919+0000", 2024-09-14T05:30:25.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8869999999999998e-06 2024-09-14T05:30:25.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197974+0000", 2024-09-14T05:30:25.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5720000000000002e-05 2024-09-14T05:30:25.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.198094+0000", 2024-09-14T05:30:25.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119739 2024-09-14T05:30:25.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206526+0000", 2024-09-14T05:30:25.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084320790000000003 2024-09-14T05:30:25.867 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.867 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.867 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.867 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1220 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:25.867 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:25.867 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.653604797, 2024-09-14T05:30:25.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012822337, 2024-09-14T05:30:25.867 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.867 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.867 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:25.868 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:25.868 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:25.868 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:25.868 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1220 2024-09-14T05:30:25.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.868 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:25.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:25.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206647+0000", 2024-09-14T05:30:25.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4600000000000003e-07 2024-09-14T05:30:25.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206652+0000", 2024-09-14T05:30:25.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6619999999999996e-06 2024-09-14T05:30:25.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206662+0000", 2024-09-14T05:30:25.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.995e-06 2024-09-14T05:30:25.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206667+0000", 2024-09-14T05:30:25.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-14T05:30:25.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206706+0000", 2024-09-14T05:30:25.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9523000000000001e-05 2024-09-14T05:30:25.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206755+0000", 2024-09-14T05:30:25.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000002e-05 2024-09-14T05:30:25.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:25.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206877+0000", 2024-09-14T05:30:25.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121938 2024-09-14T05:30:25.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:25.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.207310+0000", 2024-09-14T05:30:25.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000432452 2024-09-14T05:30:25.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:25.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219364+0000", 2024-09-14T05:30:25.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012054481000000001 2024-09-14T05:30:25.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:25.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219425+0000", 2024-09-14T05:30:25.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0618000000000001e-05 2024-09-14T05:30:25.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219468+0000", 2024-09-14T05:30:25.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3739000000000003e-05 2024-09-14T05:30:25.873 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.873 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.873 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.873 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1243 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:25.873 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:25.873 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.640639569999998, 2024-09-14T05:30:25.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078660580000000004, 2024-09-14T05:30:25.873 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.873 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.874 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:25.874 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:25.874 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:25.874 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:25.874 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1243 2024-09-14T05:30:25.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.874 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:25.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:25.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219612+0000", 2024-09-14T05:30:25.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.09e-07 2024-09-14T05:30:25.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219616+0000", 2024-09-14T05:30:25.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.216e-06 2024-09-14T05:30:25.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219630+0000", 2024-09-14T05:30:25.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3064999999999999e-05 2024-09-14T05:30:25.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219637+0000", 2024-09-14T05:30:25.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7409999999999992e-06 2024-09-14T05:30:25.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219697+0000", 2024-09-14T05:30:25.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.02e-05 2024-09-14T05:30:25.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219833+0000", 2024-09-14T05:30:25.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013595600000000001 2024-09-14T05:30:25.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227477+0000", 2024-09-14T05:30:25.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076439710000000003 2024-09-14T05:30:25.878 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.878 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.878 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.878 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1264 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:25.878 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:25.878 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.632628777000001, 2024-09-14T05:30:25.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080867990000000004, 2024-09-14T05:30:25.878 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.878 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.878 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:25.878 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:25.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:25.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:25.879 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1264 2024-09-14T05:30:25.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.879 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:25.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:25.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227623+0000", 2024-09-14T05:30:25.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:30:25.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227627+0000", 2024-09-14T05:30:25.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.194e-06 2024-09-14T05:30:25.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227639+0000", 2024-09-14T05:30:25.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1296e-05 2024-09-14T05:30:25.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227648+0000", 2024-09-14T05:30:25.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2280000000000004e-06 2024-09-14T05:30:25.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227689+0000", 2024-09-14T05:30:25.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1329999999999999e-05 2024-09-14T05:30:25.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227741+0000", 2024-09-14T05:30:25.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1641000000000002e-05 2024-09-14T05:30:25.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:30:25.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227914+0000", 2024-09-14T05:30:25.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000173215 2024-09-14T05:30:25.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:25.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228474+0000", 2024-09-14T05:30:25.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055980199999999998 2024-09-14T05:30:25.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:25.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235608+0000", 2024-09-14T05:30:25.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0071343830000000002 2024-09-14T05:30:25.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:25.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235669+0000", 2024-09-14T05:30:25.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1068999999999994e-05 2024-09-14T05:30:25.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235709+0000", 2024-09-14T05:30:25.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9671e-05 2024-09-14T05:30:25.884 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.884 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.884 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.884 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1520 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:25.884 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:30:25.885 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.537621158, 2024-09-14T05:30:25.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061963629999999999, 2024-09-14T05:30:25.885 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.885 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.885 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:25.885 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:25.885 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:25.885 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:25.885 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1520 2024-09-14T05:30:25.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.885 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:30:25.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322630+0000", 2024-09-14T05:30:25.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322631+0000", 2024-09-14T05:30:25.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.54e-07 2024-09-14T05:30:25.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322635+0000", 2024-09-14T05:30:25.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0450000000000002e-06 2024-09-14T05:30:25.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322646+0000", 2024-09-14T05:30:25.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1919999999999999e-05 2024-09-14T05:30:25.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322654+0000", 2024-09-14T05:30:25.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3139999999999998e-06 2024-09-14T05:30:25.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322704+0000", 2024-09-14T05:30:25.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0420000000000002e-05 2024-09-14T05:30:25.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.322794+0000", 2024-09-14T05:30:25.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0037000000000005e-05 2024-09-14T05:30:25.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328826+0000", 2024-09-14T05:30:25.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060318730000000001 2024-09-14T05:30:25.890 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.890 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.890 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.891 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1526 3.4 3:2695cbe4:::rbd_data.112dd25fb4c.0000000000000119:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:25.891 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:25.891 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.534088214000001, 2024-09-14T05:30:25.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0079537359999999994, 2024-09-14T05:30:25.891 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.891 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.891 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:25.891 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:25.891 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:25.891 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:25.891 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1526 2024-09-14T05:30:25.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.891 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:25.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:25.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:25.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3600000000000003e-07 2024-09-14T05:30:25.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326168+0000", 2024-09-14T05:30:25.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3460000000000004e-06 2024-09-14T05:30:25.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326178+0000", 2024-09-14T05:30:25.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0291e-05 2024-09-14T05:30:25.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326183+0000", 2024-09-14T05:30:25.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-14T05:30:25.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326230+0000", 2024-09-14T05:30:25.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6650999999999998e-05 2024-09-14T05:30:25.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326295+0000", 2024-09-14T05:30:25.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5158000000000002e-05 2024-09-14T05:30:25.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:25.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326408+0000", 2024-09-14T05:30:25.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113039 2024-09-14T05:30:25.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:25.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.331648+0000", 2024-09-14T05:30:25.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052404929999999997 2024-09-14T05:30:25.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:25.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334011+0000", 2024-09-14T05:30:25.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002362669 2024-09-14T05:30:25.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:25.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334064+0000", 2024-09-14T05:30:25.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3198999999999997e-05 2024-09-14T05:30:25.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334116+0000", 2024-09-14T05:30:25.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2133999999999999e-05 2024-09-14T05:30:25.897 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.897 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.897 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.897 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1529 3.1 3:809746f7:::rbd_data.112dd25fb4c.0000000000000118:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:25.898 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:30:25.898 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.532323664000003, 2024-09-14T05:30:25.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061981060000000001, 2024-09-14T05:30:25.898 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.898 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.898 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:25.898 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:25.898 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:25.898 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:25.898 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1529 2024-09-14T05:30:25.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.898 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:30:25.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:30:25.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327928+0000", 2024-09-14T05:30:25.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0100000000000004e-07 2024-09-14T05:30:25.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327933+0000", 2024-09-14T05:30:25.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4689999999999999e-06 2024-09-14T05:30:25.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327944+0000", 2024-09-14T05:30:25.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1623999999999999e-05 2024-09-14T05:30:25.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327951+0000", 2024-09-14T05:30:25.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9759999999999998e-06 2024-09-14T05:30:25.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328003+0000", 2024-09-14T05:30:25.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1521999999999999e-05 2024-09-14T05:30:25.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328087+0000", 2024-09-14T05:30:25.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3988999999999998e-05 2024-09-14T05:30:25.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:25.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328304+0000", 2024-09-14T05:30:25.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000217254 2024-09-14T05:30:25.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:25.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.333363+0000", 2024-09-14T05:30:25.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050586499999999996 2024-09-14T05:30:25.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:25.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334026+0000", 2024-09-14T05:30:25.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066303499999999997 2024-09-14T05:30:25.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:25.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334080+0000", 2024-09-14T05:30:25.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4261000000000001e-05 2024-09-14T05:30:25.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334125+0000", 2024-09-14T05:30:25.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5525000000000002e-05 2024-09-14T05:30:25.903 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.904 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.904 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.904 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1531 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:25.904 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:25.904 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.531270048000003, 2024-09-14T05:30:25.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082114309999999999, 2024-09-14T05:30:25.904 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.904 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.904 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:25.904 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:25.904 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:25.905 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:25.905 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1531 2024-09-14T05:30:25.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.905 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:25.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:25.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328982+0000", 2024-09-14T05:30:25.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:30:25.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328985+0000", 2024-09-14T05:30:25.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7900000000000001e-06 2024-09-14T05:30:25.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328996+0000", 2024-09-14T05:30:25.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.028e-05 2024-09-14T05:30:25.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329001+0000", 2024-09-14T05:30:25.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0610000000000003e-06 2024-09-14T05:30:25.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329035+0000", 2024-09-14T05:30:25.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4366999999999998e-05 2024-09-14T05:30:25.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329082+0000", 2024-09-14T05:30:25.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6646e-05 2024-09-14T05:30:25.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:25.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329195+0000", 2024-09-14T05:30:25.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113279 2024-09-14T05:30:25.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:25.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334293+0000", 2024-09-14T05:30:25.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050980399999999999 2024-09-14T05:30:25.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:25.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337101+0000", 2024-09-14T05:30:25.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0028076170000000001 2024-09-14T05:30:25.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:25.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337154+0000", 2024-09-14T05:30:25.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3641999999999999e-05 2024-09-14T05:30:25.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337192+0000", 2024-09-14T05:30:25.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7932000000000002e-05 2024-09-14T05:30:25.909 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.909 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.909 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.910 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:2131 3.b e30/24 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head v 30'36, mlcod=30'34)", 2024-09-14T05:30:25.910 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:25.910 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.067922693, 2024-09-14T05:30:25.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031871709999999998, 2024-09-14T05:30:25.910 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.910 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.910 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:25.910 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:25.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:25.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792331+0000", 2024-09-14T05:30:25.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6900000000000001e-06 2024-09-14T05:30:25.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797319+0000", 2024-09-14T05:30:25.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049876349999999998 2024-09-14T05:30:25.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797349+0000", 2024-09-14T05:30:25.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0046e-05 2024-09-14T05:30:25.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797359+0000", 2024-09-14T05:30:25.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0037999999999999e-05 2024-09-14T05:30:25.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797445+0000", 2024-09-14T05:30:25.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6132999999999997e-05 2024-09-14T05:30:25.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797472+0000", 2024-09-14T05:30:25.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7478999999999999e-05 2024-09-14T05:30:25.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:25.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824161+0000", 2024-09-14T05:30:25.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026688714999999998 2024-09-14T05:30:25.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824200+0000", 2024-09-14T05:30:25.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8974000000000003e-05 2024-09-14T05:30:25.914 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.914 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.914 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.914 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4259 3.4 3:25f21dc9:::rbd_data.112d3d611e54.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 294912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:25.914 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:25.914 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.468750079000003, 2024-09-14T05:30:25.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010043553, 2024-09-14T05:30:25.914 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.914 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.915 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:25.915 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:25.915 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:25.915 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:25.915 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4259 2024-09-14T05:30:25.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.915 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:25.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:25.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391503+0000", 2024-09-14T05:30:25.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9099999999999999e-06 2024-09-14T05:30:25.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391821+0000", 2024-09-14T05:30:25.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031871800000000001 2024-09-14T05:30:25.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391844+0000", 2024-09-14T05:30:25.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2045e-05 2024-09-14T05:30:25.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391853+0000", 2024-09-14T05:30:25.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7249999999999992e-06 2024-09-14T05:30:25.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391917+0000", 2024-09-14T05:30:25.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3458000000000001e-05 2024-09-14T05:30:25.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392037+0000", 2024-09-14T05:30:25.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120449 2024-09-14T05:30:25.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:25.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392305+0000", 2024-09-14T05:30:25.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026739299999999999 2024-09-14T05:30:25.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:25.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.393609+0000", 2024-09-14T05:30:25.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013042450000000001 2024-09-14T05:30:25.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:25.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401430+0000", 2024-09-14T05:30:25.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078211670000000004 2024-09-14T05:30:25.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:25.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401499+0000", 2024-09-14T05:30:25.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9374000000000004e-05 2024-09-14T05:30:25.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401544+0000", 2024-09-14T05:30:25.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5068999999999998e-05 2024-09-14T05:30:25.920 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.920 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.920 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.920 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4452 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:25.920 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:30:25.920 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 53.977937470999997, 2024-09-14T05:30:25.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066061380000000001, 2024-09-14T05:30:25.921 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.921 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.921 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:25.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:25.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:25.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:25.921 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4452 2024-09-14T05:30:25.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.921 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:30:25.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:30:25.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882314+0000", 2024-09-14T05:30:25.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5899999999999998e-07 2024-09-14T05:30:25.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882315+0000", 2024-09-14T05:30:25.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6169999999999999e-06 2024-09-14T05:30:25.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882319+0000", 2024-09-14T05:30:25.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3629999999999998e-06 2024-09-14T05:30:25.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882324+0000", 2024-09-14T05:30:25.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1370000000000001e-06 2024-09-14T05:30:25.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883369+0000", 2024-09-14T05:30:25.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010448320000000001 2024-09-14T05:30:25.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883421+0000", 2024-09-14T05:30:25.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2401000000000002e-05 2024-09-14T05:30:25.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:25.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883638+0000", 2024-09-14T05:30:25.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021725600000000001 2024-09-14T05:30:25.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:25.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.884464+0000", 2024-09-14T05:30:25.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00082569800000000004 2024-09-14T05:30:25.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:25.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888860+0000", 2024-09-14T05:30:25.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0043963370000000002 2024-09-14T05:30:25.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:25.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888896+0000", 2024-09-14T05:30:25.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6022999999999997e-05 2024-09-14T05:30:25.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888920+0000", 2024-09-14T05:30:25.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3215000000000001e-05 2024-09-14T05:30:25.926 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.926 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.926 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.926 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4467 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:25.926 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:25.926 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 53.959569021, 2024-09-14T05:30:25.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0065700189999999999, 2024-09-14T05:30:25.927 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.927 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.927 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:25.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:25.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:25.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:25.927 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4467 2024-09-14T05:30:25.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.927 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:25.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:25.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:25.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-14T05:30:25.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900684+0000", 2024-09-14T05:30:25.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.629e-06 2024-09-14T05:30:25.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900688+0000", 2024-09-14T05:30:25.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8469999999999998e-06 2024-09-14T05:30:25.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900691+0000", 2024-09-14T05:30:25.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-14T05:30:25.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.901775+0000", 2024-09-14T05:30:25.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001084177 2024-09-14T05:30:25.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.901862+0000", 2024-09-14T05:30:25.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7167000000000001e-05 2024-09-14T05:30:25.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:25.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.902223+0000", 2024-09-14T05:30:25.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036068800000000001 2024-09-14T05:30:25.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:25.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.902843+0000", 2024-09-14T05:30:25.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000620489 2024-09-14T05:30:25.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:25.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907167+0000", 2024-09-14T05:30:25.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0043242649999999999 2024-09-14T05:30:25.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:25.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907217+0000", 2024-09-14T05:30:25.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9648000000000003e-05 2024-09-14T05:30:25.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907252+0000", 2024-09-14T05:30:25.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4780000000000002e-05 2024-09-14T05:30:25.932 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.932 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.932 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.932 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4554 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:25.932 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:25.932 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 53.875849268000003, 2024-09-14T05:30:25.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069998999999999999, 2024-09-14T05:30:25.933 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.933 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.933 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:25.933 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:25.933 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:25.933 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:25.933 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4554 2024-09-14T05:30:25.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.933 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:25.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:25.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:25.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9299999999999999e-07 2024-09-14T05:30:25.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984404+0000", 2024-09-14T05:30:25.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.855e-06 2024-09-14T05:30:25.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984407+0000", 2024-09-14T05:30:25.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-14T05:30:25.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984410+0000", 2024-09-14T05:30:25.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8159999999999998e-06 2024-09-14T05:30:25.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985511+0000", 2024-09-14T05:30:25.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00110053 2024-09-14T05:30:25.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985561+0000", 2024-09-14T05:30:25.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0389000000000002e-05 2024-09-14T05:30:25.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:25.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985772+0000", 2024-09-14T05:30:25.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000210571 2024-09-14T05:30:25.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:25.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.987075+0000", 2024-09-14T05:30:25.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001303128 2024-09-14T05:30:25.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:25.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991303+0000", 2024-09-14T05:30:25.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042283310000000001 2024-09-14T05:30:25.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:25.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991359+0000", 2024-09-14T05:30:25.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6335999999999997e-05 2024-09-14T05:30:25.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991402+0000", 2024-09-14T05:30:25.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2074999999999999e-05 2024-09-14T05:30:25.938 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.938 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.938 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.938 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4711 3.4 3:226df663:::rbd_data.112d3d611e54.000000000000002d:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:25.938 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:25.938 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 53.693178111999998, 2024-09-14T05:30:25.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011304193000000001, 2024-09-14T05:30:25.939 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.939 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.939 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:25.939 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:25.939 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:25.939 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:25.939 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4711 2024-09-14T05:30:25.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.939 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:25.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:25.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167074+0000", 2024-09-14T05:30:25.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0999999999999997e-07 2024-09-14T05:30:25.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167078+0000", 2024-09-14T05:30:25.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.878e-06 2024-09-14T05:30:25.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167087+0000", 2024-09-14T05:30:25.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6940000000000007e-06 2024-09-14T05:30:25.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167093+0000", 2024-09-14T05:30:25.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7740000000000001e-06 2024-09-14T05:30:25.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167128+0000", 2024-09-14T05:30:25.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5231999999999997e-05 2024-09-14T05:30:25.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167207+0000", 2024-09-14T05:30:25.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9036999999999996e-05 2024-09-14T05:30:25.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:25.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.169140+0000", 2024-09-14T05:30:25.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001932897 2024-09-14T05:30:25.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:25.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178258+0000", 2024-09-14T05:30:25.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091180819999999996 2024-09-14T05:30:25.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:25.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178314+0000", 2024-09-14T05:30:25.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5571e-05 2024-09-14T05:30:25.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:25.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178348+0000", 2024-09-14T05:30:25.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3747000000000001e-05 2024-09-14T05:30:25.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178377+0000", 2024-09-14T05:30:25.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9471e-05 2024-09-14T05:30:25.944 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.944 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.944 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.944 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4721 3.1 3:86302a6e:::rbd_data.112d3d611e54.0000000000000011:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:25.945 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:30:25.945 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 53.661192626000002, 2024-09-14T05:30:25.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066831750000000004, 2024-09-14T05:30:25.945 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.945 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.945 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:25.945 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:25.945 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:25.945 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:25.945 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4721 2024-09-14T05:30:25.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.945 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:30:25.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:30:25.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199059+0000", 2024-09-14T05:30:25.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.06e-07 2024-09-14T05:30:25.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199062+0000", 2024-09-14T05:30:25.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2279999999999998e-06 2024-09-14T05:30:25.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199072+0000", 2024-09-14T05:30:25.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5999999999999996e-06 2024-09-14T05:30:25.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199078+0000", 2024-09-14T05:30:25.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9109999999999998e-06 2024-09-14T05:30:25.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199123+0000", 2024-09-14T05:30:25.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5278000000000003e-05 2024-09-14T05:30:25.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199202+0000", 2024-09-14T05:30:25.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8521000000000003e-05 2024-09-14T05:30:25.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:25.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199271+0000", 2024-09-14T05:30:25.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9857999999999994e-05 2024-09-14T05:30:25.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:25.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199941+0000", 2024-09-14T05:30:25.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066984599999999998 2024-09-14T05:30:25.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:25.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205653+0000", 2024-09-14T05:30:25.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057117319999999997 2024-09-14T05:30:25.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:25.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205701+0000", 2024-09-14T05:30:25.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8254000000000003e-05 2024-09-14T05:30:25.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205741+0000", 2024-09-14T05:30:25.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0241000000000003e-05 2024-09-14T05:30:25.950 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.950 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.950 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.951 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4819 3.12 3:4e8efced:::rbd_data.112d3d611e54.000000000000000e:head [stat,truncate 1114112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:25.951 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:30:25.951 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 53.575878467000003, 2024-09-14T05:30:25.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062422770000000001, 2024-09-14T05:30:25.951 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.951 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.951 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:25.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:25.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:25.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:25.951 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4819 2024-09-14T05:30:25.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.951 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:30:25.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:30:25.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284373+0000", 2024-09-14T05:30:25.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3399999999999998e-07 2024-09-14T05:30:25.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284377+0000", 2024-09-14T05:30:25.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8539999999999999e-06 2024-09-14T05:30:25.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284388+0000", 2024-09-14T05:30:25.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1328999999999999e-05 2024-09-14T05:30:25.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284395+0000", 2024-09-14T05:30:25.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9759999999999998e-06 2024-09-14T05:30:25.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284723+0000", 2024-09-14T05:30:25.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032775799999999999 2024-09-14T05:30:25.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284801+0000", 2024-09-14T05:30:25.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7562e-05 2024-09-14T05:30:25.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:30:25.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284933+0000", 2024-09-14T05:30:25.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000131989 2024-09-14T05:30:25.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:25.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.287450+0000", 2024-09-14T05:30:25.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025177699999999999 2024-09-14T05:30:25.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:25.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290531+0000", 2024-09-14T05:30:25.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0030809959999999999 2024-09-14T05:30:25.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:25.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290588+0000", 2024-09-14T05:30:25.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6177000000000001e-05 2024-09-14T05:30:25.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290615+0000", 2024-09-14T05:30:25.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7132e-05 2024-09-14T05:30:25.956 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.956 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.956 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.957 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:7067 3.5 e35/24 3:a65d0cff:::rbd_data.112df7f2fe0a.0000000000000028:head v 35'89, mlcod=34'86)", 2024-09-14T05:30:25.957 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:25.957 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.717485195999998, 2024-09-14T05:30:25.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0064513080000000002, 2024-09-14T05:30:25.957 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.957 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.957 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:25.957 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:25.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:25.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:25.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:25.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142771+0000", 2024-09-14T05:30:25.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8949999999999997e-06 2024-09-14T05:30:25.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142779+0000", 2024-09-14T05:30:25.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8699999999999992e-06 2024-09-14T05:30:25.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142785+0000", 2024-09-14T05:30:25.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4899999999999997e-06 2024-09-14T05:30:25.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142843+0000", 2024-09-14T05:30:25.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7377000000000003e-05 2024-09-14T05:30:25.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142871+0000", 2024-09-14T05:30:25.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8305000000000001e-05 2024-09-14T05:30:25.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:25.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149177+0000", 2024-09-14T05:30:25.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063058639999999996 2024-09-14T05:30:25.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149217+0000", 2024-09-14T05:30:25.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0080999999999998e-05 2024-09-14T05:30:25.961 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.961 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.961 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.961 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:72582 3.b e47/24 3:d5072567:::rbd_data.112d4a90444c.0000000000000119:head v 47'386, mlcod=46'383)", 2024-09-14T05:30:25.961 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:25.961 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.335224259, 2024-09-14T05:30:25.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0067720940000000002, 2024-09-14T05:30:25.961 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.962 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.962 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:25.962 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:25.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:25.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:25.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1100000000000002e-07 2024-09-14T05:30:25.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525031+0000", 2024-09-14T05:30:25.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0740000000000003e-06 2024-09-14T05:30:25.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525040+0000", 2024-09-14T05:30:25.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5329999999999994e-06 2024-09-14T05:30:25.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525045+0000", 2024-09-14T05:30:25.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:30:25.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525090+0000", 2024-09-14T05:30:25.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4743000000000002e-05 2024-09-14T05:30:25.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525102+0000", 2024-09-14T05:30:25.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2067e-05 2024-09-14T05:30:25.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:25.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531757+0000", 2024-09-14T05:30:25.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066550380000000003 2024-09-14T05:30:25.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531799+0000", 2024-09-14T05:30:25.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1640000000000001e-05 2024-09-14T05:30:25.965 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.965 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.965 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.966 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:79671 3.15 e47/24 3:ab572765:::rbd_data.112d956b542f.0000000000000013:head v 47'120, mlcod=47'118)", 2024-09-14T05:30:25.966 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:30:25.966 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 35.709244040999998, 2024-09-14T05:30:25.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066700609999999997, 2024-09-14T05:30:25.966 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.966 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.966 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:25.966 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:30:25.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:30:25.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151008+0000", 2024-09-14T05:30:25.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:30:25.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151264+0000", 2024-09-14T05:30:25.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025656099999999999 2024-09-14T05:30:25.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151278+0000", 2024-09-14T05:30:25.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3288999999999999e-05 2024-09-14T05:30:25.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151287+0000", 2024-09-14T05:30:25.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7909999999999993e-06 2024-09-14T05:30:25.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151361+0000", 2024-09-14T05:30:25.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3979000000000004e-05 2024-09-14T05:30:25.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151383+0000", 2024-09-14T05:30:25.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2039999999999999e-05 2024-09-14T05:30:25.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:25.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.157542+0000", 2024-09-14T05:30:25.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006158915 2024-09-14T05:30:25.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.157677+0000", 2024-09-14T05:30:25.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013544300000000001 2024-09-14T05:30:25.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.970 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.970 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:123378 3.4 e71/39 3:24969376:::rbd_data.112d471e22e6.0000000000000009:head v 71'440, mlcod=69'438)", 2024-09-14T05:30:25.971 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:17.791627+0000", 2024-09-14T05:30:25.971 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0686243290000004, 2024-09-14T05:30:25.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066686189999999998, 2024-09-14T05:30:25.971 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:25.971 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:25.971 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:25.971 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:25.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:25.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.791627+0000", 2024-09-14T05:30:25.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:25.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.791627+0000", 2024-09-14T05:30:25.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:25.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:25.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.791627+0000", 2024-09-14T05:30:25.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.65e-07 2024-09-14T05:30:25.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:25.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792806+0000", 2024-09-14T05:30:25.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001179193 2024-09-14T05:30:25.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:25.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792832+0000", 2024-09-14T05:30:25.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5514999999999999e-05 2024-09-14T05:30:25.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:25.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792842+0000", 2024-09-14T05:30:25.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0481e-05 2024-09-14T05:30:25.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:25.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792919+0000", 2024-09-14T05:30:25.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6720999999999999e-05 2024-09-14T05:30:25.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:25.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792945+0000", 2024-09-14T05:30:25.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6367999999999999e-05 2024-09-14T05:30:25.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:25.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.798239+0000", 2024-09-14T05:30:25.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052942509999999998 2024-09-14T05:30:25.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:25.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:25.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:25.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.798295+0000", 2024-09-14T05:30:25.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5825000000000002e-05 2024-09-14T05:30:25.975 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.975 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.975 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.975 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:25.975 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:25.975 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:25.977 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:30:25.977 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:25.976+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:26.077 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:26.077+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:26.103 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T05:30:26.103 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T05:30:26.103 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T05:30:26.103 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T05:30:26.121 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:26.121 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:26.121 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:26.121 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:26.129 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:30:26.179 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:26.178+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:26.274 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:26.274 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:26.274 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:26.275 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:26.275 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:26.280 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:26.279+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:26.283 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:30:26.346 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-14T05:30:26.379 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:26.379+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:26.400 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:26.400 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:26.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:26.400 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:26.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.401 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2131 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:26.401 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:26.401 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.622139306999998, 2024-09-14T05:30:26.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049526593000000001, 2024-09-14T05:30:26.401 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.401 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.401 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:26.401 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.401 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.402 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2131 2024-09-14T05:30:26.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.402 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:26.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:26.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774867+0000", 2024-09-14T05:30:26.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4430000000000002e-06 2024-09-14T05:30:26.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790357+0000", 2024-09-14T05:30:26.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015489466 2024-09-14T05:30:26.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790383+0000", 2024-09-14T05:30:26.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6247000000000001e-05 2024-09-14T05:30:26.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790394+0000", 2024-09-14T05:30:26.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0728e-05 2024-09-14T05:30:26.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790471+0000", 2024-09-14T05:30:26.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7539000000000004e-05 2024-09-14T05:30:26.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790639+0000", 2024-09-14T05:30:26.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016787700000000001 2024-09-14T05:30:26.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-14T05:30:26.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790998+0000", 2024-09-14T05:30:26.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035931299999999999 2024-09-14T05:30:26.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:26.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.823529+0000", 2024-09-14T05:30:26.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032530798 2024-09-14T05:30:26.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:26.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824324+0000", 2024-09-14T05:30:26.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00079442399999999995 2024-09-14T05:30:26.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:26.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824362+0000", 2024-09-14T05:30:26.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8398000000000001e-05 2024-09-14T05:30:26.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824391+0000", 2024-09-14T05:30:26.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.936e-05 2024-09-14T05:30:26.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.408 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.408 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3892 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7602176~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.408 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:26.408 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.040561308999997, 2024-09-14T05:30:26.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047082951999999997, 2024-09-14T05:30:26.408 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.409 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.409 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.409 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3892 2024-09-14T05:30:26.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.409 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:26.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:26.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:26.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0199999999999998e-07 2024-09-14T05:30:26.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356454+0000", 2024-09-14T05:30:26.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0414999999999999e-05 2024-09-14T05:30:26.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356463+0000", 2024-09-14T05:30:26.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6739999999999992e-06 2024-09-14T05:30:26.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356468+0000", 2024-09-14T05:30:26.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5680000000000001e-06 2024-09-14T05:30:26.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403143+0000", 2024-09-14T05:30:26.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046675051000000002 2024-09-14T05:30:26.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403206+0000", 2024-09-14T05:30:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3127999999999996e-05 2024-09-14T05:30:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403526+0000", 2024-09-14T05:30:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031981400000000002 2024-09-14T05:30:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.414 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3893 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8093696~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.414 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:26.414 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.040516472, 2024-09-14T05:30:26.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047516305000000002, 2024-09-14T05:30:26.414 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.414 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.414 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.414 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3893 2024-09-14T05:30:26.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.414 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:26.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:26.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:26.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6899999999999997e-07 2024-09-14T05:30:26.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356491+0000", 2024-09-14T05:30:26.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8109999999999999e-06 2024-09-14T05:30:26.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356498+0000", 2024-09-14T05:30:26.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5320000000000003e-06 2024-09-14T05:30:26.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356502+0000", 2024-09-14T05:30:26.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000002e-06 2024-09-14T05:30:26.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403579+0000", 2024-09-14T05:30:26.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047076706000000003 2024-09-14T05:30:26.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403664+0000", 2024-09-14T05:30:26.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5240000000000004e-05 2024-09-14T05:30:26.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404004+0000", 2024-09-14T05:30:26.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033974500000000002 2024-09-14T05:30:26.418 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.418 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.418 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.418 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3993 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 262144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.418 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:26.418 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.040494600999999, 2024-09-14T05:30:26.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048088476999999998, 2024-09-14T05:30:26.418 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.419 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.419 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.419 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3993 2024-09-14T05:30:26.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.419 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:26.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:26.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:26.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.46e-07 2024-09-14T05:30:26.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356512+0000", 2024-09-14T05:30:26.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.827e-06 2024-09-14T05:30:26.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356518+0000", 2024-09-14T05:30:26.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7780000000000004e-06 2024-09-14T05:30:26.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356522+0000", 2024-09-14T05:30:26.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.865e-06 2024-09-14T05:30:26.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404036+0000", 2024-09-14T05:30:26.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047513725999999999 2024-09-14T05:30:26.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404135+0000", 2024-09-14T05:30:26.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9352999999999996e-05 2024-09-14T05:30:26.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404598+0000", 2024-09-14T05:30:26.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046258200000000001 2024-09-14T05:30:26.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.423 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.423 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3994 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.424 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:26.424 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.040452588999997, 2024-09-14T05:30:26.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048274625000000002, 2024-09-14T05:30:26.424 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.424 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.424 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.424 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3994 2024-09-14T05:30:26.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.424 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:26.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:26.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:26.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3799999999999997e-07 2024-09-14T05:30:26.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356554+0000", 2024-09-14T05:30:26.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2259999999999999e-06 2024-09-14T05:30:26.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356560+0000", 2024-09-14T05:30:26.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-14T05:30:26.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356566+0000", 2024-09-14T05:30:26.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3909999999999997e-06 2024-09-14T05:30:26.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404632+0000", 2024-09-14T05:30:26.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048066154 2024-09-14T05:30:26.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404719+0000", 2024-09-14T05:30:26.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7440000000000003e-05 2024-09-14T05:30:26.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404826+0000", 2024-09-14T05:30:26.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106813 2024-09-14T05:30:26.428 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.428 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.428 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.429 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3995 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.429 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:26.429 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.040427450000003, 2024-09-14T05:30:26.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048828209999999997, 2024-09-14T05:30:26.429 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.429 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.429 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.430 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.430 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.430 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3995 2024-09-14T05:30:26.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.430 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:26.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:26.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:26.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6599999999999996e-07 2024-09-14T05:30:26.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356579+0000", 2024-09-14T05:30:26.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0719999999999998e-06 2024-09-14T05:30:26.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356585+0000", 2024-09-14T05:30:26.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.959e-06 2024-09-14T05:30:26.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356590+0000", 2024-09-14T05:30:26.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.464e-06 2024-09-14T05:30:26.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404875+0000", 2024-09-14T05:30:26.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048285649999999999 2024-09-14T05:30:26.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404985+0000", 2024-09-14T05:30:26.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109417 2024-09-14T05:30:26.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405405+0000", 2024-09-14T05:30:26.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042008200000000001 2024-09-14T05:30:26.433 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.434 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.434 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.434 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3996 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.434 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:26.434 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.040118567, 2024-09-14T05:30:26.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049061595, 2024-09-14T05:30:26.434 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.435 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.435 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.435 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.435 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.435 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.435 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3996 2024-09-14T05:30:26.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.435 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:26.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:26.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:26.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.61e-07 2024-09-14T05:30:26.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356892+0000", 2024-09-14T05:30:26.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9050000000000002e-06 2024-09-14T05:30:26.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356901+0000", 2024-09-14T05:30:26.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4149999999999999e-06 2024-09-14T05:30:26.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356907+0000", 2024-09-14T05:30:26.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0140000000000004e-06 2024-09-14T05:30:26.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405448+0000", 2024-09-14T05:30:26.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048541647 2024-09-14T05:30:26.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405542+0000", 2024-09-14T05:30:26.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3639000000000003e-05 2024-09-14T05:30:26.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405947+0000", 2024-09-14T05:30:26.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040521399999999999 2024-09-14T05:30:26.439 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.439 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.440 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.440 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3997 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2228224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.440 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:26.440 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.039904585000002, 2024-09-14T05:30:26.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049308774999999999, 2024-09-14T05:30:26.440 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.440 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.440 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.441 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3997 2024-09-14T05:30:26.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.441 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:26.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:26.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:26.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6499999999999999e-07 2024-09-14T05:30:26.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357104+0000", 2024-09-14T05:30:26.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4989999999999999e-06 2024-09-14T05:30:26.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357112+0000", 2024-09-14T05:30:26.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4279999999999996e-06 2024-09-14T05:30:26.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357117+0000", 2024-09-14T05:30:26.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6190000000000002e-06 2024-09-14T05:30:26.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405989+0000", 2024-09-14T05:30:26.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048872080999999998 2024-09-14T05:30:26.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406074+0000", 2024-09-14T05:30:26.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4676e-05 2024-09-14T05:30:26.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406408+0000", 2024-09-14T05:30:26.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033480700000000001 2024-09-14T05:30:26.445 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.445 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.445 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.446 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3998 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.446 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:26.446 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.039583155999999, 2024-09-14T05:30:26.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049462314, 2024-09-14T05:30:26.446 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.446 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.447 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.447 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3998 2024-09-14T05:30:26.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.447 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:26.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:26.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357422+0000", 2024-09-14T05:30:26.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3099999999999996e-07 2024-09-14T05:30:26.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357429+0000", 2024-09-14T05:30:26.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2880000000000004e-06 2024-09-14T05:30:26.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357439+0000", 2024-09-14T05:30:26.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.302e-06 2024-09-14T05:30:26.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357445+0000", 2024-09-14T05:30:26.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4269999999999996e-06 2024-09-14T05:30:26.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406448+0000", 2024-09-14T05:30:26.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049003096000000003 2024-09-14T05:30:26.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406537+0000", 2024-09-14T05:30:26.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9135999999999995e-05 2024-09-14T05:30:26.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406883+0000", 2024-09-14T05:30:26.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000346134 2024-09-14T05:30:26.451 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.451 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.451 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.451 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3999 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.451 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:26.452 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.039512799000001, 2024-09-14T05:30:26.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049797556, 2024-09-14T05:30:26.452 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.452 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.452 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.452 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.452 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.452 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.452 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3999 2024-09-14T05:30:26.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.452 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:26.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:26.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357492+0000", 2024-09-14T05:30:26.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8299999999999996e-07 2024-09-14T05:30:26.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357495+0000", 2024-09-14T05:30:26.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0450000000000001e-06 2024-09-14T05:30:26.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357503+0000", 2024-09-14T05:30:26.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9689999999999994e-06 2024-09-14T05:30:26.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357510+0000", 2024-09-14T05:30:26.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-14T05:30:26.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406914+0000", 2024-09-14T05:30:26.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049404293000000002 2024-09-14T05:30:26.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406965+0000", 2024-09-14T05:30:26.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0470999999999998e-05 2024-09-14T05:30:26.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407289+0000", 2024-09-14T05:30:26.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032434899999999998 2024-09-14T05:30:26.457 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.457 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.457 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.457 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4000 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3702784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.457 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:26.457 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.039166453999997, 2024-09-14T05:30:26.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049936728999999999, 2024-09-14T05:30:26.458 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.458 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.458 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.458 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.458 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.458 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.458 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4000 2024-09-14T05:30:26.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.458 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:26.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:26.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357839+0000", 2024-09-14T05:30:26.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-14T05:30:26.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357842+0000", 2024-09-14T05:30:26.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4850000000000001e-06 2024-09-14T05:30:26.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357851+0000", 2024-09-14T05:30:26.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1819999999999995e-06 2024-09-14T05:30:26.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357859+0000", 2024-09-14T05:30:26.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8129999999999996e-06 2024-09-14T05:30:26.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407337+0000", 2024-09-14T05:30:26.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049478084999999998 2024-09-14T05:30:26.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407442+0000", 2024-09-14T05:30:26.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000105302 2024-09-14T05:30:26.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407774+0000", 2024-09-14T05:30:26.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033200100000000001 2024-09-14T05:30:26.463 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.463 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.463 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.463 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4001 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.463 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:26.463 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.039071098999997, 2024-09-14T05:30:26.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050304205999999997, 2024-09-14T05:30:26.463 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.464 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.464 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.464 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4001 2024-09-14T05:30:26.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.464 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:26.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:26.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357934+0000", 2024-09-14T05:30:26.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000004e-07 2024-09-14T05:30:26.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357937+0000", 2024-09-14T05:30:26.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0869999999999998e-06 2024-09-14T05:30:26.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357946+0000", 2024-09-14T05:30:26.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1439999999999992e-06 2024-09-14T05:30:26.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357952+0000", 2024-09-14T05:30:26.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.609e-06 2024-09-14T05:30:26.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407820+0000", 2024-09-14T05:30:26.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049868580000000003 2024-09-14T05:30:26.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407909+0000", 2024-09-14T05:30:26.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8887000000000005e-05 2024-09-14T05:30:26.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408237+0000", 2024-09-14T05:30:26.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032824599999999997 2024-09-14T05:30:26.467 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.467 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.467 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.468 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4002 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.468 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:26.468 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.038854504, 2024-09-14T05:30:26.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050539529999999999, 2024-09-14T05:30:26.468 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.468 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.468 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.468 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.469 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4002 2024-09-14T05:30:26.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.469 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:26.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:26.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:26.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5099999999999999e-07 2024-09-14T05:30:26.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358154+0000", 2024-09-14T05:30:26.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6799999999999999e-06 2024-09-14T05:30:26.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358165+0000", 2024-09-14T05:30:26.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0899e-05 2024-09-14T05:30:26.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358172+0000", 2024-09-14T05:30:26.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8689999999999998e-06 2024-09-14T05:30:26.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408281+0000", 2024-09-14T05:30:26.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050108699 2024-09-14T05:30:26.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408369+0000", 2024-09-14T05:30:26.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8542999999999996e-05 2024-09-14T05:30:26.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408689+0000", 2024-09-14T05:30:26.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032008900000000002 2024-09-14T05:30:26.473 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.473 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.473 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.473 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4003 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5177344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.473 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:26.473 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.038814668999997, 2024-09-14T05:30:26.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050868759, 2024-09-14T05:30:26.473 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.473 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.474 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.474 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.474 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.474 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.474 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4003 2024-09-14T05:30:26.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.474 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:26.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:26.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:26.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-07 2024-09-14T05:30:26.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358193+0000", 2024-09-14T05:30:26.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.588e-06 2024-09-14T05:30:26.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358204+0000", 2024-09-14T05:30:26.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1525999999999999e-05 2024-09-14T05:30:26.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358213+0000", 2024-09-14T05:30:26.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5010000000000004e-06 2024-09-14T05:30:26.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408719+0000", 2024-09-14T05:30:26.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050506708999999997 2024-09-14T05:30:26.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408766+0000", 2024-09-14T05:30:26.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6579999999999998e-05 2024-09-14T05:30:26.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409058+0000", 2024-09-14T05:30:26.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000292365 2024-09-14T05:30:26.477 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.477 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.477 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.478 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4004 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5668864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.478 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:26.478 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.038391826000002, 2024-09-14T05:30:26.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050839848, 2024-09-14T05:30:26.478 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.478 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.478 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.478 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.478 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.479 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.479 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4004 2024-09-14T05:30:26.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.479 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:26.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:26.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358613+0000", 2024-09-14T05:30:26.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:30:26.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358618+0000", 2024-09-14T05:30:26.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6009999999999997e-06 2024-09-14T05:30:26.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358630+0000", 2024-09-14T05:30:26.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2122e-05 2024-09-14T05:30:26.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358638+0000", 2024-09-14T05:30:26.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3969999999999994e-06 2024-09-14T05:30:26.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409088+0000", 2024-09-14T05:30:26.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050449565000000002 2024-09-14T05:30:26.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409153+0000", 2024-09-14T05:30:26.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4728999999999996e-05 2024-09-14T05:30:26.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409452+0000", 2024-09-14T05:30:26.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029958900000000001 2024-09-14T05:30:26.483 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.483 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.483 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.483 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4005 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6160384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.483 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:26.483 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.038174892000001, 2024-09-14T05:30:26.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051054752000000002, 2024-09-14T05:30:26.483 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.483 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.483 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.484 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.484 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.484 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.484 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4005 2024-09-14T05:30:26.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.484 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:26.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:26.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358830+0000", 2024-09-14T05:30:26.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-14T05:30:26.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358833+0000", 2024-09-14T05:30:26.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9529999999999999e-06 2024-09-14T05:30:26.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358843+0000", 2024-09-14T05:30:26.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8069999999999994e-06 2024-09-14T05:30:26.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358849+0000", 2024-09-14T05:30:26.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1040000000000003e-06 2024-09-14T05:30:26.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409487+0000", 2024-09-14T05:30:26.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050638626999999999 2024-09-14T05:30:26.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409535+0000", 2024-09-14T05:30:26.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7525000000000003e-05 2024-09-14T05:30:26.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409884+0000", 2024-09-14T05:30:26.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034914800000000003 2024-09-14T05:30:26.487 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.487 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.487 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.488 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4006 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.488 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:26.488 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.03813778, 2024-09-14T05:30:26.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051425323000000002, 2024-09-14T05:30:26.488 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.488 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.488 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.488 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4006 2024-09-14T05:30:26.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.488 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:26.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:26.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358867+0000", 2024-09-14T05:30:26.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2900000000000004e-07 2024-09-14T05:30:26.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358869+0000", 2024-09-14T05:30:26.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4210000000000002e-06 2024-09-14T05:30:26.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358876+0000", 2024-09-14T05:30:26.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4980000000000003e-06 2024-09-14T05:30:26.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358881+0000", 2024-09-14T05:30:26.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3090000000000003e-06 2024-09-14T05:30:26.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409915+0000", 2024-09-14T05:30:26.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051033348999999999 2024-09-14T05:30:26.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409964+0000", 2024-09-14T05:30:26.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9858000000000003e-05 2024-09-14T05:30:26.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410292+0000", 2024-09-14T05:30:26.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032735899999999999 2024-09-14T05:30:26.492 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.492 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.492 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.493 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4007 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.493 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:26.493 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.038115187000002, 2024-09-14T05:30:26.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05184076, 2024-09-14T05:30:26.493 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.493 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.493 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.493 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4007 2024-09-14T05:30:26.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.493 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:26.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:26.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:26.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:30:26.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358891+0000", 2024-09-14T05:30:26.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9819999999999998e-06 2024-09-14T05:30:26.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358897+0000", 2024-09-14T05:30:26.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9390000000000002e-06 2024-09-14T05:30:26.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358902+0000", 2024-09-14T05:30:26.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6770000000000003e-06 2024-09-14T05:30:26.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410335+0000", 2024-09-14T05:30:26.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051432991999999997 2024-09-14T05:30:26.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410416+0000", 2024-09-14T05:30:26.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1316999999999994e-05 2024-09-14T05:30:26.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410730+0000", 2024-09-14T05:30:26.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000313419 2024-09-14T05:30:26.497 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.497 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.497 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.497 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4008 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7634944~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.497 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:26.497 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.038090826000001, 2024-09-14T05:30:26.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052208082000000003, 2024-09-14T05:30:26.497 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.498 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.498 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.498 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4008 2024-09-14T05:30:26.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.498 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:26.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:26.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358914+0000", 2024-09-14T05:30:26.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6100000000000001e-07 2024-09-14T05:30:26.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358916+0000", 2024-09-14T05:30:26.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.902e-06 2024-09-14T05:30:26.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358923+0000", 2024-09-14T05:30:26.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4780000000000002e-06 2024-09-14T05:30:26.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358929+0000", 2024-09-14T05:30:26.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0970000000000002e-06 2024-09-14T05:30:26.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410761+0000", 2024-09-14T05:30:26.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051831436000000002 2024-09-14T05:30:26.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410829+0000", 2024-09-14T05:30:26.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7874999999999996e-05 2024-09-14T05:30:26.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411121+0000", 2024-09-14T05:30:26.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029283299999999999 2024-09-14T05:30:26.501 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.501 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.501 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.502 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4009 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8126464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:26.502 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:26.502 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.037943601000002, 2024-09-14T05:30:26.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052518169000000003, 2024-09-14T05:30:26.502 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:26.502 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:26.502 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:26.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:26.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:26.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.502 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4009 2024-09-14T05:30:26.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.503 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:26.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:26.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:26.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:26.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:26.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:26.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:26.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:26.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0200000000000002e-07 2024-09-14T05:30:26.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:26.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359066+0000", 2024-09-14T05:30:26.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7899999999999999e-06 2024-09-14T05:30:26.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:26.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359074+0000", 2024-09-14T05:30:26.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2579999999999995e-06 2024-09-14T05:30:26.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359082+0000", 2024-09-14T05:30:26.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.464e-06 2024-09-14T05:30:26.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:26.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411160+0000", 2024-09-14T05:30:26.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052078078 2024-09-14T05:30:26.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:26.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411244+0000", 2024-09-14T05:30:26.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4041000000000004e-05 2024-09-14T05:30:26.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:26.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:26.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:26.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411579+0000", 2024-09-14T05:30:26.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033493599999999998 2024-09-14T05:30:26.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.506 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:26.506 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:26.506 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:26.509 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:30:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:30:26.509 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 77, 2024-09-14T05:30:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "1127be44-5f14-4e7d-ad23-f608afe154f5", 2024-09-14T05:30:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2024-09-14T05:28:54.203072+0000", 2024-09-14T05:30:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-14T05:30:26.344954+0000", 2024-09-14T05:30:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2024-09-14T05:29:06.755531+0000", 2024-09-14T05:30:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2024-09-14T05:28:56.535048+0000", 2024-09-14T05:30:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T05:30:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5806080, 2024-09-14T05:30:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2024-09-14T05:30:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "nodeep-scrub", 2024-09-14T05:30:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "noscrub", 2024-09-14T05:30:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2024-09-14T05:30:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2024-09-14T05:30:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2024-09-14T05:30:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2024-09-14T05:30:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:30:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 6, 2024-09-14T05:30:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T05:30:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T05:30:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T05:30:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2024-09-14T05:30:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 3, 2024-09-14T05:30:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2024-09-14T05:30:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2024-09-14T05:30:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2024-09-14T05:30:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "squid", 2024-09-14T05:30:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2024-09-14T05:30:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2024-09-14T05:30:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2024-09-14T05:30:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2024-09-14T05:30:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-14T05:29:06.452744+0000", 2024-09-14T05:30:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-14T05:30:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2024-09-14T05:30:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-14T05:30:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-14T05:30:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-14T05:30:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-14T05:30:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-14T05:30:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-14T05:30:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T05:30:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T05:30:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-14T05:30:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-14T05:30:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-14T05:30:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2024-09-14T05:30:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2024-09-14T05:30:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2024-09-14T05:30:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2024-09-14T05:30:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2024-09-14T05:30:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-14T05:30:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-14T05:30:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-14T05:30:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-14T05:30:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-14T05:30:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-14T05:30:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-14T05:30:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "18", 2024-09-14T05:30:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-14T05:30:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T05:30:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T05:30:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-14T05:30:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-14T05:30:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2024-09-14T05:30:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2024-09-14T05:30:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-14T05:30:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-14T05:30:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-14T05:30:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-14T05:30:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-14T05:30:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-14T05:30:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-14T05:30:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-14T05:30:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-14T05:30:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-14T05:30:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-14T05:30:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T05:30:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T05:30:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T05:30:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-14T05:30:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-14T05:30:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-14T05:30:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-14T05:30:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-14T05:30:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-14T05:30:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-14T05:30:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-14T05:30:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-14T05:30:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-14T05:30:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T05:30:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-14T05:30:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-14T05:30:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-14T05:30:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-14T05:30:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-14T05:30:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2024-09-14T05:30:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2024-09-14T05:30:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2024-09-14T05:30:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-14T05:30:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2024-09-14T05:30:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-14T05:30:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-14T05:30:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2024-09-14T05:30:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2024-09-14T05:30:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2024-09-14T05:30:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2024-09-14T05:30:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2024-09-14T05:30:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.87708091735839844, 2024-09-14T05:30:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T05:30:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T05:30:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 2, 2024-09-14T05:30:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "rbd", 2024-09-14T05:30:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-14T05:29:12.380843+0000", 2024-09-14T05:30:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 8193, 2024-09-14T05:30:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T05:30:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-14T05:30:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-14T05:30:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-14T05:30:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-14T05:30:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-14T05:30:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-14T05:30:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T05:30:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T05:30:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-14T05:30:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-14T05:30:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-14T05:30:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 8, 2024-09-14T05:30:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 8, 2024-09-14T05:30:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 8, 2024-09-14T05:30:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 8, 2024-09-14T05:30:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 8, 2024-09-14T05:30:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-14T05:30:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-14T05:30:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-14T05:30:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-14T05:30:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-14T05:30:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-14T05:30:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-14T05:30:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "22", 2024-09-14T05:30:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-14T05:30:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T05:30:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T05:30:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-14T05:30:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-14T05:30:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 2, 2024-09-14T05:30:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 22, 2024-09-14T05:30:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-14T05:30:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-14T05:30:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-14T05:30:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-14T05:30:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-14T05:30:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-14T05:30:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-14T05:30:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-14T05:30:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-14T05:30:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-14T05:30:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-14T05:30:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T05:30:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T05:30:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T05:30:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-14T05:30:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-14T05:30:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-14T05:30:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-14T05:30:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-14T05:30:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-14T05:30:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-14T05:30:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-14T05:30:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-14T05:30:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-14T05:30:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T05:30:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-14T05:30:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-14T05:30:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-14T05:30:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-14T05:30:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-14T05:30:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "options": {}, 2024-09-14T05:30:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-14T05:30:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd": {} 2024-09-14T05:30:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-14T05:30:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-14T05:30:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 4.9800000190734863, 2024-09-14T05:30:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 4.9800000190734863, 2024-09-14T05:30:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T05:30:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 4.380000114440918, 2024-09-14T05:30:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 4.380000114440918, 2024-09-14T05:30:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.87708091735839844, 2024-09-14T05:30:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T05:30:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T05:30:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 3, 2024-09-14T05:30:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "pool_client.0", 2024-09-14T05:30:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-14T05:29:22.335419+0000", 2024-09-14T05:30:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 8193, 2024-09-14T05:30:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T05:30:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-14T05:30:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-14T05:30:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-14T05:30:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-14T05:30:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-14T05:30:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-14T05:30:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T05:30:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T05:30:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-14T05:30:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-14T05:30:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-14T05:30:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2024-09-14T05:30:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2024-09-14T05:30:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2024-09-14T05:30:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2024-09-14T05:30:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2024-09-14T05:30:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-14T05:30:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-14T05:30:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-14T05:30:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-14T05:30:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-14T05:30:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-14T05:30:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-14T05:30:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "77", 2024-09-14T05:30:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-14T05:30:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T05:30:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T05:30:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-14T05:30:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-14T05:30:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 40, 2024-09-14T05:30:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 77, 2024-09-14T05:30:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-14T05:30:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-14T05:30:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-14T05:30:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-14T05:30:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-14T05:30:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-14T05:30:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-14T05:30:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-14T05:30:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-14T05:30:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-14T05:30:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-14T05:30:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T05:30:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T05:30:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T05:30:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-14T05:30:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-14T05:30:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-14T05:30:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-14T05:30:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-14T05:30:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-14T05:30:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-14T05:30:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-14T05:30:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-14T05:30:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-14T05:30:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T05:30:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-14T05:30:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-14T05:30:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-14T05:30:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-14T05:30:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-14T05:30:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "options": {}, 2024-09-14T05:30:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-14T05:30:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd": {} 2024-09-14T05:30:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-14T05:30:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-14T05:30:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2024-09-14T05:30:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2024-09-14T05:30:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2024-09-14T05:30:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2024-09-14T05:30:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2024-09-14T05:30:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.87708091735839844, 2024-09-14T05:30:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T05:30:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T05:30:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:30:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2024-09-14T05:30:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2024-09-14T05:30:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "e7c7817f-4a65-4f66-9af5-d31d0a92cf30", 2024-09-14T05:30:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:30:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:30:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:30:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:30:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:30:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:30:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 16, 2024-09-14T05:30:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:30:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:30:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:30:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:30:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2024-09-14T05:30:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:30:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2024-09-14T05:30:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:30:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:30:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2024-09-14T05:30:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:30:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2024-09-14T05:30:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:30:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:30:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2024-09-14T05:30:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:30:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2024-09-14T05:30:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:30:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:30:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2024-09-14T05:30:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:30:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2024-09-14T05:30:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:30:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6819/2926109885", 2024-09-14T05:30:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6821/2926109885", 2024-09-14T05:30:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6825/2926109885", 2024-09-14T05:30:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6823/2926109885", 2024-09-14T05:30:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:30:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:30:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:30:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2024-09-14T05:30:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "e184604e-b973-4b7a-bba2-6f471753a0fd", 2024-09-14T05:30:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:30:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:30:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:30:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:30:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:30:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:30:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:30:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:30:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:30:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:30:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:30:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2024-09-14T05:30:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:30:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2024-09-14T05:30:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:30:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:30:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2024-09-14T05:30:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:30:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2024-09-14T05:30:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:30:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:30:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6832", 2024-09-14T05:30:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:30:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6833", 2024-09-14T05:30:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:30:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:30:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2024-09-14T05:30:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:30:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2024-09-14T05:30:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:30:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6827/345649648", 2024-09-14T05:30:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6829/345649648", 2024-09-14T05:30:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6833/345649648", 2024-09-14T05:30:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6831/345649648", 2024-09-14T05:30:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:30:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:30:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:30:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2024-09-14T05:30:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "d8b70f09-5afb-4296-b8c9-719c172e0e70", 2024-09-14T05:30:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:30:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:30:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:30:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:30:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:30:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:30:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:30:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 39, 2024-09-14T05:30:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:30:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:30:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:30:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2024-09-14T05:30:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:30:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2024-09-14T05:30:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:30:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:30:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2024-09-14T05:30:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:30:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2024-09-14T05:30:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:30:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:30:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2024-09-14T05:30:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:30:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2024-09-14T05:30:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:30:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:30:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2024-09-14T05:30:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:30:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2024-09-14T05:30:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:30:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/2721860691", 2024-09-14T05:30:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/2721860691", 2024-09-14T05:30:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/2721860691", 2024-09-14T05:30:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/2721860691", 2024-09-14T05:30:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:30:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:30:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:30:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2024-09-14T05:30:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c", 2024-09-14T05:30:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:30:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:30:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:30:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:30:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:30:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:30:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:30:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 39, 2024-09-14T05:30:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:30:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:30:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:30:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2024-09-14T05:30:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:30:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2024-09-14T05:30:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:30:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:30:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2024-09-14T05:30:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:30:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2024-09-14T05:30:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:30:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:30:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2024-09-14T05:30:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:30:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2024-09-14T05:30:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:30:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:30:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2024-09-14T05:30:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:30:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2024-09-14T05:30:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:30:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/494607463", 2024-09-14T05:30:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6811/494607463", 2024-09-14T05:30:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6815/494607463", 2024-09-14T05:30:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6813/494607463", 2024-09-14T05:30:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:30:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:30:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:30:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2024-09-14T05:30:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "515cb7d6-16eb-4aff-973a-7f7ed49325ba", 2024-09-14T05:30:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:30:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:30:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:30:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:30:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:30:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:30:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:30:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:30:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:30:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:30:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:30:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6816", 2024-09-14T05:30:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:30:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6817", 2024-09-14T05:30:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:30:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:30:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6818", 2024-09-14T05:30:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:30:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6819", 2024-09-14T05:30:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:30:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:30:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6822", 2024-09-14T05:30:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:30:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6823", 2024-09-14T05:30:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:30:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:30:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6820", 2024-09-14T05:30:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:30:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6821", 2024-09-14T05:30:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:30:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6817/2240133417", 2024-09-14T05:30:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6819/2240133417", 2024-09-14T05:30:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6823/2240133417", 2024-09-14T05:30:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6821/2240133417", 2024-09-14T05:30:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:30:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:30:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:30:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2024-09-14T05:30:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "c434b53a-04be-45b9-b89a-193654133950", 2024-09-14T05:30:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:30:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:30:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:30:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:30:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:30:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:30:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:30:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 39, 2024-09-14T05:30:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:30:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:30:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:30:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6808", 2024-09-14T05:30:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:30:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6809", 2024-09-14T05:30:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:30:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:30:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6810", 2024-09-14T05:30:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:30:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6811", 2024-09-14T05:30:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:30:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:30:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.568 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6814", 2024-09-14T05:30:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:30:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6815", 2024-09-14T05:30:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:30:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:30:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6812", 2024-09-14T05:30:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:30:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6813", 2024-09-14T05:30:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:30:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6809/2393825223", 2024-09-14T05:30:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6811/2393825223", 2024-09-14T05:30:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6815/2393825223", 2024-09-14T05:30:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6813/2393825223", 2024-09-14T05:30:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:30:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:30:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:30:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2024-09-14T05:30:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "3d2f8405-9286-4b40-8acd-036f67373ac6", 2024-09-14T05:30:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:30:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:30:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:30:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 0.0166473388671875, 2024-09-14T05:30:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:30:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:30:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:30:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:30:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:30:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:30:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:30:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6800", 2024-09-14T05:30:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:30:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6801", 2024-09-14T05:30:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:30:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:30:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6802", 2024-09-14T05:30:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:30:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6803", 2024-09-14T05:30:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:30:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:30:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6806", 2024-09-14T05:30:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:30:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6807", 2024-09-14T05:30:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:30:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:30:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6804", 2024-09-14T05:30:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:30:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6805", 2024-09-14T05:30:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:30:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6801/3235729544", 2024-09-14T05:30:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6803/3235729544", 2024-09-14T05:30:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6807/3235729544", 2024-09-14T05:30:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6805/3235729544", 2024-09-14T05:30:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:30:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:30:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:30:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2024-09-14T05:30:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "6724f080-4dc3-465a-939f-b023985bf998", 2024-09-14T05:30:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:30:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:30:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:30:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:30:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:30:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:30:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:30:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:30:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:30:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:30:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:30:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6824", 2024-09-14T05:30:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:30:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6825", 2024-09-14T05:30:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:30:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:30:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6826", 2024-09-14T05:30:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:30:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6827", 2024-09-14T05:30:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:30:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:30:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6830", 2024-09-14T05:30:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:30:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6831", 2024-09-14T05:30:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:30:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:30:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6828", 2024-09-14T05:30:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:30:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6829", 2024-09-14T05:30:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:30:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6825/2381001355", 2024-09-14T05:30:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6827/2381001355", 2024-09-14T05:30:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6831/2381001355", 2024-09-14T05:30:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6829/2381001355", 2024-09-14T05:30:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:30:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:30:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:30:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:30:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2024-09-14T05:30:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2024-09-14T05:30:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:30:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:30:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:30:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:30:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:30:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.548803+0000", 2024-09-14T05:30:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:30:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2024-09-14T05:30:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:30:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:30:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:30:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:30:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:30:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.075023+0000", 2024-09-14T05:30:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:30:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2024-09-14T05:30:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:30:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:30:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:30:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:30:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:30:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.117036+0000", 2024-09-14T05:30:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:30:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2024-09-14T05:30:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:30:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:30:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:30:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:30:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:30:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.332113+0000", 2024-09-14T05:30:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:30:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2024-09-14T05:30:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:30:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:30:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:30:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:30:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:30:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.330371+0000", 2024-09-14T05:30:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:30:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2024-09-14T05:30:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:30:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:30:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:30:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:30:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:30:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:02.753946+0000", 2024-09-14T05:30:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:30:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2024-09-14T05:30:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:30:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:30:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:30:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:30:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:30:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:02.071186+0000", 2024-09-14T05:30:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:30:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2024-09-14T05:30:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:30:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:30:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:30:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:30:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:30:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.378781+0000", 2024-09-14T05:30:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:30:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:30:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2024-09-14T05:30:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2024-09-14T05:30:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2024-09-14T05:30:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2024-09-14T05:30:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2024-09-14T05:30:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2024-09-14T05:30:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2024-09-14T05:30:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2024-09-14T05:30:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2024-09-14T05:30:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2024-09-14T05:30:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2024-09-14T05:30:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2024-09-14T05:30:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2024-09-14T05:30:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2024-09-14T05:30:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [ 2024-09-14T05:30:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 3, 2024-09-14T05:30:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: "snaps": [ 2024-09-14T05:30:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: "begin": 36, 2024-09-14T05:30:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: "length": 5 2024-09-14T05:30:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:30:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [ 2024-09-14T05:30:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 3, 2024-09-14T05:30:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "snaps": [ 2024-09-14T05:30:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "begin": 36, 2024-09-14T05:30:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "length": 1 2024-09-14T05:30:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:30:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2024-09-14T05:30:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2024-09-14T05:30:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2024-09-14T05:30:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2024-09-14T05:30:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2024-09-14T05:30:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2024-09-14T05:30:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2024-09-14T05:30:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2024-09-14T05:30:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2024-09-14T05:30:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.594 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:26.595 INFO:tasks.thrashosds.thrasher:j is {'epoch': 77, 'fsid': '1127be44-5f14-4e7d-ad23-f608afe154f5', 'created': '2024-09-14T05:28:54.203072+0000', 'modified': '2024-09-14T05:30:26.344954+0000', 'last_up_change': '2024-09-14T05:29:06.755531+0000', 'last_in_change': '2024-09-14T05:28:56.535048+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T05:29:06.452744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8770809173583984, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T05:29:12.380843+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.8770809173583984, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T05:29:22.335419+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 40, 'snap_epoch': 77, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8770809173583984, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'e7c7817f-4a65-4f66-9af5-d31d0a92cf30', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6818', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6819', 'nonce': 2926109885}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6820', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6821', 'nonce': 2926109885}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6824', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6825', 'nonce': 2926109885}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6822', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6823', 'nonce': 2926109885}]}, 'public_addr': '172.21.15.71:6819/2926109885', 'cluster_addr': '172.21.15.71:6821/2926109885', 'heartbeat_back_addr': '172.21.15.71:6825/2926109885', 'heartbeat_front_addr': '172.21.15.71:6823/2926109885', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e184604e-b973-4b7a-bba2-6f471753a0fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6826', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6827', 'nonce': 345649648}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6828', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6829', 'nonce': 345649648}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6832', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6833', 'nonce': 345649648}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6830', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6831', 'nonce': 345649648}]}, 'public_addr': '172.21.15.71:6827/345649648', 'cluster_addr': '172.21.15.71:6829/345649648', 'heartbeat_back_addr': '172.21.15.71:6833/345649648', 'heartbeat_front_addr': '172.21.15.71:6831/345649648', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd8b70f09-5afb-4296-b8c9-719c172e0e70', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6800', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6801', 'nonce': 2721860691}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6802', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6803', 'nonce': 2721860691}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6806', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6807', 'nonce': 2721860691}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6804', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6805', 'nonce': 2721860691}]}, 'public_addr': '172.21.15.71:6801/2721860691', 'cluster_addr': '172.21.15.71:6803/2721860691', 'heartbeat_back_addr': '172.21.15.71:6807/2721860691', 'heartbeat_front_addr': '172.21.15.71:6805/2721860691', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6808', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6809', 'nonce': 494607463}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6810', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6811', 'nonce': 494607463}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6814', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6815', 'nonce': 494607463}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6812', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6813', 'nonce': 494607463}]}, 'public_addr': '172.21.15.71:6809/494607463', 'cluster_addr': '172.21.15.71:6811/494607463', 'heartbeat_back_addr': '172.21.15.71:6815/494607463', 'heartbeat_front_addr': '172.21.15.71:6813/494607463', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '515cb7d6-16eb-4aff-973a-7f7ed49325ba', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6816', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6817', 'nonce': 2240133417}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6818', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6819', 'nonce': 2240133417}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6822', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6823', 'nonce': 2240133417}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6820', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6821', 'nonce': 2240133417}]}, 'public_addr': '172.21.15.150:6817/2240133417', 'cluster_addr': '172.21.15.150:6819/2240133417', 'heartbeat_back_addr': '172.21.15.150:6823/2240133417', 'heartbeat_front_addr': '172.21.15.150:6821/2240133417', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'c434b53a-04be-45b9-b89a-193654133950', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6808', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6809', 'nonce': 2393825223}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6810', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6811', 'nonce': 2393825223}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6814', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6815', 'nonce': 2393825223}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6812', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6813', 'nonce': 2393825223}]}, 'public_addr': '172.21.15.150:6809/2393825223', 'cluster_addr': '172.21.15.150:6811/2393825223', 'heartbeat_back_addr': '172.21.15.150:6815/2393825223', 'heartbeat_front_addr': '172.21.15.150:6813/2393825223', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '3d2f8405-9286-4b40-8acd-036f67373ac6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0166473388671875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6800', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6801', 'nonce': 3235729544}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6802', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6803', 'nonce': 3235729544}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6806', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6807', 'nonce': 3235729544}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6804', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6805', 'nonce': 3235729544}]}, 'public_addr': '172.21.15.150:6801/3235729544', 'cluster_addr': '172.21.15.150:6803/3235729544', 'heartbeat_back_addr': '172.21.15.150:6807/3235729544', 'heartbeat_front_addr': '172.21.15.150:6805/3235729544', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '6724f080-4dc3-465a-939f-b023985bf998', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6824', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6825', 'nonce': 2381001355}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6826', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6827', 'nonce': 2381001355}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6830', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6831', 'nonce': 2381001355}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6828', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6829', 'nonce': 2381001355}]}, 'public_addr': '172.21.15.150:6825/2381001355', 'cluster_addr': '172.21.15.150:6827/2381001355', 'heartbeat_back_addr': '172.21.15.150:6831/2381001355', 'heartbeat_front_addr': '172.21.15.150:6829/2381001355', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.548803+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.075023+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.117036+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.332113+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.330371+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:02.753946+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:02.071186+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.378781+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 36, 'length': 5}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 36, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T05:30:26.595 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:30:26.596 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:26.507+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:26.640 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:26.639+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:26.705 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:26.705 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:26.705 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:26.705 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:26.708 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:26.708+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:26.714 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:30:26.809 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:26.808+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:26.858 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:26.858 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:26.858 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:26.858 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:26.858 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:26.867 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:30:26.896 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:30:26.897 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:30:26.909 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:26.909+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:26.912 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-09-14T05:30:26.448681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1261597106,"num_objects":2177,"num_object_clones":597,"num_object_copies":4354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2177,"num_whiteouts":1081,"num_read":170884,"num_read_kb":552006,"num_write":18774,"num_write_kb":280563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15901,"ondisk_log_size":15901,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":620956,"kb_used_data":314232,"kb_used_omap":12,"kb_used_meta":306227,"kb_avail":754353764,"statfs":{"total":773094113280,"available":772458254336,"internally_reserved":0,"allocated":321773568,"data_stored":350304311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":313577008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1631,"num_read_kb":9706,"num_write":342,"num_write_kb":8006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000497"},"pg_stats":[{"pgid":"3.1f","version":"72'633","reported_seq":10350,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.241803+0000","last_change":"2024-09-14T05:29:26.019843+0000","last_active":"2024-09-14T05:30:21.241803+0000","last_peered":"2024-09-14T05:30:21.241803+0000","last_clean":"2024-09-14T05:30:21.241803+0000","last_became_active":"2024-09-14T05:29:24.018892+0000","last_became_peered":"2024-09-14T05:29:24.018892+0000","last_unstale":"2024-09-14T05:30:21.241803+0000","last_undegraded":"2024-09-14T05:30:21.241803+0000","last_fullsized":"2024-09-14T05:30:21.241803+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:12:26.014170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011130109999999999,"stat_sum":{"num_bytes":52444001,"num_objects":77,"num_object_clones":21,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":36,"num_read":11732,"num_read_kb":36940,"num_write":747,"num_write_kb":31060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"72'339","reported_seq":2924,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.193585+0000","last_change":"2024-09-14T05:29:26.019068+0000","last_active":"2024-09-14T05:30:21.193585+0000","last_peered":"2024-09-14T05:30:21.193585+0000","last_clean":"2024-09-14T05:30:21.193585+0000","last_became_active":"2024-09-14T05:29:24.014468+0000","last_became_peered":"2024-09-14T05:29:24.014468+0000","last_unstale":"2024-09-14T05:30:21.193585+0000","last_undegraded":"2024-09-14T05:30:21.193585+0000","last_fullsized":"2024-09-14T05:30:21.193585+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:51:04.264292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000399031,"stat_sum":{"num_bytes":74450643,"num_objects":73,"num_object_clones":22,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":31,"num_read":2764,"num_read_kb":32470,"num_write":372,"num_write_kb":21059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"72'963","reported_seq":9492,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.146383+0000","last_change":"2024-09-14T05:29:26.165031+0000","last_active":"2024-09-14T05:30:21.146383+0000","last_peered":"2024-09-14T05:30:21.146383+0000","last_clean":"2024-09-14T05:30:21.146383+0000","last_became_active":"2024-09-14T05:29:24.014251+0000","last_became_peered":"2024-09-14T05:29:24.014251+0000","last_unstale":"2024-09-14T05:30:21.146383+0000","last_undegraded":"2024-09-14T05:30:21.146383+0000","last_fullsized":"2024-09-14T05:30:21.146383+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":963,"log_dups_size":0,"ondisk_log_size":963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:26:04.419840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010261580000000001,"stat_sum":{"num_bytes":62784795,"num_objects":73,"num_object_clones":26,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":36,"num_read":12220,"num_read_kb":23789,"num_write":1520,"num_write_kb":6967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"72'443","reported_seq":9793,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.315780+0000","last_change":"2024-09-14T05:29:26.014243+0000","last_active":"2024-09-14T05:30:21.315780+0000","last_peered":"2024-09-14T05:30:21.315780+0000","last_clean":"2024-09-14T05:30:21.315780+0000","last_became_active":"2024-09-14T05:29:24.012427+0000","last_became_peered":"2024-09-14T05:29:24.012427+0000","last_unstale":"2024-09-14T05:30:21.315780+0000","last_undegraded":"2024-09-14T05:30:21.315780+0000","last_fullsized":"2024-09-14T05:30:21.315780+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:07:38.346834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045797999999999998,"stat_sum":{"num_bytes":67940246,"num_objects":84,"num_object_clones":25,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":43,"num_read":10312,"num_read_kb":34631,"num_write":429,"num_write_kb":4602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"72'477","reported_seq":3066,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:20.937475+0000","last_change":"2024-09-14T05:29:26.014462+0000","last_active":"2024-09-14T05:30:20.937475+0000","last_peered":"2024-09-14T05:30:20.937475+0000","last_clean":"2024-09-14T05:30:20.937475+0000","last_became_active":"2024-09-14T05:29:24.012079+0000","last_became_peered":"2024-09-14T05:29:24.012079+0000","last_unstale":"2024-09-14T05:30:20.937475+0000","last_undegraded":"2024-09-14T05:30:20.937475+0000","last_fullsized":"2024-09-14T05:30:20.937475+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:34:32.666698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070589999999999997,"stat_sum":{"num_bytes":42519132,"num_objects":71,"num_object_clones":20,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":38,"num_read":3899,"num_read_kb":25779,"num_write":515,"num_write_kb":21834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"72'871","reported_seq":5520,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.436793+0000","last_change":"2024-09-14T05:29:26.014453+0000","last_active":"2024-09-14T05:30:21.436793+0000","last_peered":"2024-09-14T05:30:21.436793+0000","last_clean":"2024-09-14T05:30:21.436793+0000","last_became_active":"2024-09-14T05:29:24.012461+0000","last_became_peered":"2024-09-14T05:29:24.012461+0000","last_unstale":"2024-09-14T05:30:21.436793+0000","last_undegraded":"2024-09-14T05:30:21.436793+0000","last_fullsized":"2024-09-14T05:30:21.436793+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:44:50.817340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061290100000000001,"stat_sum":{"num_bytes":27927446,"num_objects":75,"num_object_clones":19,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":36,"num_read":9562,"num_read_kb":33231,"num_write":1189,"num_write_kb":8411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"72'397","reported_seq":1737,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.460652+0000","last_change":"2024-09-14T05:29:26.014685+0000","last_active":"2024-09-14T05:30:21.460652+0000","last_peered":"2024-09-14T05:30:21.460652+0000","last_clean":"2024-09-14T05:30:21.460652+0000","last_became_active":"2024-09-14T05:29:24.018753+0000","last_became_peered":"2024-09-14T05:29:24.018753+0000","last_unstale":"2024-09-14T05:30:21.460652+0000","last_undegraded":"2024-09-14T05:30:21.460652+0000","last_fullsized":"2024-09-14T05:30:21.460652+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:48:00.049285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089943399999999995,"stat_sum":{"num_bytes":41249397,"num_objects":74,"num_object_clones":19,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":31,"num_read":2015,"num_read_kb":13129,"num_write":470,"num_write_kb":9150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"72'563","reported_seq":2250,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.229872+0000","last_change":"2024-09-14T05:29:26.019928+0000","last_active":"2024-09-14T05:30:21.229872+0000","last_peered":"2024-09-14T05:30:21.229872+0000","last_clean":"2024-09-14T05:30:21.229872+0000","last_became_active":"2024-09-14T05:29:24.011176+0000","last_became_peered":"2024-09-14T05:29:24.011176+0000","last_unstale":"2024-09-14T05:30:21.229872+0000","last_undegraded":"2024-09-14T05:30:21.229872+0000","last_fullsized":"2024-09-14T05:30:21.229872+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:40:54.366995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011097850000000001,"stat_sum":{"num_bytes":18622431,"num_objects":61,"num_object_clones":17,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":29,"num_read":4031,"num_read_kb":6915,"num_write":611,"num_write_kb":3354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"72'291","reported_seq":2732,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.207394+0000","last_change":"2024-09-14T05:29:26.019132+0000","last_active":"2024-09-14T05:30:21.207394+0000","last_peered":"2024-09-14T05:30:21.207394+0000","last_clean":"2024-09-14T05:30:21.207394+0000","last_became_active":"2024-09-14T05:29:24.014005+0000","last_became_peered":"2024-09-14T05:29:24.014005+0000","last_unstale":"2024-09-14T05:30:21.207394+0000","last_undegraded":"2024-09-14T05:30:21.207394+0000","last_fullsized":"2024-09-14T05:30:21.207394+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:31:27.717364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045225300000000001,"stat_sum":{"num_bytes":13475953,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":22,"num_read":2968,"num_read_kb":10798,"num_write":281,"num_write_kb":5574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"72'283","reported_seq":9562,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:20.632211+0000","last_change":"2024-09-14T05:29:26.028571+0000","last_active":"2024-09-14T05:30:20.632211+0000","last_peered":"2024-09-14T05:30:20.632211+0000","last_clean":"2024-09-14T05:30:20.632211+0000","last_became_active":"2024-09-14T05:29:24.013434+0000","last_became_peered":"2024-09-14T05:29:24.013434+0000","last_unstale":"2024-09-14T05:30:20.632211+0000","last_undegraded":"2024-09-14T05:30:20.632211+0000","last_fullsized":"2024-09-14T05:30:20.632211+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:31:43.651494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038301200000000001,"stat_sum":{"num_bytes":62642231,"num_objects":72,"num_object_clones":22,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":39,"num_read":9226,"num_read_kb":22920,"num_write":283,"num_write_kb":7693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"72'362","reported_seq":2289,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-14T05:30:22.355563+0000","last_change":"2024-09-14T05:30:22.356105+0000","last_active":"2024-09-14T05:30:22.355563+0000","last_peered":"2024-09-14T05:30:22.355563+0000","last_clean":"2024-09-14T05:30:22.355563+0000","last_became_active":"2024-09-14T05:29:24.014457+0000","last_became_peered":"2024-09-14T05:29:24.014457+0000","last_unstale":"2024-09-14T05:30:22.355563+0000","last_undegraded":"2024-09-14T05:30:22.355563+0000","last_fullsized":"2024-09-14T05:30:22.355563+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:34:39.767077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.020216951,"stat_sum":{"num_bytes":73089410,"num_objects":66,"num_object_clones":21,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":30,"num_read":2400,"num_read_kb":9169,"num_write":385,"num_write_kb":12869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"28","length":"1"}]},{"pgid":"3.14","version":"72'757","reported_seq":2117,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.343984+0000","last_change":"2024-09-14T05:29:26.164291+0000","last_active":"2024-09-14T05:30:21.343984+0000","last_peered":"2024-09-14T05:30:21.343984+0000","last_clean":"2024-09-14T05:30:21.343984+0000","last_became_active":"2024-09-14T05:29:24.013266+0000","last_became_peered":"2024-09-14T05:29:24.013266+0000","last_unstale":"2024-09-14T05:30:21.343984+0000","last_undegraded":"2024-09-14T05:30:21.343984+0000","last_fullsized":"2024-09-14T05:30:21.343984+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:17:27.975699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060129699999999999,"stat_sum":{"num_bytes":22121511,"num_objects":63,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":38,"num_read":4134,"num_read_kb":5437,"num_write":1016,"num_write_kb":3941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":120,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:20.329204+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:30:20.329204+0000","last_peered":"2024-09-14T05:30:20.329204+0000","last_clean":"2024-09-14T05:30:20.329204+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:30:20.329204+0000","last_undegraded":"2024-09-14T05:30:20.329204+0000","last_fullsized":"2024-09-14T05:30:20.329204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"72'268","reported_seq":1817,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.275181+0000","last_change":"2024-09-14T05:29:26.014725+0000","last_active":"2024-09-14T05:30:21.275181+0000","last_peered":"2024-09-14T05:30:21.275181+0000","last_clean":"2024-09-14T05:30:21.275181+0000","last_became_active":"2024-09-14T05:29:24.012099+0000","last_became_peered":"2024-09-14T05:29:24.012099+0000","last_unstale":"2024-09-14T05:30:21.275181+0000","last_undegraded":"2024-09-14T05:30:21.275181+0000","last_fullsized":"2024-09-14T05:30:21.275181+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:51:44.494815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000794599,"stat_sum":{"num_bytes":12553470,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":28,"num_read":1924,"num_read_kb":3096,"num_write":255,"num_write_kb":5452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":118,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:20.355027+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:30:20.355027+0000","last_peered":"2024-09-14T05:30:20.355027+0000","last_clean":"2024-09-14T05:30:20.355027+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:30:20.355027+0000","last_undegraded":"2024-09-14T05:30:20.355027+0000","last_fullsized":"2024-09-14T05:30:20.355027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"72'775","reported_seq":3225,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.396838+0000","last_change":"2024-09-14T05:29:26.015082+0000","last_active":"2024-09-14T05:30:21.396838+0000","last_peered":"2024-09-14T05:30:21.396838+0000","last_clean":"2024-09-14T05:30:21.396838+0000","last_became_active":"2024-09-14T05:29:24.011481+0000","last_became_peered":"2024-09-14T05:29:24.011481+0000","last_unstale":"2024-09-14T05:30:21.396838+0000","last_undegraded":"2024-09-14T05:30:21.396838+0000","last_fullsized":"2024-09-14T05:30:21.396838+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:40:29.105353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011470110000000001,"stat_sum":{"num_bytes":47207274,"num_objects":69,"num_object_clones":17,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":31,"num_read":5909,"num_read_kb":6900,"num_write":969,"num_write_kb":5101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":120,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:20.329446+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:30:20.329446+0000","last_peered":"2024-09-14T05:30:20.329446+0000","last_clean":"2024-09-14T05:30:20.329446+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:30:20.329446+0000","last_undegraded":"2024-09-14T05:30:20.329446+0000","last_fullsized":"2024-09-14T05:30:20.329446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"72'486","reported_seq":2252,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.127438+0000","last_change":"2024-09-14T05:29:40.095950+0000","last_active":"2024-09-14T05:30:21.127438+0000","last_peered":"2024-09-14T05:30:21.127438+0000","last_clean":"2024-09-14T05:30:21.127438+0000","last_became_active":"2024-09-14T05:29:40.095441+0000","last_became_peered":"2024-09-14T05:29:40.095441+0000","last_unstale":"2024-09-14T05:30:21.127438+0000","last_undegraded":"2024-09-14T05:30:21.127438+0000","last_fullsized":"2024-09-14T05:30:21.127438+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:10:28.137989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053582499999999997,"stat_sum":{"num_bytes":26076097,"num_objects":68,"num_object_clones":18,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":37,"num_read":3151,"num_read_kb":11784,"num_write":570,"num_write_kb":11327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":120,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:20.333861+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:30:20.333861+0000","last_peered":"2024-09-14T05:30:20.333861+0000","last_clean":"2024-09-14T05:30:20.333861+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:30:20.333861+0000","last_undegraded":"2024-09-14T05:30:20.333861+0000","last_fullsized":"2024-09-14T05:30:20.333861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"72'283","reported_seq":1566,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.457489+0000","last_change":"2024-09-14T05:29:26.028547+0000","last_active":"2024-09-14T05:30:21.457489+0000","last_peered":"2024-09-14T05:30:21.457489+0000","last_clean":"2024-09-14T05:30:21.457489+0000","last_became_active":"2024-09-14T05:29:24.016940+0000","last_became_peered":"2024-09-14T05:29:24.016940+0000","last_unstale":"2024-09-14T05:30:21.457489+0000","last_undegraded":"2024-09-14T05:30:21.457489+0000","last_fullsized":"2024-09-14T05:30:21.457489+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:56:05.809165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042396399999999998,"stat_sum":{"num_bytes":14447384,"num_objects":59,"num_object_clones":15,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":29,"num_read":1525,"num_read_kb":4325,"num_write":293,"num_write_kb":3991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"73'435","reported_seq":2367,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-14T05:30:22.430941+0000","last_change":"2024-09-14T05:30:22.356117+0000","last_active":"2024-09-14T05:30:22.430941+0000","last_peered":"2024-09-14T05:30:22.430941+0000","last_clean":"2024-09-14T05:30:22.430941+0000","last_became_active":"2024-09-14T05:29:24.015905+0000","last_became_peered":"2024-09-14T05:29:24.015905+0000","last_unstale":"2024-09-14T05:30:22.430941+0000","last_undegraded":"2024-09-14T05:30:22.430941+0000","last_fullsized":"2024-09-14T05:30:22.430941+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:31:18.292563+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.020352354,"stat_sum":{"num_bytes":12433279,"num_objects":72,"num_object_clones":21,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":42,"num_read":2289,"num_read_kb":9472,"num_write":473,"num_write_kb":7661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"28","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":122,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:20.329038+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:30:20.329038+0000","last_peered":"2024-09-14T05:30:20.329038+0000","last_clean":"2024-09-14T05:30:20.329038+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:30:20.329038+0000","last_undegraded":"2024-09-14T05:30:20.329038+0000","last_fullsized":"2024-09-14T05:30:20.329038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"72'369","reported_seq":10341,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:20.944802+0000","last_change":"2024-09-14T05:29:26.019261+0000","last_active":"2024-09-14T05:30:20.944802+0000","last_peered":"2024-09-14T05:30:20.944802+0000","last_clean":"2024-09-14T05:30:20.944802+0000","last_became_active":"2024-09-14T05:29:24.011764+0000","last_became_peered":"2024-09-14T05:29:24.011764+0000","last_unstale":"2024-09-14T05:30:20.944802+0000","last_undegraded":"2024-09-14T05:30:20.944802+0000","last_fullsized":"2024-09-14T05:30:20.944802+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:46:17.230096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055502299999999997,"stat_sum":{"num_bytes":46878696,"num_objects":68,"num_object_clones":18,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":34,"num_read":11248,"num_read_kb":38521,"num_write":342,"num_write_kb":4828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":120,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:20.323756+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:30:20.323756+0000","last_peered":"2024-09-14T05:30:20.323756+0000","last_clean":"2024-09-14T05:30:20.323756+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:30:20.323756+0000","last_undegraded":"2024-09-14T05:30:20.323756+0000","last_fullsized":"2024-09-14T05:30:20.323756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"72'622","reported_seq":1561,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.382335+0000","last_change":"2024-09-14T05:29:40.095721+0000","last_active":"2024-09-14T05:30:21.382335+0000","last_peered":"2024-09-14T05:30:21.382335+0000","last_clean":"2024-09-14T05:30:21.382335+0000","last_became_active":"2024-09-14T05:29:40.095255+0000","last_became_peered":"2024-09-14T05:29:40.095255+0000","last_unstale":"2024-09-14T05:30:21.382335+0000","last_undegraded":"2024-09-14T05:30:21.382335+0000","last_fullsized":"2024-09-14T05:30:21.382335+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:31:51.507282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077857800000000004,"stat_sum":{"num_bytes":51944624,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":26,"num_read":2914,"num_read_kb":7078,"num_write":875,"num_write_kb":10022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":120,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:20.328739+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:30:20.328739+0000","last_peered":"2024-09-14T05:30:20.328739+0000","last_clean":"2024-09-14T05:30:20.328739+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:30:20.328739+0000","last_undegraded":"2024-09-14T05:30:20.328739+0000","last_fullsized":"2024-09-14T05:30:20.328739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"72'497","reported_seq":2288,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.356051+0000","last_change":"2024-09-14T05:29:26.015124+0000","last_active":"2024-09-14T05:30:21.356051+0000","last_peered":"2024-09-14T05:30:21.356051+0000","last_clean":"2024-09-14T05:30:21.356051+0000","last_became_active":"2024-09-14T05:29:24.015792+0000","last_became_peered":"2024-09-14T05:29:24.015792+0000","last_unstale":"2024-09-14T05:30:21.356051+0000","last_undegraded":"2024-09-14T05:30:21.356051+0000","last_fullsized":"2024-09-14T05:30:21.356051+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:41:42.525146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001049964,"stat_sum":{"num_bytes":23084481,"num_objects":74,"num_object_clones":18,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":42,"num_read":3303,"num_read_kb":9238,"num_write":571,"num_write_kb":5545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":121,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:20.329072+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:30:20.329072+0000","last_peered":"2024-09-14T05:30:20.329072+0000","last_clean":"2024-09-14T05:30:20.329072+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:30:20.329072+0000","last_undegraded":"2024-09-14T05:30:20.329072+0000","last_fullsized":"2024-09-14T05:30:20.329072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":211,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:20.331688+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:30:20.331688+0000","last_peered":"2024-09-14T05:30:20.331688+0000","last_clean":"2024-09-14T05:30:20.331688+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:30:20.331688+0000","last_undegraded":"2024-09-14T05:30:20.331688+0000","last_fullsized":"2024-09-14T05:30:20.331688+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"72'327","reported_seq":1805,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:20.513752+0000","last_change":"2024-09-14T05:29:26.014430+0000","last_active":"2024-09-14T05:30:20.513752+0000","last_peered":"2024-09-14T05:30:20.513752+0000","last_clean":"2024-09-14T05:30:20.513752+0000","last_became_active":"2024-09-14T05:29:24.012222+0000","last_became_peered":"2024-09-14T05:29:24.012222+0000","last_unstale":"2024-09-14T05:30:20.513752+0000","last_undegraded":"2024-09-14T05:30:20.513752+0000","last_fullsized":"2024-09-14T05:30:20.513752+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:08:29.880796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076580099999999998,"stat_sum":{"num_bytes":29863869,"num_objects":61,"num_object_clones":15,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":29,"num_read":2536,"num_read_kb":5780,"num_write":294,"num_write_kb":3872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"72'535","reported_seq":2038,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.458744+0000","last_change":"2024-09-14T05:29:26.164542+0000","last_active":"2024-09-14T05:30:21.458744+0000","last_peered":"2024-09-14T05:30:21.458744+0000","last_clean":"2024-09-14T05:30:21.458744+0000","last_became_active":"2024-09-14T05:29:24.012041+0000","last_became_peered":"2024-09-14T05:29:24.012041+0000","last_unstale":"2024-09-14T05:30:21.458744+0000","last_undegraded":"2024-09-14T05:30:21.458744+0000","last_fullsized":"2024-09-14T05:30:21.458744+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:53.558456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078334300000000004,"stat_sum":{"num_bytes":25503682,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":28,"num_read":2824,"num_read_kb":6906,"num_write":703,"num_write_kb":6315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"72'508","reported_seq":2034,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.341294+0000","last_change":"2024-09-14T05:29:26.165164+0000","last_active":"2024-09-14T05:30:21.341294+0000","last_peered":"2024-09-14T05:30:21.341294+0000","last_clean":"2024-09-14T05:30:21.341294+0000","last_became_active":"2024-09-14T05:29:24.011403+0000","last_became_peered":"2024-09-14T05:29:24.011403+0000","last_unstale":"2024-09-14T05:30:21.341294+0000","last_undegraded":"2024-09-14T05:30:21.341294+0000","last_fullsized":"2024-09-14T05:30:21.341294+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:52:01.008367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083114500000000004,"stat_sum":{"num_bytes":19112233,"num_objects":66,"num_object_clones":18,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":30,"num_read":3344,"num_read_kb":7513,"num_write":610,"num_write_kb":4930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"72'495","reported_seq":9510,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:20.719749+0000","last_change":"2024-09-14T05:29:26.163783+0000","last_active":"2024-09-14T05:30:20.719749+0000","last_peered":"2024-09-14T05:30:20.719749+0000","last_clean":"2024-09-14T05:30:20.719749+0000","last_became_active":"2024-09-14T05:29:24.006836+0000","last_became_peered":"2024-09-14T05:29:24.006836+0000","last_unstale":"2024-09-14T05:30:20.719749+0000","last_undegraded":"2024-09-14T05:30:20.719749+0000","last_fullsized":"2024-09-14T05:30:20.719749+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:16:10.595295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031188400000000001,"stat_sum":{"num_bytes":45603650,"num_objects":71,"num_object_clones":13,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":37,"num_read":11251,"num_read_kb":29135,"num_write":591,"num_write_kb":13031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"73'388","reported_seq":3195,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-14T05:30:22.355502+0000","last_change":"2024-09-14T05:30:22.355502+0000","last_active":"2024-09-14T05:30:22.355502+0000","last_peered":"2024-09-14T05:30:22.355502+0000","last_clean":"2024-09-14T05:30:22.355502+0000","last_became_active":"2024-09-14T05:29:24.012574+0000","last_became_peered":"2024-09-14T05:29:24.012574+0000","last_unstale":"2024-09-14T05:30:22.355502+0000","last_undegraded":"2024-09-14T05:30:22.355502+0000","last_fullsized":"2024-09-14T05:30:22.355502+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:19:21.460165+0000","scrub_duration":0,"objects_trimmed":9,"snaptrim_duration":0.019671755999999999,"stat_sum":{"num_bytes":44989371,"num_objects":69,"num_object_clones":24,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":33,"num_read":2807,"num_read_kb":13312,"num_write":349,"num_write_kb":9799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"28","length":"1"}]},{"pgid":"3.d","version":"72'943","reported_seq":10416,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.447675+0000","last_change":"2024-09-14T05:29:26.164736+0000","last_active":"2024-09-14T05:30:21.447675+0000","last_peered":"2024-09-14T05:30:21.447675+0000","last_clean":"2024-09-14T05:30:21.447675+0000","last_became_active":"2024-09-14T05:29:24.012401+0000","last_became_peered":"2024-09-14T05:29:24.012401+0000","last_unstale":"2024-09-14T05:30:21.447675+0000","last_undegraded":"2024-09-14T05:30:21.447675+0000","last_fullsized":"2024-09-14T05:30:21.447675+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":943,"log_dups_size":0,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:38:52.454823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085260200000000005,"stat_sum":{"num_bytes":47297626,"num_objects":84,"num_object_clones":22,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":44,"num_read":13627,"num_read_kb":21227,"num_write":1217,"num_write_kb":6370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"72'309","reported_seq":2503,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:20.629796+0000","last_change":"2024-09-14T05:29:26.028811+0000","last_active":"2024-09-14T05:30:20.629796+0000","last_peered":"2024-09-14T05:30:20.629796+0000","last_clean":"2024-09-14T05:30:20.629796+0000","last_became_active":"2024-09-14T05:29:24.015315+0000","last_became_peered":"2024-09-14T05:29:24.015315+0000","last_unstale":"2024-09-14T05:30:20.629796+0000","last_undegraded":"2024-09-14T05:30:20.629796+0000","last_fullsized":"2024-09-14T05:30:20.629796+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:46:58.186141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046485199999999997,"stat_sum":{"num_bytes":34233457,"num_objects":74,"num_object_clones":25,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":43,"num_read":2147,"num_read_kb":8870,"num_write":282,"num_write_kb":3905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"73'414","reported_seq":11271,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-14T05:30:22.361382+0000","last_change":"2024-09-14T05:30:22.361382+0000","last_active":"2024-09-14T05:30:22.361382+0000","last_peered":"2024-09-14T05:30:22.361382+0000","last_clean":"2024-09-14T05:30:22.361382+0000","last_became_active":"2024-09-14T05:29:24.018854+0000","last_became_peered":"2024-09-14T05:29:24.018854+0000","last_unstale":"2024-09-14T05:30:22.361382+0000","last_undegraded":"2024-09-14T05:30:22.361382+0000","last_fullsized":"2024-09-14T05:30:22.361382+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:43:35.396560+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.025298172000000001,"stat_sum":{"num_bytes":35786346,"num_objects":87,"num_object_clones":24,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":46,"num_read":11661,"num_read_kb":31578,"num_write":379,"num_write_kb":11831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"28","length":"1"}]},{"pgid":"3.10","version":"72'335","reported_seq":2683,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:20.671713+0000","last_change":"2024-09-14T05:29:26.014200+0000","last_active":"2024-09-14T05:30:20.671713+0000","last_peered":"2024-09-14T05:30:20.671713+0000","last_clean":"2024-09-14T05:30:20.671713+0000","last_became_active":"2024-09-14T05:29:24.010897+0000","last_became_peered":"2024-09-14T05:29:24.010897+0000","last_unstale":"2024-09-14T05:30:20.671713+0000","last_undegraded":"2024-09-14T05:30:20.671713+0000","last_fullsized":"2024-09-14T05:30:20.671713+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:52:36.817113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051531799999999996,"stat_sum":{"num_bytes":52185752,"num_objects":63,"num_object_clones":19,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":30,"num_read":2368,"num_read_kb":22454,"num_write":322,"num_write_kb":8937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"72'458","reported_seq":3427,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.091010+0000","last_change":"2024-09-14T05:29:26.164788+0000","last_active":"2024-09-14T05:30:21.091010+0000","last_peered":"2024-09-14T05:30:21.091010+0000","last_clean":"2024-09-14T05:30:21.091010+0000","last_became_active":"2024-09-14T05:29:24.013792+0000","last_became_peered":"2024-09-14T05:29:24.013792+0000","last_unstale":"2024-09-14T05:30:21.091010+0000","last_undegraded":"2024-09-14T05:30:21.091010+0000","last_fullsized":"2024-09-14T05:30:21.091010+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:14:55.892857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090671299999999996,"stat_sum":{"num_bytes":38077795,"num_objects":66,"num_object_clones":17,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":32,"num_read":4679,"num_read_kb":24040,"num_write":451,"num_write_kb":3219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"72'404","reported_seq":2143,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.417296+0000","last_change":"2024-09-14T05:29:40.095745+0000","last_active":"2024-09-14T05:30:21.417296+0000","last_peered":"2024-09-14T05:30:21.417296+0000","last_clean":"2024-09-14T05:30:21.417296+0000","last_became_active":"2024-09-14T05:29:40.095331+0000","last_became_peered":"2024-09-14T05:29:40.095331+0000","last_unstale":"2024-09-14T05:30:21.417296+0000","last_undegraded":"2024-09-14T05:30:21.417296+0000","last_fullsized":"2024-09-14T05:30:21.417296+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:06:09.832139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000774981,"stat_sum":{"num_bytes":33711766,"num_objects":53,"num_object_clones":15,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":23,"num_read":2742,"num_read_kb":15680,"num_write":450,"num_write_kb":10792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"72'596","reported_seq":2081,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T05:30:21.282774+0000","last_change":"2024-09-14T05:29:26.164295+0000","last_active":"2024-09-14T05:30:21.282774+0000","last_peered":"2024-09-14T05:30:21.282774+0000","last_clean":"2024-09-14T05:30:21.282774+0000","last_became_active":"2024-09-14T05:29:24.014371+0000","last_became_peered":"2024-09-14T05:29:24.014371+0000","last_unstale":"2024-09-14T05:30:21.282774+0000","last_undegraded":"2024-09-14T05:30:21.282774+0000","last_fullsized":"2024-09-14T05:30:21.282774+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:04.426673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050570599999999997,"stat_sum":{"num_bytes":60748671,"num_objects":63,"num_object_clones":16,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":28,"num_read":3306,"num_read_kb":19834,"num_write":857,"num_write_kb":5491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1261006719,"num_objects":2173,"num_object_clones":597,"num_object_copies":4346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2173,"num_whiteouts":1081,"num_read":170818,"num_read_kb":551951,"num_write":18671,"num_write_kb":278933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":315105280,"data_stored":347678190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15826,"ondisk_log_size":15826,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476755,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79708,"kb_used_data":52448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94292132,"statfs":{"total":96636764160,"available":96555143168,"internally_reserved":0,"allocated":53706752,"data_stored":55442101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61796,"kb_used_data":26448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94310044,"statfs":{"total":96636764160,"available":96573485056,"internally_reserved":0,"allocated":27082752,"data_stored":28839683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74116,"kb_used_data":34560,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94297724,"statfs":{"total":96636764160,"available":96560869376,"internally_reserved":0,"allocated":35389440,"data_stored":40353008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77536,"kb_used_data":50508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94294304,"statfs":{"total":96636764160,"available":96557367296,"internally_reserved":0,"allocated":51720192,"data_stored":57563024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76872,"kb_used_data":37352,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94294968,"statfs":{"total":96636764160,"available":96558047232,"internally_reserved":0,"allocated":38248448,"data_stored":40133127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96944,"kb_used_data":29664,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94274896,"statfs":{"total":96636764160,"available":96537493504,"internally_reserved":0,"allocated":30375936,"data_stored":34807218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74856,"kb_used_data":31148,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94296984,"statfs":{"total":96636764160,"available":96560111616,"internally_reserved":0,"allocated":31895552,"data_stored":34699339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509460,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79128,"kb_used_data":52104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94292712,"statfs":{"total":96636764160,"available":96555737088,"internally_reserved":0,"allocated":53354496,"data_stored":58466811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":51253248,"data_stored":53459224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51957760,"data_stored":57526688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":31084544,"data_stored":34339733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":31334400,"data_stored":36615828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":37388288,"data_stored":39731869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":51175424,"data_stored":57474878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":34656256,"data_stored":40064430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":26255360,"data_stored":28465540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:30:26.914 INFO:tasks.thrashosds.thrasher:Setting 3.7 to [2, 4] 2024-09-14T05:30:26.915 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.7', '2', '4'] 2024-09-14T05:30:26.915 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.7 2 4 2024-09-14T05:30:26.979 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:26.979 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:26.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:26.979 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:26.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.979 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74187 3.f 3:f707b2d9:::rbd_data.112d46b1ca94.0000000000000061:11 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:26.979 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:26.979 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.409473611000003, 2024-09-14T05:30:26.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072367828999999995, 2024-09-14T05:30:26.979 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:26.980 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:26.980 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:26.980 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:26.980 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:26.980 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.980 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74187 2024-09-14T05:30:26.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.980 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:26.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:26.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:26.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:26.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:26.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:26.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:26.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:26.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:26.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0900000000000002e-07 2024-09-14T05:30:26.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:26.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568166+0000", 2024-09-14T05:30:26.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.683e-06 2024-09-14T05:30:26.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:26.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568172+0000", 2024-09-14T05:30:26.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0639999999999999e-06 2024-09-14T05:30:26.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568177+0000", 2024-09-14T05:30:26.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.967e-06 2024-09-14T05:30:26.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:26.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640193+0000", 2024-09-14T05:30:26.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072016284 2024-09-14T05:30:26.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:26.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640282+0000", 2024-09-14T05:30:26.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9011999999999994e-05 2024-09-14T05:30:26.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:26.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640532+0000", 2024-09-14T05:30:26.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024940999999999999 2024-09-14T05:30:26.984 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.984 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.984 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.984 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74188 3.f 3:f707b2d9:::rbd_data.112d46b1ca94.0000000000000061:11 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:26.984 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:26.985 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.409454238000002, 2024-09-14T05:30:26.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072600425999999996, 2024-09-14T05:30:26.985 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:26.985 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:26.985 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:26.985 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:26.985 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:26.985 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.985 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74188 2024-09-14T05:30:26.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.985 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:26.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:26.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:26.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:26.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:26.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:26.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:26.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:26.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568184+0000", 2024-09-14T05:30:26.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8700000000000001e-07 2024-09-14T05:30:26.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:26.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568200+0000", 2024-09-14T05:30:26.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6603e-05 2024-09-14T05:30:26.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:26.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568208+0000", 2024-09-14T05:30:26.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4800000000000004e-06 2024-09-14T05:30:26.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568213+0000", 2024-09-14T05:30:26.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6860000000000003e-06 2024-09-14T05:30:26.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:26.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640563+0000", 2024-09-14T05:30:26.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072349438000000002 2024-09-14T05:30:26.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:26.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640607+0000", 2024-09-14T05:30:26.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4014999999999998e-05 2024-09-14T05:30:26.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:26.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640784+0000", 2024-09-14T05:30:26.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017681700000000001 2024-09-14T05:30:26.989 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.990 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.990 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.990 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74194 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:26.990 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:26.990 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.409416106000002, 2024-09-14T05:30:26.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072795199000000005, 2024-09-14T05:30:26.990 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:26.990 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:26.990 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:26.991 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:26.991 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:26.991 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.991 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74194 2024-09-14T05:30:26.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.991 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:26.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:26.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:26.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:26.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:26.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:26.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:26.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:26.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568222+0000", 2024-09-14T05:30:26.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1300000000000001e-07 2024-09-14T05:30:26.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:26.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568224+0000", 2024-09-14T05:30:26.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0660000000000002e-06 2024-09-14T05:30:26.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:26.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568230+0000", 2024-09-14T05:30:26.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4470000000000003e-06 2024-09-14T05:30:26.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568235+0000", 2024-09-14T05:30:26.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1220000000000003e-06 2024-09-14T05:30:26.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:26.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640824+0000", 2024-09-14T05:30:26.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072588707000000002 2024-09-14T05:30:26.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:26.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640915+0000", 2024-09-14T05:30:26.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1008999999999995e-05 2024-09-14T05:30:26.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:26.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641017+0000", 2024-09-14T05:30:26.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010143499999999999 2024-09-14T05:30:26.994 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.994 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:26.994 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.995 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74208 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2693120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:26.995 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:26.995 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.409395529999998, 2024-09-14T05:30:26.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072905555999999996, 2024-09-14T05:30:26.995 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:26.995 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:26.995 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:26.995 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:26.995 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:26.996 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:26.996 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74208 2024-09-14T05:30:26.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.996 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:26.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:26.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:26.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:26.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:26.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:26.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:26.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:26.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:26.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9799999999999999e-07 2024-09-14T05:30:26.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:26.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568244+0000", 2024-09-14T05:30:26.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0339999999999999e-06 2024-09-14T05:30:26.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:26.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568251+0000", 2024-09-14T05:30:26.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4060000000000002e-06 2024-09-14T05:30:26.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:26.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568255+0000", 2024-09-14T05:30:26.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6589999999999998e-06 2024-09-14T05:30:26.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:26.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641048+0000", 2024-09-14T05:30:26.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072792647000000002 2024-09-14T05:30:26.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:26.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641086+0000", 2024-09-14T05:30:26.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8031999999999998e-05 2024-09-14T05:30:26.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:26.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:26.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:26.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641147+0000", 2024-09-14T05:30:26.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1379999999999998e-05 2024-09-14T05:30:26.999 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:26.999 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.000 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.000 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74236 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2662400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:27.000 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:27.000 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.408761867000003, 2024-09-14T05:30:27.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072517776000000006, 2024-09-14T05:30:27.000 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.000 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.000 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.000 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.000 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.001 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.001 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74236 2024-09-14T05:30:27.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.001 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:27.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:27.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568881+0000", 2024-09-14T05:30:27.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2340000000000002e-06 2024-09-14T05:30:27.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568886+0000", 2024-09-14T05:30:27.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9450000000000001e-06 2024-09-14T05:30:27.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568898+0000", 2024-09-14T05:30:27.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2150000000000001e-05 2024-09-14T05:30:27.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568907+0000", 2024-09-14T05:30:27.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9239999999999996e-06 2024-09-14T05:30:27.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641185+0000", 2024-09-14T05:30:27.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.07227807 2024-09-14T05:30:27.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641265+0000", 2024-09-14T05:30:27.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0575999999999996e-05 2024-09-14T05:30:27.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641393+0000", 2024-09-14T05:30:27.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012787700000000001 2024-09-14T05:30:27.004 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.004 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.004 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.004 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74239 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2695168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:27.004 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:27.004 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.408717473999999, 2024-09-14T05:30:27.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072593816000000005, 2024-09-14T05:30:27.005 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.005 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.005 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.005 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.005 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.005 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.005 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74239 2024-09-14T05:30:27.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.005 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:27.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:27.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:27.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4300000000000003e-07 2024-09-14T05:30:27.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568923+0000", 2024-09-14T05:30:27.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2019999999999998e-06 2024-09-14T05:30:27.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568933+0000", 2024-09-14T05:30:27.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0376e-05 2024-09-14T05:30:27.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568938+0000", 2024-09-14T05:30:27.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2850000000000002e-06 2024-09-14T05:30:27.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641424+0000", 2024-09-14T05:30:27.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072485230999999997 2024-09-14T05:30:27.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641464+0000", 2024-09-14T05:30:27.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0574000000000002e-05 2024-09-14T05:30:27.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641514+0000", 2024-09-14T05:30:27.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9604999999999997e-05 2024-09-14T05:30:27.008 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.009 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.009 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.009 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74252 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2664448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:27.009 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:27.009 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.408537559999999, 2024-09-14T05:30:27.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072631237000000001, 2024-09-14T05:30:27.009 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.009 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.009 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.009 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.009 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.009 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.010 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74252 2024-09-14T05:30:27.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.010 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:27.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:27.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569101+0000", 2024-09-14T05:30:27.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8700000000000005e-07 2024-09-14T05:30:27.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569104+0000", 2024-09-14T05:30:27.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.163e-06 2024-09-14T05:30:27.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569115+0000", 2024-09-14T05:30:27.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1335000000000001e-05 2024-09-14T05:30:27.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569121+0000", 2024-09-14T05:30:27.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0800000000000002e-06 2024-09-14T05:30:27.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641552+0000", 2024-09-14T05:30:27.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072430743000000006 2024-09-14T05:30:27.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641631+0000", 2024-09-14T05:30:27.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9388999999999995e-05 2024-09-14T05:30:27.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641731+0000", 2024-09-14T05:30:27.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9840000000000006e-05 2024-09-14T05:30:27.013 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.013 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.013 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.013 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74256 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2697216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:27.013 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:27.013 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.408506504000002, 2024-09-14T05:30:27.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072721090000000002, 2024-09-14T05:30:27.014 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.014 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.014 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.014 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.014 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.014 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.014 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74256 2024-09-14T05:30:27.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.014 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:27.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:27.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:27.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0299999999999999e-07 2024-09-14T05:30:27.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569133+0000", 2024-09-14T05:30:27.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.046e-06 2024-09-14T05:30:27.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569139+0000", 2024-09-14T05:30:27.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0279999999999997e-06 2024-09-14T05:30:27.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569144+0000", 2024-09-14T05:30:27.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6929999999999998e-06 2024-09-14T05:30:27.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641762+0000", 2024-09-14T05:30:27.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072617904999999996 2024-09-14T05:30:27.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641801+0000", 2024-09-14T05:30:27.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8902999999999997e-05 2024-09-14T05:30:27.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641852+0000", 2024-09-14T05:30:27.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1011999999999999e-05 2024-09-14T05:30:27.018 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.018 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.018 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.019 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74285 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2672640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:27.019 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:27.019 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.407661068000003, 2024-09-14T05:30:27.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072422069000000006, 2024-09-14T05:30:27.019 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.019 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.019 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.019 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.019 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.019 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.019 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74285 2024-09-14T05:30:27.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.020 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:27.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:27.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569977+0000", 2024-09-14T05:30:27.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3099999999999998e-07 2024-09-14T05:30:27.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569978+0000", 2024-09-14T05:30:27.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.598e-06 2024-09-14T05:30:27.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569984+0000", 2024-09-14T05:30:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4689999999999996e-06 2024-09-14T05:30:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569989+0000", 2024-09-14T05:30:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8409999999999999e-06 2024-09-14T05:30:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642229+0000", 2024-09-14T05:30:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072240046000000002 2024-09-14T05:30:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642306+0000", 2024-09-14T05:30:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7347000000000003e-05 2024-09-14T05:30:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642398+0000", 2024-09-14T05:30:27.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2336999999999994e-05 2024-09-14T05:30:27.024 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.024 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.024 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.024 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74292 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2705408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:27.024 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:27.024 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.407406360000003, 2024-09-14T05:30:27.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072361018999999999, 2024-09-14T05:30:27.024 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.024 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.025 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.025 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.025 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.025 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.025 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74292 2024-09-14T05:30:27.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.025 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:27.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:27.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570232+0000", 2024-09-14T05:30:27.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.06e-07 2024-09-14T05:30:27.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570235+0000", 2024-09-14T05:30:27.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3189999999999999e-06 2024-09-14T05:30:27.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570248+0000", 2024-09-14T05:30:27.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3006e-05 2024-09-14T05:30:27.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570254+0000", 2024-09-14T05:30:27.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.908e-06 2024-09-14T05:30:27.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642433+0000", 2024-09-14T05:30:27.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072179192000000003 2024-09-14T05:30:27.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642500+0000", 2024-09-14T05:30:27.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7080000000000001e-05 2024-09-14T05:30:27.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642592+0000", 2024-09-14T05:30:27.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1807999999999999e-05 2024-09-14T05:30:27.029 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.029 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.029 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.029 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74304 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2707456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:27.029 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:27.029 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.407049694000001, 2024-09-14T05:30:27.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072444960000000003, 2024-09-14T05:30:27.029 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.030 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.030 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.030 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.030 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.030 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.030 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74304 2024-09-14T05:30:27.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.030 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:27.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:27.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:27.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9800000000000004e-07 2024-09-14T05:30:27.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570590+0000", 2024-09-14T05:30:27.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0810000000000001e-06 2024-09-14T05:30:27.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570596+0000", 2024-09-14T05:30:27.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9789999999999998e-06 2024-09-14T05:30:27.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570601+0000", 2024-09-14T05:30:27.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6750000000000001e-06 2024-09-14T05:30:27.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642835+0000", 2024-09-14T05:30:27.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.07223359 2024-09-14T05:30:27.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642917+0000", 2024-09-14T05:30:27.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2013000000000002e-05 2024-09-14T05:30:27.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643033+0000", 2024-09-14T05:30:27.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116124 2024-09-14T05:30:27.034 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.034 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.034 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.034 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74321 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2709504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:27.034 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:27.034 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.406567703999997, 2024-09-14T05:30:27.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072350611999999995, 2024-09-14T05:30:27.034 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.034 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.035 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.035 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.035 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.035 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.035 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74321 2024-09-14T05:30:27.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.035 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:27.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:27.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:27.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3e-07 2024-09-14T05:30:27.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571073+0000", 2024-09-14T05:30:27.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6879999999999999e-06 2024-09-14T05:30:27.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571080+0000", 2024-09-14T05:30:27.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0319999999999998e-06 2024-09-14T05:30:27.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571085+0000", 2024-09-14T05:30:27.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2109999999999997e-06 2024-09-14T05:30:27.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643309+0000", 2024-09-14T05:30:27.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072223812999999998 2024-09-14T05:30:27.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643351+0000", 2024-09-14T05:30:27.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2239e-05 2024-09-14T05:30:27.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643420+0000", 2024-09-14T05:30:27.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8998999999999994e-05 2024-09-14T05:30:27.038 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.039 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.039 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.039 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74332 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2678784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:27.039 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:27.039 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.406244541, 2024-09-14T05:30:27.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072259062999999998, 2024-09-14T05:30:27.039 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.039 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.039 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.039 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.039 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.039 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.040 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74332 2024-09-14T05:30:27.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.040 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:27.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:27.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571394+0000", 2024-09-14T05:30:27.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:30:27.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571397+0000", 2024-09-14T05:30:27.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8469999999999998e-06 2024-09-14T05:30:27.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571411+0000", 2024-09-14T05:30:27.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.34e-05 2024-09-14T05:30:27.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571418+0000", 2024-09-14T05:30:27.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9149999999999999e-06 2024-09-14T05:30:27.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643458+0000", 2024-09-14T05:30:27.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072039881 2024-09-14T05:30:27.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643539+0000", 2024-09-14T05:30:27.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1650000000000006e-05 2024-09-14T05:30:27.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643652+0000", 2024-09-14T05:30:27.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000112593 2024-09-14T05:30:27.043 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.043 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.043 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.043 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74336 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2711552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:27.043 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:27.043 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.406208091000003, 2024-09-14T05:30:27.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072354038999999995, 2024-09-14T05:30:27.044 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.044 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.044 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.044 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.044 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.044 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.044 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74336 2024-09-14T05:30:27.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.044 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:27.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.045 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.045 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:27.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.045 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.045 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571430+0000", 2024-09-14T05:30:27.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8999999999999996e-07 2024-09-14T05:30:27.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.045 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.045 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571432+0000", 2024-09-14T05:30:27.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4159999999999998e-06 2024-09-14T05:30:27.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571439+0000", 2024-09-14T05:30:27.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.657e-06 2024-09-14T05:30:27.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571444+0000", 2024-09-14T05:30:27.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1329999999999998e-06 2024-09-14T05:30:27.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643690+0000", 2024-09-14T05:30:27.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072246088 2024-09-14T05:30:27.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643731+0000", 2024-09-14T05:30:27.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0311000000000001e-05 2024-09-14T05:30:27.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643783+0000", 2024-09-14T05:30:27.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2843999999999998e-05 2024-09-14T05:30:27.047 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.047 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.048 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.048 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74349 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:27.048 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:27.048 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.406187082000002, 2024-09-14T05:30:27.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072554297000000004, 2024-09-14T05:30:27.048 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.048 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.048 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.048 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.048 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.048 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.048 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74349 2024-09-14T05:30:27.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.049 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:27.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:27.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571451+0000", 2024-09-14T05:30:27.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.41e-07 2024-09-14T05:30:27.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571453+0000", 2024-09-14T05:30:27.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8360000000000001e-06 2024-09-14T05:30:27.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571475+0000", 2024-09-14T05:30:27.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2452000000000001e-05 2024-09-14T05:30:27.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571480+0000", 2024-09-14T05:30:27.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4430000000000002e-06 2024-09-14T05:30:27.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643821+0000", 2024-09-14T05:30:27.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072340347999999999 2024-09-14T05:30:27.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643904+0000", 2024-09-14T05:30:27.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3534000000000004e-05 2024-09-14T05:30:27.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644005+0000", 2024-09-14T05:30:27.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100343 2024-09-14T05:30:27.052 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.052 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.053 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.053 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74357 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2713600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:27.053 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:27.053 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.406143049000001, 2024-09-14T05:30:27.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072651144000000001, 2024-09-14T05:30:27.053 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.053 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.053 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.053 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.053 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.054 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.054 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74357 2024-09-14T05:30:27.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.054 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.054 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.054 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:27.054 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.054 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:27.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571495+0000", 2024-09-14T05:30:27.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8899999999999999e-07 2024-09-14T05:30:27.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571498+0000", 2024-09-14T05:30:27.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5840000000000002e-06 2024-09-14T05:30:27.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571504+0000", 2024-09-14T05:30:27.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6549999999999998e-06 2024-09-14T05:30:27.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571509+0000", 2024-09-14T05:30:27.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6500000000000004e-06 2024-09-14T05:30:27.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644043+0000", 2024-09-14T05:30:27.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072533832000000006 2024-09-14T05:30:27.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644090+0000", 2024-09-14T05:30:27.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7426000000000003e-05 2024-09-14T05:30:27.058 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.058 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.058 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.058 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644146+0000", 2024-09-14T05:30:27.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5408000000000002e-05 2024-09-14T05:30:27.058 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.058 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.059 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.059 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74364 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2682880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:27.059 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:27.059 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.405806277000003, 2024-09-14T05:30:27.059 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072531227000000004, 2024-09-14T05:30:27.059 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.059 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.059 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.060 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.060 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.060 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.060 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74364 2024-09-14T05:30:27.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.060 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.060 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:27.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.060 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:27.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571832+0000", 2024-09-14T05:30:27.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4e-07 2024-09-14T05:30:27.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571838+0000", 2024-09-14T05:30:27.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0519999999999998e-06 2024-09-14T05:30:27.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571865+0000", 2024-09-14T05:30:27.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7109000000000001e-05 2024-09-14T05:30:27.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571883+0000", 2024-09-14T05:30:27.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7428e-05 2024-09-14T05:30:27.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644182+0000", 2024-09-14T05:30:27.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072299551000000004 2024-09-14T05:30:27.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.063 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.063 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644260+0000", 2024-09-14T05:30:27.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7551000000000003e-05 2024-09-14T05:30:27.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.063 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.063 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644362+0000", 2024-09-14T05:30:27.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010259600000000001 2024-09-14T05:30:27.063 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.063 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.063 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.064 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74367 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2715648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:27.064 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:27.064 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.405442970000003, 2024-09-14T05:30:27.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072368259000000004, 2024-09-14T05:30:27.064 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.064 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.064 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.064 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.064 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.064 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.064 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74367 2024-09-14T05:30:27.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.065 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:27.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:27.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572195+0000", 2024-09-14T05:30:27.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.78e-07 2024-09-14T05:30:27.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572200+0000", 2024-09-14T05:30:27.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2520000000000001e-06 2024-09-14T05:30:27.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572213+0000", 2024-09-14T05:30:27.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3618000000000001e-05 2024-09-14T05:30:27.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.067 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.067 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572226+0000", 2024-09-14T05:30:27.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3040999999999999e-05 2024-09-14T05:30:27.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.067 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.067 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644409+0000", 2024-09-14T05:30:27.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072182982000000007 2024-09-14T05:30:27.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.067 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644489+0000", 2024-09-14T05:30:27.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9429000000000001e-05 2024-09-14T05:30:27.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644563+0000", 2024-09-14T05:30:27.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4059000000000004e-05 2024-09-14T05:30:27.068 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.068 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.068 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.069 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74380 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2684928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:27.069 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:27.069 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.405309191999997, 2024-09-14T05:30:27.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072442180999999994, 2024-09-14T05:30:27.069 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.069 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.069 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.069 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.069 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.069 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.069 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74380 2024-09-14T05:30:27.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.069 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:27.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:27.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572329+0000", 2024-09-14T05:30:27.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3300000000000001e-07 2024-09-14T05:30:27.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572333+0000", 2024-09-14T05:30:27.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8e-06 2024-09-14T05:30:27.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572346+0000", 2024-09-14T05:30:27.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.307e-05 2024-09-14T05:30:27.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572353+0000", 2024-09-14T05:30:27.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9e-06 2024-09-14T05:30:27.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644597+0000", 2024-09-14T05:30:27.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072244770999999999 2024-09-14T05:30:27.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644672+0000", 2024-09-14T05:30:27.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4945999999999997e-05 2024-09-14T05:30:27.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644770+0000", 2024-09-14T05:30:27.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7961000000000006e-05 2024-09-14T05:30:27.073 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.073 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.073 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.073 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74383 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2717696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:27.073 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:27.073 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 37.405053197999997, 2024-09-14T05:30:27.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072397258000000006, 2024-09-14T05:30:27.073 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.073 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.073 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.074 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.074 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.074 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.074 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74383 2024-09-14T05:30:27.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.074 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.074 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.074 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.074 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:27.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.074 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.074 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.074 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:27.075 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.075 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.075 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572585+0000", 2024-09-14T05:30:27.075 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:30:27.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.075 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.075 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572589+0000", 2024-09-14T05:30:27.075 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:30:27.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.076 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.076 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572602+0000", 2024-09-14T05:30:27.076 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2917e-05 2024-09-14T05:30:27.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.076 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.076 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572610+0000", 2024-09-14T05:30:27.076 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.943e-06 2024-09-14T05:30:27.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.076 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.076 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644791+0000", 2024-09-14T05:30:27.076 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072181113000000005 2024-09-14T05:30:27.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.077 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.077 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644861+0000", 2024-09-14T05:30:27.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0650999999999998e-05 2024-09-14T05:30:27.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.077 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.077 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644981+0000", 2024-09-14T05:30:27.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000120185 2024-09-14T05:30:27.077 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.077 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.077 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.078 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.078 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.078 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:27.079 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:30:27.080 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:27.078+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:27.180 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:27.179+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:27.224 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:27.224 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:27.224 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:27.224 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:27.233 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:30:27.281 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:27.280+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:27.335 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:27.335 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:27.335 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:27.335 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:27.335 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:27.343 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:30:27.372 INFO:teuthology.orchestra.run.smithi071.stderr:set 3.7 pg_upmap mapping to [2,4] 2024-09-14T05:30:27.382 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:27.382+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:27.483 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:27.483+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:27.486 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:27.486 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:27.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:27.486 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:27.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.487 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:30:27.487 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:30:27.487 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 79.708635481000002, 2024-09-14T05:30:27.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.24562984099999999, 2024-09-14T05:30:27.487 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.487 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.487 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:27.487 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.487 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:30:27.487 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:30:27.487 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1 2024-09-14T05:30:27.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.487 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:30:27.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:30:27.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776565+0000", 2024-09-14T05:30:27.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4780000000000001e-06 2024-09-14T05:30:27.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776571+0000", 2024-09-14T05:30:27.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7669999999999998e-06 2024-09-14T05:30:27.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776580+0000", 2024-09-14T05:30:27.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9830000000000002e-06 2024-09-14T05:30:27.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776589+0000", 2024-09-14T05:30:27.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7579999999999998e-06 2024-09-14T05:30:27.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776662+0000", 2024-09-14T05:30:27.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3332999999999997e-05 2024-09-14T05:30:27.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:30:27.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776670+0000", 2024-09-14T05:30:27.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9850000000000006e-06 2024-09-14T05:30:27.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.020867+0000", 2024-09-14T05:30:27.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.244196265 2024-09-14T05:30:27.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021022+0000", 2024-09-14T05:30:27.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000155699 2024-09-14T05:30:27.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:27.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021169+0000", 2024-09-14T05:30:27.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014661700000000001 2024-09-14T05:30:27.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.492 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:30:27.492 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021891+0000", 2024-09-14T05:30:27.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072146700000000005 2024-09-14T05:30:27.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.492 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:27.492 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022124+0000", 2024-09-14T05:30:27.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023354199999999999 2024-09-14T05:30:27.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:30:27.493 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022164+0000", 2024-09-14T05:30:27.493 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9897000000000001e-05 2024-09-14T05:30:27.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.493 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022193+0000", 2024-09-14T05:30:27.493 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9050000000000001e-05 2024-09-14T05:30:27.493 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.493 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.493 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.496 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:1989 3.1f e30/24 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head v 30'29, mlcod=30'27)", 2024-09-14T05:30:27.496 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:30:27.496 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.891864494000004, 2024-09-14T05:30:27.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.064242492999999998, 2024-09-14T05:30:27.496 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.496 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.496 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:27.496 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.496 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:30:27.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:30:27.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593336+0000", 2024-09-14T05:30:27.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3909999999999999e-06 2024-09-14T05:30:27.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616632+0000", 2024-09-14T05:30:27.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.023296598000000002 2024-09-14T05:30:27.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616657+0000", 2024-09-14T05:30:27.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5228e-05 2024-09-14T05:30:27.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616684+0000", 2024-09-14T05:30:27.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6441999999999999e-05 2024-09-14T05:30:27.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616755+0000", 2024-09-14T05:30:27.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1452000000000006e-05 2024-09-14T05:30:27.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616780+0000", 2024-09-14T05:30:27.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4436000000000001e-05 2024-09-14T05:30:27.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:30:27.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657527+0000", 2024-09-14T05:30:27.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.040747323000000002 2024-09-14T05:30:27.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657577+0000", 2024-09-14T05:30:27.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9623000000000002e-05 2024-09-14T05:30:27.500 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.500 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.500 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.500 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124262 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4481024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:27.500 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867845+0000", 2024-09-14T05:30:27.501 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.6173532950000009, 2024-09-14T05:30:27.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048759513999999997, 2024-09-14T05:30:27.501 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.501 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.501 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.501 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.501 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.501 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.501 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124262 2024-09-14T05:30:27.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.501 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.501 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.501 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867845+0000", 2024-09-14T05:30:27.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867845+0000", 2024-09-14T05:30:27.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867846+0000", 2024-09-14T05:30:27.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2e-07 2024-09-14T05:30:27.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867848+0000", 2024-09-14T05:30:27.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0339999999999999e-06 2024-09-14T05:30:27.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867854+0000", 2024-09-14T05:30:27.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7350000000000001e-06 2024-09-14T05:30:27.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867859+0000", 2024-09-14T05:30:27.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9690000000000002e-06 2024-09-14T05:30:27.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916436+0000", 2024-09-14T05:30:27.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048577694999999997 2024-09-14T05:30:27.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916515+0000", 2024-09-14T05:30:27.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8982999999999999e-05 2024-09-14T05:30:27.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916605+0000", 2024-09-14T05:30:27.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9578000000000002e-05 2024-09-14T05:30:27.505 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.505 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.505 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.505 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124263 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4587520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:27.505 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:30:27.505 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.6173336680000006, 2024-09-14T05:30:27.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048866746000000003, 2024-09-14T05:30:27.505 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.505 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.505 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.505 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.506 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.506 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.506 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124263 2024-09-14T05:30:27.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.506 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.506 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.506 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:30:27.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.506 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.506 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:30:27.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:30:27.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2199999999999999e-07 2024-09-14T05:30:27.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867867+0000", 2024-09-14T05:30:27.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.091e-06 2024-09-14T05:30:27.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867873+0000", 2024-09-14T05:30:27.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8869999999999997e-06 2024-09-14T05:30:27.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867884+0000", 2024-09-14T05:30:27.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0784999999999999e-05 2024-09-14T05:30:27.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916636+0000", 2024-09-14T05:30:27.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048752141999999998 2024-09-14T05:30:27.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916680+0000", 2024-09-14T05:30:27.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3621000000000001e-05 2024-09-14T05:30:27.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916732+0000", 2024-09-14T05:30:27.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1798000000000001e-05 2024-09-14T05:30:27.509 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.509 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.509 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.510 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124264 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4694016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:27.510 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867891+0000", 2024-09-14T05:30:27.510 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.6173073129999995, 2024-09-14T05:30:27.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049046027999999998, 2024-09-14T05:30:27.510 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.510 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.510 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.510 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.510 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.510 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.510 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124264 2024-09-14T05:30:27.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.510 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867891+0000", 2024-09-14T05:30:27.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867891+0000", 2024-09-14T05:30:27.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867892+0000", 2024-09-14T05:30:27.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.39e-07 2024-09-14T05:30:27.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867894+0000", 2024-09-14T05:30:27.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.278e-06 2024-09-14T05:30:27.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867900+0000", 2024-09-14T05:30:27.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2469999999999997e-06 2024-09-14T05:30:27.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867905+0000", 2024-09-14T05:30:27.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0209999999999999e-06 2024-09-14T05:30:27.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916764+0000", 2024-09-14T05:30:27.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048859163999999997 2024-09-14T05:30:27.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916845+0000", 2024-09-14T05:30:27.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0309999999999995e-05 2024-09-14T05:30:27.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916937+0000", 2024-09-14T05:30:27.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2569000000000001e-05 2024-09-14T05:30:27.514 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.514 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.514 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.514 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124265 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:27.514 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:30:27.514 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.6172871079999993, 2024-09-14T05:30:27.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049155040999999997, 2024-09-14T05:30:27.514 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.514 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.515 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.515 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.515 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.515 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.515 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124265 2024-09-14T05:30:27.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.515 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.515 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.515 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:30:27.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:30:27.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:30:27.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7099999999999997e-07 2024-09-14T05:30:27.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867914+0000", 2024-09-14T05:30:27.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9810000000000002e-06 2024-09-14T05:30:27.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867920+0000", 2024-09-14T05:30:27.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.643e-06 2024-09-14T05:30:27.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867924+0000", 2024-09-14T05:30:27.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.605e-06 2024-09-14T05:30:27.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916970+0000", 2024-09-14T05:30:27.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049045501999999998 2024-09-14T05:30:27.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917013+0000", 2024-09-14T05:30:27.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3857999999999999e-05 2024-09-14T05:30:27.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917067+0000", 2024-09-14T05:30:27.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3081000000000002e-05 2024-09-14T05:30:27.518 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.518 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.518 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.519 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124266 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4907008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:27.519 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867930+0000", 2024-09-14T05:30:27.519 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.6172682490000003, 2024-09-14T05:30:27.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049337389000000002, 2024-09-14T05:30:27.519 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.519 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.519 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.519 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.519 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.519 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.519 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124266 2024-09-14T05:30:27.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.520 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.520 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.520 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867930+0000", 2024-09-14T05:30:27.520 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.520 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.520 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867930+0000", 2024-09-14T05:30:27.520 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867931+0000", 2024-09-14T05:30:27.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9999999999999998e-07 2024-09-14T05:30:27.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867933+0000", 2024-09-14T05:30:27.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.046e-06 2024-09-14T05:30:27.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.522 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867938+0000", 2024-09-14T05:30:27.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5589999999999996e-06 2024-09-14T05:30:27.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.522 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867941+0000", 2024-09-14T05:30:27.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.791e-06 2024-09-14T05:30:27.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.522 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917100+0000", 2024-09-14T05:30:27.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049158544999999998 2024-09-14T05:30:27.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.523 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917179+0000", 2024-09-14T05:30:27.523 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9350000000000004e-05 2024-09-14T05:30:27.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.523 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917268+0000", 2024-09-14T05:30:27.523 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8697999999999997e-05 2024-09-14T05:30:27.523 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.523 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.523 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.524 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124267 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5013504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:27.524 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:30:27.524 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.6172536930000003, 2024-09-14T05:30:27.524 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049449223, 2024-09-14T05:30:27.524 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.524 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.524 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.524 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.524 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.524 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.525 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124267 2024-09-14T05:30:27.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.525 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:30:27.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:30:27.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:30:27.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.28e-07 2024-09-14T05:30:27.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867946+0000", 2024-09-14T05:30:27.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.116e-06 2024-09-14T05:30:27.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867950+0000", 2024-09-14T05:30:27.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2049999999999998e-06 2024-09-14T05:30:27.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867952+0000", 2024-09-14T05:30:27.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6800000000000002e-06 2024-09-14T05:30:27.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917300+0000", 2024-09-14T05:30:27.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049347633000000002 2024-09-14T05:30:27.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917343+0000", 2024-09-14T05:30:27.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3022000000000002e-05 2024-09-14T05:30:27.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917394+0000", 2024-09-14T05:30:27.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1338999999999998e-05 2024-09-14T05:30:27.528 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.528 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.528 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.529 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124268 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5120000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:27.529 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867955+0000", 2024-09-14T05:30:27.529 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.6172432860000008, 2024-09-14T05:30:27.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049638350999999997, 2024-09-14T05:30:27.529 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.529 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.529 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.529 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.529 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.529 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.529 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124268 2024-09-14T05:30:27.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.530 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867955+0000", 2024-09-14T05:30:27.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867955+0000", 2024-09-14T05:30:27.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.531 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867956+0000", 2024-09-14T05:30:27.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1299999999999999e-07 2024-09-14T05:30:27.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.531 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867961+0000", 2024-09-14T05:30:27.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4269999999999999e-06 2024-09-14T05:30:27.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867965+0000", 2024-09-14T05:30:27.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.563e-06 2024-09-14T05:30:27.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867967+0000", 2024-09-14T05:30:27.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6469999999999998e-06 2024-09-14T05:30:27.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917427+0000", 2024-09-14T05:30:27.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049460102999999998 2024-09-14T05:30:27.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917506+0000", 2024-09-14T05:30:27.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8246999999999998e-05 2024-09-14T05:30:27.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917594+0000", 2024-09-14T05:30:27.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8151000000000004e-05 2024-09-14T05:30:27.533 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.533 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.533 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.534 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124269 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:27.534 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:30:27.534 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.617227669, 2024-09-14T05:30:27.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049749122999999999, 2024-09-14T05:30:27.534 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.534 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.534 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.534 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.534 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.534 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.534 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124269 2024-09-14T05:30:27.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.535 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:30:27.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:30:27.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:30:27.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2499999999999999e-07 2024-09-14T05:30:27.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.536 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867972+0000", 2024-09-14T05:30:27.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0440000000000001e-06 2024-09-14T05:30:27.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.536 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867975+0000", 2024-09-14T05:30:27.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9699999999999999e-06 2024-09-14T05:30:27.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.537 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867978+0000", 2024-09-14T05:30:27.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5909999999999998e-06 2024-09-14T05:30:27.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.537 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917625+0000", 2024-09-14T05:30:27.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049647450000000003 2024-09-14T05:30:27.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.537 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917668+0000", 2024-09-14T05:30:27.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3108000000000001e-05 2024-09-14T05:30:27.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917720+0000", 2024-09-14T05:30:27.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1734999999999998e-05 2024-09-14T05:30:27.538 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.538 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.538 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.538 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124270 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5292032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:27.538 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:30:27.539 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.617217729, 2024-09-14T05:30:27.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049946404, 2024-09-14T05:30:27.539 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.539 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.539 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.539 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.539 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.539 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.539 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124270 2024-09-14T05:30:27.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.539 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:30:27.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:30:27.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:30:27.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5600000000000002e-07 2024-09-14T05:30:27.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867982+0000", 2024-09-14T05:30:27.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.95e-07 2024-09-14T05:30:27.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867985+0000", 2024-09-14T05:30:27.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8890000000000002e-06 2024-09-14T05:30:27.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867988+0000", 2024-09-14T05:30:27.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5900000000000002e-06 2024-09-14T05:30:27.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917753+0000", 2024-09-14T05:30:27.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049765113 2024-09-14T05:30:27.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917833+0000", 2024-09-14T05:30:27.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0055000000000005e-05 2024-09-14T05:30:27.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.543 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917927+0000", 2024-09-14T05:30:27.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4506000000000006e-05 2024-09-14T05:30:27.543 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.543 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.543 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.543 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124271 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5398528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:27.543 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868010+0000", 2024-09-14T05:30:27.543 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.6171885340000003, 2024-09-14T05:30:27.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050049336, 2024-09-14T05:30:27.543 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.543 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.543 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.544 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.544 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.544 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.544 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124271 2024-09-14T05:30:27.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.544 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868010+0000", 2024-09-14T05:30:27.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868010+0000", 2024-09-14T05:30:27.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868011+0000", 2024-09-14T05:30:27.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:27.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868012+0000", 2024-09-14T05:30:27.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5719999999999999e-06 2024-09-14T05:30:27.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868016+0000", 2024-09-14T05:30:27.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.861e-06 2024-09-14T05:30:27.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868019+0000", 2024-09-14T05:30:27.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7580000000000001e-06 2024-09-14T05:30:27.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917961+0000", 2024-09-14T05:30:27.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049941885999999998 2024-09-14T05:30:27.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918007+0000", 2024-09-14T05:30:27.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6187999999999999e-05 2024-09-14T05:30:27.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918059+0000", 2024-09-14T05:30:27.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2645000000000002e-05 2024-09-14T05:30:27.547 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.547 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.547 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.548 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124272 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:27.548 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:30:27.548 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.6171765069999999, 2024-09-14T05:30:27.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050239936999999998, 2024-09-14T05:30:27.548 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.548 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.548 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.548 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.548 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.548 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.548 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124272 2024-09-14T05:30:27.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.549 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:30:27.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:30:27.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:30:27.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.16e-07 2024-09-14T05:30:27.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868023+0000", 2024-09-14T05:30:27.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7999999999999993e-07 2024-09-14T05:30:27.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868026+0000", 2024-09-14T05:30:27.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9859999999999999e-06 2024-09-14T05:30:27.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868029+0000", 2024-09-14T05:30:27.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.039e-06 2024-09-14T05:30:27.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918093+0000", 2024-09-14T05:30:27.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050063689000000001 2024-09-14T05:30:27.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918173+0000", 2024-09-14T05:30:27.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0131999999999998e-05 2024-09-14T05:30:27.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918262+0000", 2024-09-14T05:30:27.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8894999999999995e-05 2024-09-14T05:30:27.552 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.552 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.552 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.552 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124273 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:27.552 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868032+0000", 2024-09-14T05:30:27.552 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.6171663580000004, 2024-09-14T05:30:27.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050357587000000002, 2024-09-14T05:30:27.552 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.552 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.553 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.553 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.553 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.553 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.553 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124273 2024-09-14T05:30:27.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.553 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868032+0000", 2024-09-14T05:30:27.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868032+0000", 2024-09-14T05:30:27.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868033+0000", 2024-09-14T05:30:27.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9799999999999999e-07 2024-09-14T05:30:27.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868034+0000", 2024-09-14T05:30:27.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1119999999999999e-06 2024-09-14T05:30:27.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868037+0000", 2024-09-14T05:30:27.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0189999999999998e-06 2024-09-14T05:30:27.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868039+0000", 2024-09-14T05:30:27.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.57e-06 2024-09-14T05:30:27.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918294+0000", 2024-09-14T05:30:27.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050254778999999999 2024-09-14T05:30:27.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918338+0000", 2024-09-14T05:30:27.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4288e-05 2024-09-14T05:30:27.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918390+0000", 2024-09-14T05:30:27.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1520999999999997e-05 2024-09-14T05:30:27.556 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.556 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.556 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.557 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124274 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:27.557 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:30:27.557 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.6171565159999997, 2024-09-14T05:30:27.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050548334, 2024-09-14T05:30:27.557 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.557 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.557 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.557 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.557 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.557 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.557 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124274 2024-09-14T05:30:27.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.558 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:30:27.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:30:27.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:30:27.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1799999999999999e-07 2024-09-14T05:30:27.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.559 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.559 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868043+0000", 2024-09-14T05:30:27.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9999999999999995e-07 2024-09-14T05:30:27.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.559 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.559 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868046+0000", 2024-09-14T05:30:27.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8880000000000001e-06 2024-09-14T05:30:27.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.559 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868049+0000", 2024-09-14T05:30:27.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7e-06 2024-09-14T05:30:27.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.560 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918423+0000", 2024-09-14T05:30:27.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050374249000000003 2024-09-14T05:30:27.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.560 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918503+0000", 2024-09-14T05:30:27.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0049000000000006e-05 2024-09-14T05:30:27.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918590+0000", 2024-09-14T05:30:27.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7230000000000003e-05 2024-09-14T05:30:27.561 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.561 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.561 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.561 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124275 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5824512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:27.561 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:30:27.561 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.6171469300000005, 2024-09-14T05:30:27.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050665241999999999, 2024-09-14T05:30:27.562 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.562 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.562 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.562 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.562 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.562 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.562 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124275 2024-09-14T05:30:27.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.562 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:30:27.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:30:27.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:30:27.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1e-07 2024-09-14T05:30:27.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.564 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868053+0000", 2024-09-14T05:30:27.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.049e-06 2024-09-14T05:30:27.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.564 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868056+0000", 2024-09-14T05:30:27.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9519999999999999e-06 2024-09-14T05:30:27.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.564 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868058+0000", 2024-09-14T05:30:27.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4789999999999999e-06 2024-09-14T05:30:27.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.565 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918622+0000", 2024-09-14T05:30:27.565 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050563679 2024-09-14T05:30:27.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.565 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918666+0000", 2024-09-14T05:30:27.565 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3600000000000003e-05 2024-09-14T05:30:27.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918717+0000", 2024-09-14T05:30:27.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1273000000000002e-05 2024-09-14T05:30:27.566 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.566 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.566 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.566 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124276 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:27.566 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:30:27.566 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.6171373859999996, 2024-09-14T05:30:27.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050860996999999998, 2024-09-14T05:30:27.566 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.566 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.567 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.567 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.567 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.567 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.567 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124276 2024-09-14T05:30:27.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.567 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:30:27.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:30:27.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:30:27.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1e-07 2024-09-14T05:30:27.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868062+0000", 2024-09-14T05:30:27.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2099999999999995e-07 2024-09-14T05:30:27.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868065+0000", 2024-09-14T05:30:27.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9500000000000001e-06 2024-09-14T05:30:27.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868068+0000", 2024-09-14T05:30:27.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4760000000000001e-06 2024-09-14T05:30:27.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918750+0000", 2024-09-14T05:30:27.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050681926000000002 2024-09-14T05:30:27.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.570 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.570 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918829+0000", 2024-09-14T05:30:27.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9264000000000005e-05 2024-09-14T05:30:27.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.570 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.570 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918922+0000", 2024-09-14T05:30:27.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3250000000000003e-05 2024-09-14T05:30:27.570 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.570 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.570 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.571 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124277 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 6037504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:27.571 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:30:27.571 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.6171277780000004, 2024-09-14T05:30:27.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050980939000000003, 2024-09-14T05:30:27.571 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.571 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.571 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.571 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.571 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.571 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.571 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124277 2024-09-14T05:30:27.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.572 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:30:27.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:30:27.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:30:27.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.11e-07 2024-09-14T05:30:27.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868072+0000", 2024-09-14T05:30:27.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9999999999999995e-07 2024-09-14T05:30:27.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868075+0000", 2024-09-14T05:30:27.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9969999999999999e-06 2024-09-14T05:30:27.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868078+0000", 2024-09-14T05:30:27.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5349999999999999e-06 2024-09-14T05:30:27.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918956+0000", 2024-09-14T05:30:27.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050878067999999999 2024-09-14T05:30:27.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919000+0000", 2024-09-14T05:30:27.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3837000000000001e-05 2024-09-14T05:30:27.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.575 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.575 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919052+0000", 2024-09-14T05:30:27.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2290999999999998e-05 2024-09-14T05:30:27.575 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.575 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.575 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.575 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124278 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 6144000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:27.575 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868080+0000", 2024-09-14T05:30:27.575 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.6171183399999993, 2024-09-14T05:30:27.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.051170446000000001, 2024-09-14T05:30:27.576 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.576 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.576 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.576 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.576 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.576 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.576 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124278 2024-09-14T05:30:27.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.576 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.576 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868080+0000", 2024-09-14T05:30:27.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868080+0000", 2024-09-14T05:30:27.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868081+0000", 2024-09-14T05:30:27.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1500000000000001e-07 2024-09-14T05:30:27.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868081+0000", 2024-09-14T05:30:27.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9400000000000004e-07 2024-09-14T05:30:27.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868084+0000", 2024-09-14T05:30:27.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.943e-06 2024-09-14T05:30:27.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868087+0000", 2024-09-14T05:30:27.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6929999999999999e-06 2024-09-14T05:30:27.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.579 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919084+0000", 2024-09-14T05:30:27.579 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050996581999999999 2024-09-14T05:30:27.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.579 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919163+0000", 2024-09-14T05:30:27.579 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8931999999999996e-05 2024-09-14T05:30:27.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.579 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919251+0000", 2024-09-14T05:30:27.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8187000000000001e-05 2024-09-14T05:30:27.580 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.580 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.580 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.580 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124279 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 6250496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:27.580 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:30:27.580 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 9.61710873, 2024-09-14T05:30:27.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.051284430999999998, 2024-09-14T05:30:27.580 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.580 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.581 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:27.581 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.581 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.581 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.581 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124279 2024-09-14T05:30:27.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.581 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.581 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:30:27.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:30:27.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:30:27.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.11e-07 2024-09-14T05:30:27.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868091+0000", 2024-09-14T05:30:27.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8299999999999995e-07 2024-09-14T05:30:27.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.583 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.583 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868094+0000", 2024-09-14T05:30:27.583 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8870000000000001e-06 2024-09-14T05:30:27.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.583 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.583 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868097+0000", 2024-09-14T05:30:27.583 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5330000000000001e-06 2024-09-14T05:30:27.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.583 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.584 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919276+0000", 2024-09-14T05:30:27.584 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.051179740000000001 2024-09-14T05:30:27.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.584 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.584 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:27.584 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919322+0000", 2024-09-14T05:30:27.584 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5475000000000001e-05 2024-09-14T05:30:27.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.584 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.584 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:27.584 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919374+0000", 2024-09-14T05:30:27.584 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2602000000000002e-05 2024-09-14T05:30:27.584 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.585 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.585 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.585 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.585 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.585 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:27.586 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:30:27.587 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:27.585+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:27.674 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:30:27.687 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:27.687+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:27.732 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:27.732 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:27.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:136690 3.7 3.c34a33a7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T05:30:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:27.461595+0000", 2024-09-14T05:30:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.27084997799999999, 2024-09-14T05:30:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.27087082600000001, 2024-09-14T05:30:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "waiting for peered", 2024-09-14T05:30:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 136690 2024-09-14T05:30:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:27.461595+0000", 2024-09-14T05:30:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:27.461595+0000", 2024-09-14T05:30:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:27.461596+0000", 2024-09-14T05:30:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.238e-06 2024-09-14T05:30:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:27.461601+0000", 2024-09-14T05:30:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1370000000000001e-06 2024-09-14T05:30:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:27.461613+0000", 2024-09-14T05:30:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1858000000000001e-05 2024-09-14T05:30:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:27.461618+0000", 2024-09-14T05:30:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5289999999999999e-06 2024-09-14T05:30:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:27.461670+0000", 2024-09-14T05:30:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1579000000000002e-05 2024-09-14T05:30:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:30:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:27.461676+0000", 2024-09-14T05:30:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1179999999999997e-06 2024-09-14T05:30:27.737 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.737 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:27.737 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.737 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:27.737 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:30:27.737 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 1 2024-09-14T05:30:27.737 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:27.741 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:30:27.788 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:27.787+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:27.798 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:27.798 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:27.816 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:27.816 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:27.832 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:27.832 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:27.850 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:27.850 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:27.850 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:27.850 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:27.859 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:30:27.867 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:27.867 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:27.885 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:27.888 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:27.887+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:27.902 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:27.902 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:27.919 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:27.919 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:27.963 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:27.964 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:27.972 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:30:27.989 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:27.988+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:28.089 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:28.089+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:28.115 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:28.115 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:28.124 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:30:28.191 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:28.190+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:28.267 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:28.267 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:28.276 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:30:28.320 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:28.319+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:28.373 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T05:30:28.382 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:28.383 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:28.392 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:30:28.393 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:28.393+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:28.494 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:28.493+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:28.505 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:28.505 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:28.514 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:30:28.595 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:28.594+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:28.618 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:28.618 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:28.626 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:30:28.697 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:28.696+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:28.730 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:28.730 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:28.739 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:30:28.797 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:28.797+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:28.843 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:28.843 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:28.852 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:30:28.937 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:28.936+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:28.995 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:28.995 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:28.999 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:28.998+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:29.004 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:30:29.099 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:29.099+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:29.147 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:29.147 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:29.156 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:30:29.201 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:29.200+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:29.260 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:29.260 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:29.269 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:30:29.310 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:29.309+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:29.368 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-14T05:30:29.414 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:29.414 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:29.422 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:30:29.425 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:29.424+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:29.504 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:29.504+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:29.525 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:29.525 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:29.533 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:30:29.606 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:29.605+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:29.637 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:29.637 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:29.645 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:30:29.706 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:29.706+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:29.749 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:29.749 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:29.757 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:30:29.807 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:29.806+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:29.868 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:29.868 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:29.877 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:30:29.921 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:29.920+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:29.930 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:30:29.987 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:29.987 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:29.996 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:30:30.040 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:30.039+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:30.053 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:30.054 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:30.071 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:30.071 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:30.087 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:30.087 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:30.103 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:30.104 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:30.104 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:30.104 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:30.104 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:30.104 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:30.104 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:30.112 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:30.111+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:30.113 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:30:30.121 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:30.121 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:30.137 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:30.138 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:30.154 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:30.156 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:30.170 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:30.171 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:30.213 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:30.212+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:30.227 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74187 3.f 3:f707b2d9:::rbd_data.112d46b1ca94.0000000000000061:11 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:30.227 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.657421034999999, 2024-09-14T05:30:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072367828999999995, 2024-09-14T05:30:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74187 2024-09-14T05:30:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.228 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0900000000000002e-07 2024-09-14T05:30:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568166+0000", 2024-09-14T05:30:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.683e-06 2024-09-14T05:30:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568172+0000", 2024-09-14T05:30:30.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0639999999999999e-06 2024-09-14T05:30:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568177+0000", 2024-09-14T05:30:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.967e-06 2024-09-14T05:30:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640193+0000", 2024-09-14T05:30:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072016284 2024-09-14T05:30:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640282+0000", 2024-09-14T05:30:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9011999999999994e-05 2024-09-14T05:30:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640532+0000", 2024-09-14T05:30:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024940999999999999 2024-09-14T05:30:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.232 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74188 3.f 3:f707b2d9:::rbd_data.112d46b1ca94.0000000000000061:11 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.657401661999998, 2024-09-14T05:30:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072600425999999996, 2024-09-14T05:30:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.233 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74188 2024-09-14T05:30:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:30.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568184+0000", 2024-09-14T05:30:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8700000000000001e-07 2024-09-14T05:30:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568200+0000", 2024-09-14T05:30:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6603e-05 2024-09-14T05:30:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568208+0000", 2024-09-14T05:30:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4800000000000004e-06 2024-09-14T05:30:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568213+0000", 2024-09-14T05:30:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6860000000000003e-06 2024-09-14T05:30:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640563+0000", 2024-09-14T05:30:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072349438000000002 2024-09-14T05:30:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640607+0000", 2024-09-14T05:30:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4014999999999998e-05 2024-09-14T05:30:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640784+0000", 2024-09-14T05:30:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017681700000000001 2024-09-14T05:30:30.237 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74194 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.657363529999998, 2024-09-14T05:30:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072795199000000005, 2024-09-14T05:30:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.238 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.239 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.239 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.239 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74194 2024-09-14T05:30:30.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.239 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:30.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:30.240 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568222+0000", 2024-09-14T05:30:30.240 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1300000000000001e-07 2024-09-14T05:30:30.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568224+0000", 2024-09-14T05:30:30.241 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0660000000000002e-06 2024-09-14T05:30:30.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.241 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.241 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568230+0000", 2024-09-14T05:30:30.241 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4470000000000003e-06 2024-09-14T05:30:30.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.241 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.241 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568235+0000", 2024-09-14T05:30:30.241 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1220000000000003e-06 2024-09-14T05:30:30.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.242 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.242 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640824+0000", 2024-09-14T05:30:30.242 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072588707000000002 2024-09-14T05:30:30.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.242 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.242 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640915+0000", 2024-09-14T05:30:30.242 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1008999999999995e-05 2024-09-14T05:30:30.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.242 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.242 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641017+0000", 2024-09-14T05:30:30.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010143499999999999 2024-09-14T05:30:30.243 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.243 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.243 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.243 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74208 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2693120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.243 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:30.243 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.657342954000001, 2024-09-14T05:30:30.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072905555999999996, 2024-09-14T05:30:30.243 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.244 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.244 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.244 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.244 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.244 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.244 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74208 2024-09-14T05:30:30.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.244 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:30.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.245 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:30.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.245 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.245 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:30.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9799999999999999e-07 2024-09-14T05:30:30.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.246 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568244+0000", 2024-09-14T05:30:30.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0339999999999999e-06 2024-09-14T05:30:30.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.246 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568251+0000", 2024-09-14T05:30:30.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4060000000000002e-06 2024-09-14T05:30:30.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.246 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568255+0000", 2024-09-14T05:30:30.247 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6589999999999998e-06 2024-09-14T05:30:30.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.247 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.247 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641048+0000", 2024-09-14T05:30:30.247 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072792647000000002 2024-09-14T05:30:30.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.247 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.247 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641086+0000", 2024-09-14T05:30:30.247 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8031999999999998e-05 2024-09-14T05:30:30.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.248 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.248 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641147+0000", 2024-09-14T05:30:30.248 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1379999999999998e-05 2024-09-14T05:30:30.248 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.248 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.248 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.248 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74236 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2662400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.248 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:30.249 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.656709290999999, 2024-09-14T05:30:30.249 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072517776000000006, 2024-09-14T05:30:30.249 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.249 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.249 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.249 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.249 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.249 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.249 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74236 2024-09-14T05:30:30.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.249 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.250 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.250 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:30.250 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.250 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.250 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:30.250 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.250 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.250 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568881+0000", 2024-09-14T05:30:30.251 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2340000000000002e-06 2024-09-14T05:30:30.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.251 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.251 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568886+0000", 2024-09-14T05:30:30.251 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9450000000000001e-06 2024-09-14T05:30:30.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.251 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.251 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568898+0000", 2024-09-14T05:30:30.251 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2150000000000001e-05 2024-09-14T05:30:30.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.252 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568907+0000", 2024-09-14T05:30:30.252 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9239999999999996e-06 2024-09-14T05:30:30.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.252 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641185+0000", 2024-09-14T05:30:30.252 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.07227807 2024-09-14T05:30:30.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.253 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641265+0000", 2024-09-14T05:30:30.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0575999999999996e-05 2024-09-14T05:30:30.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.253 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.253 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641393+0000", 2024-09-14T05:30:30.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012787700000000001 2024-09-14T05:30:30.253 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.253 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.253 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.254 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74239 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2695168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.254 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:30.254 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.656664898000002, 2024-09-14T05:30:30.254 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072593816000000005, 2024-09-14T05:30:30.254 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.254 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.254 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.254 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.254 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.254 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.254 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74239 2024-09-14T05:30:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.256 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.256 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:30.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4300000000000003e-07 2024-09-14T05:30:30.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.257 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568923+0000", 2024-09-14T05:30:30.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2019999999999998e-06 2024-09-14T05:30:30.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.257 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568933+0000", 2024-09-14T05:30:30.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0376e-05 2024-09-14T05:30:30.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568938+0000", 2024-09-14T05:30:30.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2850000000000002e-06 2024-09-14T05:30:30.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641424+0000", 2024-09-14T05:30:30.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072485230999999997 2024-09-14T05:30:30.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641464+0000", 2024-09-14T05:30:30.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0574000000000002e-05 2024-09-14T05:30:30.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.259 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.259 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641514+0000", 2024-09-14T05:30:30.259 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9604999999999997e-05 2024-09-14T05:30:30.259 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.259 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.259 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.259 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74252 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2664448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.259 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:30.259 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.656484984000002, 2024-09-14T05:30:30.260 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072631237000000001, 2024-09-14T05:30:30.260 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.260 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.260 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.260 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.260 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.260 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.260 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74252 2024-09-14T05:30:30.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.260 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.260 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.261 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:30.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.261 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.261 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:30.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.261 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.261 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569101+0000", 2024-09-14T05:30:30.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8700000000000005e-07 2024-09-14T05:30:30.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.262 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.262 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569104+0000", 2024-09-14T05:30:30.262 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.163e-06 2024-09-14T05:30:30.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.262 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.262 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569115+0000", 2024-09-14T05:30:30.262 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1335000000000001e-05 2024-09-14T05:30:30.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569121+0000", 2024-09-14T05:30:30.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0800000000000002e-06 2024-09-14T05:30:30.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641552+0000", 2024-09-14T05:30:30.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072430743000000006 2024-09-14T05:30:30.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641631+0000", 2024-09-14T05:30:30.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9388999999999995e-05 2024-09-14T05:30:30.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641731+0000", 2024-09-14T05:30:30.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9840000000000006e-05 2024-09-14T05:30:30.264 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.264 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.264 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.264 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74256 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2697216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.265 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:30.265 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.656453927999998, 2024-09-14T05:30:30.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072721090000000002, 2024-09-14T05:30:30.265 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.265 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.265 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.265 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.265 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.265 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.265 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74256 2024-09-14T05:30:30.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.265 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.266 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:30.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.266 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:30.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.266 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:30.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0299999999999999e-07 2024-09-14T05:30:30.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569133+0000", 2024-09-14T05:30:30.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.046e-06 2024-09-14T05:30:30.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569139+0000", 2024-09-14T05:30:30.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0279999999999997e-06 2024-09-14T05:30:30.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569144+0000", 2024-09-14T05:30:30.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6929999999999998e-06 2024-09-14T05:30:30.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641762+0000", 2024-09-14T05:30:30.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072617904999999996 2024-09-14T05:30:30.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.269 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.269 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641801+0000", 2024-09-14T05:30:30.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8902999999999997e-05 2024-09-14T05:30:30.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.269 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.269 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641852+0000", 2024-09-14T05:30:30.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1011999999999999e-05 2024-09-14T05:30:30.269 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.269 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.269 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.270 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74285 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2672640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.270 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:30.270 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.655608491999999, 2024-09-14T05:30:30.270 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072422069000000006, 2024-09-14T05:30:30.270 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.270 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.270 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.270 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.270 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.270 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.270 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74285 2024-09-14T05:30:30.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.271 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.271 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.271 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:30.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.271 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.271 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:30.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.272 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569977+0000", 2024-09-14T05:30:30.272 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3099999999999998e-07 2024-09-14T05:30:30.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.272 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569978+0000", 2024-09-14T05:30:30.272 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.598e-06 2024-09-14T05:30:30.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.272 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569984+0000", 2024-09-14T05:30:30.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4689999999999996e-06 2024-09-14T05:30:30.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569989+0000", 2024-09-14T05:30:30.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8409999999999999e-06 2024-09-14T05:30:30.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642229+0000", 2024-09-14T05:30:30.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072240046000000002 2024-09-14T05:30:30.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.274 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.274 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642306+0000", 2024-09-14T05:30:30.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7347000000000003e-05 2024-09-14T05:30:30.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.274 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.274 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642398+0000", 2024-09-14T05:30:30.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2336999999999994e-05 2024-09-14T05:30:30.274 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.274 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.274 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.275 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74292 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2705408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.275 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:30.275 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.655353783999999, 2024-09-14T05:30:30.275 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072361018999999999, 2024-09-14T05:30:30.275 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.275 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.275 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.275 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.275 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.276 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.276 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74292 2024-09-14T05:30:30.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.276 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.276 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.276 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:30.276 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.276 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.276 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:30.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570232+0000", 2024-09-14T05:30:30.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.06e-07 2024-09-14T05:30:30.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570235+0000", 2024-09-14T05:30:30.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3189999999999999e-06 2024-09-14T05:30:30.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570248+0000", 2024-09-14T05:30:30.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3006e-05 2024-09-14T05:30:30.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570254+0000", 2024-09-14T05:30:30.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.908e-06 2024-09-14T05:30:30.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.279 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.279 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642433+0000", 2024-09-14T05:30:30.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072179192000000003 2024-09-14T05:30:30.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.279 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.279 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642500+0000", 2024-09-14T05:30:30.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7080000000000001e-05 2024-09-14T05:30:30.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.279 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.280 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642592+0000", 2024-09-14T05:30:30.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1807999999999999e-05 2024-09-14T05:30:30.280 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.280 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.280 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.280 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74304 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2707456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.280 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:30.280 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.654997117999997, 2024-09-14T05:30:30.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072444960000000003, 2024-09-14T05:30:30.281 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.281 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.281 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.281 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.281 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.281 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.281 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74304 2024-09-14T05:30:30.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.281 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.281 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:30.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:30.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:30.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9800000000000004e-07 2024-09-14T05:30:30.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570590+0000", 2024-09-14T05:30:30.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0810000000000001e-06 2024-09-14T05:30:30.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570596+0000", 2024-09-14T05:30:30.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9789999999999998e-06 2024-09-14T05:30:30.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570601+0000", 2024-09-14T05:30:30.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6750000000000001e-06 2024-09-14T05:30:30.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642835+0000", 2024-09-14T05:30:30.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.07223359 2024-09-14T05:30:30.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642917+0000", 2024-09-14T05:30:30.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2013000000000002e-05 2024-09-14T05:30:30.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.285 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643033+0000", 2024-09-14T05:30:30.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116124 2024-09-14T05:30:30.285 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.285 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.286 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.286 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74321 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2709504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.286 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:30.286 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.654515128, 2024-09-14T05:30:30.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072350611999999995, 2024-09-14T05:30:30.286 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.286 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.286 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.286 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.286 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.287 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.287 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74321 2024-09-14T05:30:30.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.287 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:30.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.288 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:30.288 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.288 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.288 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:30.288 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3e-07 2024-09-14T05:30:30.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.288 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.288 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571073+0000", 2024-09-14T05:30:30.288 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6879999999999999e-06 2024-09-14T05:30:30.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571080+0000", 2024-09-14T05:30:30.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0319999999999998e-06 2024-09-14T05:30:30.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.289 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.289 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571085+0000", 2024-09-14T05:30:30.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2109999999999997e-06 2024-09-14T05:30:30.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.290 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.290 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643309+0000", 2024-09-14T05:30:30.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072223812999999998 2024-09-14T05:30:30.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.290 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.290 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643351+0000", 2024-09-14T05:30:30.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2239e-05 2024-09-14T05:30:30.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.290 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643420+0000", 2024-09-14T05:30:30.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8998999999999994e-05 2024-09-14T05:30:30.291 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.291 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.291 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.291 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74332 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2678784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.291 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:30.291 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.654191965000003, 2024-09-14T05:30:30.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072259062999999998, 2024-09-14T05:30:30.292 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.292 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.292 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.292 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.292 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.292 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.292 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74332 2024-09-14T05:30:30.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.292 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.292 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:30.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.293 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.293 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:30.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.293 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.293 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571394+0000", 2024-09-14T05:30:30.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:30:30.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.294 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.294 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571397+0000", 2024-09-14T05:30:30.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8469999999999998e-06 2024-09-14T05:30:30.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.294 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.294 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571411+0000", 2024-09-14T05:30:30.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.34e-05 2024-09-14T05:30:30.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571418+0000", 2024-09-14T05:30:30.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9149999999999999e-06 2024-09-14T05:30:30.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643458+0000", 2024-09-14T05:30:30.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072039881 2024-09-14T05:30:30.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643539+0000", 2024-09-14T05:30:30.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1650000000000006e-05 2024-09-14T05:30:30.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643652+0000", 2024-09-14T05:30:30.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000112593 2024-09-14T05:30:30.296 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.296 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.296 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.297 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74336 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2711552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.297 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:30.297 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.654155514999999, 2024-09-14T05:30:30.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072354038999999995, 2024-09-14T05:30:30.297 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.297 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.297 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.297 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.297 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.297 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.297 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74336 2024-09-14T05:30:30.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.298 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.298 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.298 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:30.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.298 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.298 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:30.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.299 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571430+0000", 2024-09-14T05:30:30.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8999999999999996e-07 2024-09-14T05:30:30.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.299 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571432+0000", 2024-09-14T05:30:30.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4159999999999998e-06 2024-09-14T05:30:30.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.299 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.299 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571439+0000", 2024-09-14T05:30:30.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.657e-06 2024-09-14T05:30:30.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571444+0000", 2024-09-14T05:30:30.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1329999999999998e-06 2024-09-14T05:30:30.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643690+0000", 2024-09-14T05:30:30.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072246088 2024-09-14T05:30:30.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643731+0000", 2024-09-14T05:30:30.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0311000000000001e-05 2024-09-14T05:30:30.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643783+0000", 2024-09-14T05:30:30.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2843999999999998e-05 2024-09-14T05:30:30.301 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.301 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.301 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.302 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74349 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.302 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:30.302 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.654134505999998, 2024-09-14T05:30:30.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072554297000000004, 2024-09-14T05:30:30.302 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.302 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.302 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.302 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.302 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.302 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.303 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74349 2024-09-14T05:30:30.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.303 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.303 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.303 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:30.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.303 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.303 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:30.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.304 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.304 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571451+0000", 2024-09-14T05:30:30.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.41e-07 2024-09-14T05:30:30.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.304 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.304 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571453+0000", 2024-09-14T05:30:30.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8360000000000001e-06 2024-09-14T05:30:30.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571475+0000", 2024-09-14T05:30:30.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2452000000000001e-05 2024-09-14T05:30:30.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571480+0000", 2024-09-14T05:30:30.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4430000000000002e-06 2024-09-14T05:30:30.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643821+0000", 2024-09-14T05:30:30.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072340347999999999 2024-09-14T05:30:30.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643904+0000", 2024-09-14T05:30:30.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3534000000000004e-05 2024-09-14T05:30:30.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644005+0000", 2024-09-14T05:30:30.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100343 2024-09-14T05:30:30.307 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.307 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.307 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.307 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74357 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2713600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.307 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:30.307 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.654090472999997, 2024-09-14T05:30:30.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072651144000000001, 2024-09-14T05:30:30.307 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.307 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.307 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.308 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.308 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.308 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.308 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74357 2024-09-14T05:30:30.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.308 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.308 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.308 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:30.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:30.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571495+0000", 2024-09-14T05:30:30.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8899999999999999e-07 2024-09-14T05:30:30.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571498+0000", 2024-09-14T05:30:30.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5840000000000002e-06 2024-09-14T05:30:30.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571504+0000", 2024-09-14T05:30:30.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6549999999999998e-06 2024-09-14T05:30:30.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571509+0000", 2024-09-14T05:30:30.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6500000000000004e-06 2024-09-14T05:30:30.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644043+0000", 2024-09-14T05:30:30.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072533832000000006 2024-09-14T05:30:30.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644090+0000", 2024-09-14T05:30:30.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7426000000000003e-05 2024-09-14T05:30:30.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.312 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644146+0000", 2024-09-14T05:30:30.312 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5408000000000002e-05 2024-09-14T05:30:30.312 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.312 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.312 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.312 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74364 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2682880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.312 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:30.312 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.653753700999999, 2024-09-14T05:30:30.312 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072531227000000004, 2024-09-14T05:30:30.313 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.313 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.313 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.313 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.313 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.313 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.313 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74364 2024-09-14T05:30:30.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.313 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.313 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.313 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:30.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.314 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:30.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.314 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571832+0000", 2024-09-14T05:30:30.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4e-07 2024-09-14T05:30:30.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571838+0000", 2024-09-14T05:30:30.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0519999999999998e-06 2024-09-14T05:30:30.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571865+0000", 2024-09-14T05:30:30.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7109000000000001e-05 2024-09-14T05:30:30.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571883+0000", 2024-09-14T05:30:30.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7428e-05 2024-09-14T05:30:30.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.316 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644182+0000", 2024-09-14T05:30:30.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072299551000000004 2024-09-14T05:30:30.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.316 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644260+0000", 2024-09-14T05:30:30.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7551000000000003e-05 2024-09-14T05:30:30.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.317 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.317 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644362+0000", 2024-09-14T05:30:30.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010259600000000001 2024-09-14T05:30:30.317 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.317 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.317 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.317 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74367 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2715648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.318 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:30.318 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.653390393999999, 2024-09-14T05:30:30.318 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072368259000000004, 2024-09-14T05:30:30.318 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.318 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.318 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.318 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.318 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.318 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.318 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74367 2024-09-14T05:30:30.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.318 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:30.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:30.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572195+0000", 2024-09-14T05:30:30.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.78e-07 2024-09-14T05:30:30.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572200+0000", 2024-09-14T05:30:30.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2520000000000001e-06 2024-09-14T05:30:30.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572213+0000", 2024-09-14T05:30:30.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3618000000000001e-05 2024-09-14T05:30:30.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572226+0000", 2024-09-14T05:30:30.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3040999999999999e-05 2024-09-14T05:30:30.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644409+0000", 2024-09-14T05:30:30.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072182982000000007 2024-09-14T05:30:30.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644489+0000", 2024-09-14T05:30:30.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9429000000000001e-05 2024-09-14T05:30:30.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.322 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.322 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644563+0000", 2024-09-14T05:30:30.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4059000000000004e-05 2024-09-14T05:30:30.322 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.323 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.323 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.323 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74380 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2684928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.323 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:30.323 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.653256616, 2024-09-14T05:30:30.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072442180999999994, 2024-09-14T05:30:30.323 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.323 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.323 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.323 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.324 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.324 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.324 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74380 2024-09-14T05:30:30.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.324 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:30.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:30.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572329+0000", 2024-09-14T05:30:30.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3300000000000001e-07 2024-09-14T05:30:30.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572333+0000", 2024-09-14T05:30:30.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8e-06 2024-09-14T05:30:30.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.326 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572346+0000", 2024-09-14T05:30:30.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.307e-05 2024-09-14T05:30:30.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.326 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572353+0000", 2024-09-14T05:30:30.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9e-06 2024-09-14T05:30:30.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644597+0000", 2024-09-14T05:30:30.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072244770999999999 2024-09-14T05:30:30.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644672+0000", 2024-09-14T05:30:30.327 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4945999999999997e-05 2024-09-14T05:30:30.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.327 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.327 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644770+0000", 2024-09-14T05:30:30.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7961000000000006e-05 2024-09-14T05:30:30.328 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.328 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.328 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.328 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74383 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2717696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:30.328 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:30.328 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 40.653000622, 2024-09-14T05:30:30.328 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072397258000000006, 2024-09-14T05:30:30.328 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.329 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.329 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.329 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.329 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.329 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.329 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74383 2024-09-14T05:30:30.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.329 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.329 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.329 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:30.329 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:30.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.330 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572585+0000", 2024-09-14T05:30:30.330 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:30:30.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.330 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572589+0000", 2024-09-14T05:30:30.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:30:30.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572602+0000", 2024-09-14T05:30:30.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2917e-05 2024-09-14T05:30:30.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.331 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.331 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572610+0000", 2024-09-14T05:30:30.331 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.943e-06 2024-09-14T05:30:30.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.332 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.332 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644791+0000", 2024-09-14T05:30:30.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072181113000000005 2024-09-14T05:30:30.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.332 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.332 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644861+0000", 2024-09-14T05:30:30.332 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0650999999999998e-05 2024-09-14T05:30:30.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.332 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.333 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644981+0000", 2024-09-14T05:30:30.333 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000120185 2024-09-14T05:30:30.333 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.333 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.333 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.333 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.333 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.333 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:30.335 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:30:30.335 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:30.334+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:30.435 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:30.435+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:30.452 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:30.452 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:30.453 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:30.453 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:30.461 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:30:30.536 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:30.536+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:30.565 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:30.565 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:30.565 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:30.565 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:30.565 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:30.573 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:30:30.637 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:30.636+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:30.717 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:30.717 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:30.717 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:30.717 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:30.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.718 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:30:30.718 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:30:30.718 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 82.939258645999999, 2024-09-14T05:30:30.718 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.24562984099999999, 2024-09-14T05:30:30.718 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.718 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.718 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:30.718 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.718 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:30:30.718 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:30:30.718 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1 2024-09-14T05:30:30.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.718 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.719 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.719 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:30:30.719 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.719 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.719 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:30:30.719 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.719 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.719 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776565+0000", 2024-09-14T05:30:30.719 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4780000000000001e-06 2024-09-14T05:30:30.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.720 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.720 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776571+0000", 2024-09-14T05:30:30.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7669999999999998e-06 2024-09-14T05:30:30.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.720 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.720 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776580+0000", 2024-09-14T05:30:30.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9830000000000002e-06 2024-09-14T05:30:30.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.720 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.720 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776589+0000", 2024-09-14T05:30:30.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7579999999999998e-06 2024-09-14T05:30:30.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.721 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.721 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776662+0000", 2024-09-14T05:30:30.721 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3332999999999997e-05 2024-09-14T05:30:30.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.721 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:30:30.721 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776670+0000", 2024-09-14T05:30:30.721 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9850000000000006e-06 2024-09-14T05:30:30.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.721 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.721 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.020867+0000", 2024-09-14T05:30:30.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.244196265 2024-09-14T05:30:30.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.722 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.722 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021022+0000", 2024-09-14T05:30:30.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000155699 2024-09-14T05:30:30.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.722 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:30.722 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021169+0000", 2024-09-14T05:30:30.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014661700000000001 2024-09-14T05:30:30.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.722 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:30:30.723 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021891+0000", 2024-09-14T05:30:30.723 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072146700000000005 2024-09-14T05:30:30.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.723 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:30.723 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022124+0000", 2024-09-14T05:30:30.723 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023354199999999999 2024-09-14T05:30:30.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.723 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:30:30.723 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022164+0000", 2024-09-14T05:30:30.723 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9897000000000001e-05 2024-09-14T05:30:30.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.724 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.724 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022193+0000", 2024-09-14T05:30:30.724 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9050000000000001e-05 2024-09-14T05:30:30.724 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.724 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.724 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.725 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.725 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:1989 3.1f e30/24 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head v 30'29, mlcod=30'27)", 2024-09-14T05:30:30.726 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:30:30.726 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 60.122487659000001, 2024-09-14T05:30:30.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.064242492999999998, 2024-09-14T05:30:30.726 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.726 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.726 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:30.726 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.726 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:30:30.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:30:30.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593336+0000", 2024-09-14T05:30:30.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3909999999999999e-06 2024-09-14T05:30:30.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616632+0000", 2024-09-14T05:30:30.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.023296598000000002 2024-09-14T05:30:30.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616657+0000", 2024-09-14T05:30:30.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5228e-05 2024-09-14T05:30:30.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.729 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.729 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616684+0000", 2024-09-14T05:30:30.729 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6441999999999999e-05 2024-09-14T05:30:30.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616755+0000", 2024-09-14T05:30:30.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1452000000000006e-05 2024-09-14T05:30:30.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616780+0000", 2024-09-14T05:30:30.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4436000000000001e-05 2024-09-14T05:30:30.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:30:30.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657527+0000", 2024-09-14T05:30:30.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.040747323000000002 2024-09-14T05:30:30.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657577+0000", 2024-09-14T05:30:30.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9623000000000002e-05 2024-09-14T05:30:30.733 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.733 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.733 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.734 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124262 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4481024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:30.734 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867845+0000", 2024-09-14T05:30:30.734 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.84797646, 2024-09-14T05:30:30.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048759513999999997, 2024-09-14T05:30:30.734 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.734 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.734 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.735 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.735 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.735 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.735 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124262 2024-09-14T05:30:30.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.735 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867845+0000", 2024-09-14T05:30:30.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867845+0000", 2024-09-14T05:30:30.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867846+0000", 2024-09-14T05:30:30.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2e-07 2024-09-14T05:30:30.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.738 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.738 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867848+0000", 2024-09-14T05:30:30.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0339999999999999e-06 2024-09-14T05:30:30.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.738 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.738 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867854+0000", 2024-09-14T05:30:30.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7350000000000001e-06 2024-09-14T05:30:30.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867859+0000", 2024-09-14T05:30:30.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9690000000000002e-06 2024-09-14T05:30:30.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916436+0000", 2024-09-14T05:30:30.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048577694999999997 2024-09-14T05:30:30.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916515+0000", 2024-09-14T05:30:30.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8982999999999999e-05 2024-09-14T05:30:30.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.740 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.740 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916605+0000", 2024-09-14T05:30:30.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9578000000000002e-05 2024-09-14T05:30:30.740 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.740 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.740 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.740 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124263 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4587520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:30.741 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:30:30.741 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.847956833, 2024-09-14T05:30:30.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048866746000000003, 2024-09-14T05:30:30.741 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.741 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.741 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.741 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.741 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.741 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.741 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124263 2024-09-14T05:30:30.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.741 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.742 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:30:30.742 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.742 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:30:30.742 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.742 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:30:30.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2199999999999999e-07 2024-09-14T05:30:30.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867867+0000", 2024-09-14T05:30:30.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.091e-06 2024-09-14T05:30:30.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867873+0000", 2024-09-14T05:30:30.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8869999999999997e-06 2024-09-14T05:30:30.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.744 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867884+0000", 2024-09-14T05:30:30.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0784999999999999e-05 2024-09-14T05:30:30.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.744 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916636+0000", 2024-09-14T05:30:30.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048752141999999998 2024-09-14T05:30:30.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916680+0000", 2024-09-14T05:30:30.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3621000000000001e-05 2024-09-14T05:30:30.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.745 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916732+0000", 2024-09-14T05:30:30.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1798000000000001e-05 2024-09-14T05:30:30.745 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.745 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.745 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.745 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124264 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4694016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:30.745 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867891+0000", 2024-09-14T05:30:30.746 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.847930478, 2024-09-14T05:30:30.746 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049046027999999998, 2024-09-14T05:30:30.746 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.746 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.746 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.746 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.746 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.746 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.746 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124264 2024-09-14T05:30:30.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.746 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.746 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.746 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867891+0000", 2024-09-14T05:30:30.747 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.747 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.747 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867891+0000", 2024-09-14T05:30:30.747 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.747 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.747 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867892+0000", 2024-09-14T05:30:30.747 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.39e-07 2024-09-14T05:30:30.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.747 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.748 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867894+0000", 2024-09-14T05:30:30.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.278e-06 2024-09-14T05:30:30.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.748 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.748 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867900+0000", 2024-09-14T05:30:30.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2469999999999997e-06 2024-09-14T05:30:30.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.748 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.748 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867905+0000", 2024-09-14T05:30:30.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0209999999999999e-06 2024-09-14T05:30:30.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916764+0000", 2024-09-14T05:30:30.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048859163999999997 2024-09-14T05:30:30.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916845+0000", 2024-09-14T05:30:30.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0309999999999995e-05 2024-09-14T05:30:30.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916937+0000", 2024-09-14T05:30:30.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2569000000000001e-05 2024-09-14T05:30:30.750 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.750 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.750 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.750 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124265 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:30.750 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:30:30.750 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.847910273, 2024-09-14T05:30:30.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049155040999999997, 2024-09-14T05:30:30.751 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.751 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.751 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.751 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.751 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124265 2024-09-14T05:30:30.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.751 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.751 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:30:30.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.752 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.752 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:30:30.752 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:30:30.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7099999999999997e-07 2024-09-14T05:30:30.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.753 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867914+0000", 2024-09-14T05:30:30.753 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9810000000000002e-06 2024-09-14T05:30:30.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.753 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867920+0000", 2024-09-14T05:30:30.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.643e-06 2024-09-14T05:30:30.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867924+0000", 2024-09-14T05:30:30.754 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.605e-06 2024-09-14T05:30:30.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.754 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.754 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916970+0000", 2024-09-14T05:30:30.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049045501999999998 2024-09-14T05:30:30.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.755 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917013+0000", 2024-09-14T05:30:30.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3857999999999999e-05 2024-09-14T05:30:30.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.755 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.755 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917067+0000", 2024-09-14T05:30:30.755 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3081000000000002e-05 2024-09-14T05:30:30.755 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.756 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.756 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.756 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124266 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4907008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:30.756 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867930+0000", 2024-09-14T05:30:30.756 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.847891413999999, 2024-09-14T05:30:30.756 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049337389000000002, 2024-09-14T05:30:30.756 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.756 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.757 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.757 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.757 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.757 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.757 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124266 2024-09-14T05:30:30.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.757 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.758 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.758 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867930+0000", 2024-09-14T05:30:30.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867930+0000", 2024-09-14T05:30:30.759 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.759 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.759 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867931+0000", 2024-09-14T05:30:30.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9999999999999998e-07 2024-09-14T05:30:30.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867933+0000", 2024-09-14T05:30:30.760 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.046e-06 2024-09-14T05:30:30.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.760 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.760 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867938+0000", 2024-09-14T05:30:30.761 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5589999999999996e-06 2024-09-14T05:30:30.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.761 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.761 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867941+0000", 2024-09-14T05:30:30.761 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.791e-06 2024-09-14T05:30:30.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.761 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917100+0000", 2024-09-14T05:30:30.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049158544999999998 2024-09-14T05:30:30.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917179+0000", 2024-09-14T05:30:30.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9350000000000004e-05 2024-09-14T05:30:30.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.762 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.762 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917268+0000", 2024-09-14T05:30:30.762 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8697999999999997e-05 2024-09-14T05:30:30.762 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.762 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.763 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.763 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124267 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5013504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:30.763 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:30:30.763 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.847876857999999, 2024-09-14T05:30:30.763 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049449223, 2024-09-14T05:30:30.763 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.763 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.763 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.763 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.763 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.763 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.763 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124267 2024-09-14T05:30:30.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.764 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:30:30.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.764 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.764 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:30:30.764 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:30:30.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.28e-07 2024-09-14T05:30:30.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867946+0000", 2024-09-14T05:30:30.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.116e-06 2024-09-14T05:30:30.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.765 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.765 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867950+0000", 2024-09-14T05:30:30.765 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2049999999999998e-06 2024-09-14T05:30:30.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867952+0000", 2024-09-14T05:30:30.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6800000000000002e-06 2024-09-14T05:30:30.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917300+0000", 2024-09-14T05:30:30.766 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049347633000000002 2024-09-14T05:30:30.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.766 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.766 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917343+0000", 2024-09-14T05:30:30.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3022000000000002e-05 2024-09-14T05:30:30.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.767 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.767 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917394+0000", 2024-09-14T05:30:30.767 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1338999999999998e-05 2024-09-14T05:30:30.767 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.767 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.767 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.767 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124268 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5120000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:30.767 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867955+0000", 2024-09-14T05:30:30.767 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.847866451, 2024-09-14T05:30:30.768 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049638350999999997, 2024-09-14T05:30:30.768 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.768 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.768 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.768 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.768 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.768 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.768 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124268 2024-09-14T05:30:30.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.768 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.768 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.768 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867955+0000", 2024-09-14T05:30:30.769 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.769 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.769 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867955+0000", 2024-09-14T05:30:30.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.772 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.772 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867956+0000", 2024-09-14T05:30:30.772 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1299999999999999e-07 2024-09-14T05:30:30.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.773 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867961+0000", 2024-09-14T05:30:30.773 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4269999999999999e-06 2024-09-14T05:30:30.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.773 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867965+0000", 2024-09-14T05:30:30.774 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.563e-06 2024-09-14T05:30:30.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.774 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.774 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867967+0000", 2024-09-14T05:30:30.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6469999999999998e-06 2024-09-14T05:30:30.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.775 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.775 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917427+0000", 2024-09-14T05:30:30.775 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049460102999999998 2024-09-14T05:30:30.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.776 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.776 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917506+0000", 2024-09-14T05:30:30.776 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8246999999999998e-05 2024-09-14T05:30:30.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.777 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.777 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917594+0000", 2024-09-14T05:30:30.777 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8151000000000004e-05 2024-09-14T05:30:30.777 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.777 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.778 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.778 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124269 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:30.778 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:30:30.779 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.847850834000001, 2024-09-14T05:30:30.779 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049749122999999999, 2024-09-14T05:30:30.779 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.779 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.779 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.779 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.779 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.780 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.780 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124269 2024-09-14T05:30:30.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.780 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.781 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:30:30.781 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.781 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.782 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:30:30.782 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.782 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:30:30.783 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2499999999999999e-07 2024-09-14T05:30:30.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.783 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.783 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867972+0000", 2024-09-14T05:30:30.784 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0440000000000001e-06 2024-09-14T05:30:30.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.784 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.784 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867975+0000", 2024-09-14T05:30:30.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9699999999999999e-06 2024-09-14T05:30:30.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.785 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.785 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867978+0000", 2024-09-14T05:30:30.785 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5909999999999998e-06 2024-09-14T05:30:30.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.786 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.786 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917625+0000", 2024-09-14T05:30:30.786 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049647450000000003 2024-09-14T05:30:30.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.787 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.787 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917668+0000", 2024-09-14T05:30:30.787 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3108000000000001e-05 2024-09-14T05:30:30.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.788 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.788 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917720+0000", 2024-09-14T05:30:30.788 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1734999999999998e-05 2024-09-14T05:30:30.788 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.788 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.788 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.789 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124270 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5292032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:30.789 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:30:30.789 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.847840894000001, 2024-09-14T05:30:30.790 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049946404, 2024-09-14T05:30:30.790 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.790 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.790 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.790 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.790 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.791 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.791 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124270 2024-09-14T05:30:30.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.791 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.791 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.791 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:30:30.791 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:30:30.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.792 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.792 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:30:30.792 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5600000000000002e-07 2024-09-14T05:30:30.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867982+0000", 2024-09-14T05:30:30.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.95e-07 2024-09-14T05:30:30.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867985+0000", 2024-09-14T05:30:30.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8890000000000002e-06 2024-09-14T05:30:30.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.793 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.793 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867988+0000", 2024-09-14T05:30:30.793 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5900000000000002e-06 2024-09-14T05:30:30.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917753+0000", 2024-09-14T05:30:30.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049765113 2024-09-14T05:30:30.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.794 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.794 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917833+0000", 2024-09-14T05:30:30.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0055000000000005e-05 2024-09-14T05:30:30.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917927+0000", 2024-09-14T05:30:30.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4506000000000006e-05 2024-09-14T05:30:30.795 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.795 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.795 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.795 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124271 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5398528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:30.795 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868010+0000", 2024-09-14T05:30:30.795 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.847811698999999, 2024-09-14T05:30:30.795 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050049336, 2024-09-14T05:30:30.795 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.795 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.796 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.796 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.796 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.796 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.796 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124271 2024-09-14T05:30:30.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.796 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868010+0000", 2024-09-14T05:30:30.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868010+0000", 2024-09-14T05:30:30.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868011+0000", 2024-09-14T05:30:30.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:30.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868012+0000", 2024-09-14T05:30:30.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5719999999999999e-06 2024-09-14T05:30:30.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868016+0000", 2024-09-14T05:30:30.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.861e-06 2024-09-14T05:30:30.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868019+0000", 2024-09-14T05:30:30.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7580000000000001e-06 2024-09-14T05:30:30.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917961+0000", 2024-09-14T05:30:30.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049941885999999998 2024-09-14T05:30:30.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918007+0000", 2024-09-14T05:30:30.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6187999999999999e-05 2024-09-14T05:30:30.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918059+0000", 2024-09-14T05:30:30.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2645000000000002e-05 2024-09-14T05:30:30.800 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.800 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.800 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.800 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124272 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:30.800 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:30:30.800 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.847799672000001, 2024-09-14T05:30:30.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050239936999999998, 2024-09-14T05:30:30.800 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.800 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.800 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.800 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.801 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.801 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.801 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124272 2024-09-14T05:30:30.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.801 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:30:30.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:30:30.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:30:30.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.16e-07 2024-09-14T05:30:30.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868023+0000", 2024-09-14T05:30:30.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7999999999999993e-07 2024-09-14T05:30:30.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868026+0000", 2024-09-14T05:30:30.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9859999999999999e-06 2024-09-14T05:30:30.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868029+0000", 2024-09-14T05:30:30.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.039e-06 2024-09-14T05:30:30.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918093+0000", 2024-09-14T05:30:30.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050063689000000001 2024-09-14T05:30:30.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918173+0000", 2024-09-14T05:30:30.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0131999999999998e-05 2024-09-14T05:30:30.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918262+0000", 2024-09-14T05:30:30.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8894999999999995e-05 2024-09-14T05:30:30.804 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.804 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.804 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.805 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124273 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:30.805 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868032+0000", 2024-09-14T05:30:30.805 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.847789522999999, 2024-09-14T05:30:30.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050357587000000002, 2024-09-14T05:30:30.805 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.805 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.805 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.806 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.806 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.806 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.806 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124273 2024-09-14T05:30:30.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.806 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868032+0000", 2024-09-14T05:30:30.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.808 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868032+0000", 2024-09-14T05:30:30.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.808 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.808 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868033+0000", 2024-09-14T05:30:30.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9799999999999999e-07 2024-09-14T05:30:30.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868034+0000", 2024-09-14T05:30:30.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1119999999999999e-06 2024-09-14T05:30:30.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868037+0000", 2024-09-14T05:30:30.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0189999999999998e-06 2024-09-14T05:30:30.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.811 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868039+0000", 2024-09-14T05:30:30.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.57e-06 2024-09-14T05:30:30.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.812 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918294+0000", 2024-09-14T05:30:30.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050254778999999999 2024-09-14T05:30:30.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918338+0000", 2024-09-14T05:30:30.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4288e-05 2024-09-14T05:30:30.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918390+0000", 2024-09-14T05:30:30.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1520999999999997e-05 2024-09-14T05:30:30.814 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.815 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.815 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.815 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124274 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:30.816 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:30:30.816 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.847779681, 2024-09-14T05:30:30.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050548334, 2024-09-14T05:30:30.816 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.816 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.816 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.817 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.817 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.817 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.817 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124274 2024-09-14T05:30:30.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.818 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:30:30.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:30:30.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:30:30.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1799999999999999e-07 2024-09-14T05:30:30.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.821 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868043+0000", 2024-09-14T05:30:30.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9999999999999995e-07 2024-09-14T05:30:30.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868046+0000", 2024-09-14T05:30:30.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8880000000000001e-06 2024-09-14T05:30:30.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868049+0000", 2024-09-14T05:30:30.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7e-06 2024-09-14T05:30:30.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918423+0000", 2024-09-14T05:30:30.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050374249000000003 2024-09-14T05:30:30.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918503+0000", 2024-09-14T05:30:30.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0049000000000006e-05 2024-09-14T05:30:30.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918590+0000", 2024-09-14T05:30:30.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7230000000000003e-05 2024-09-14T05:30:30.826 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.826 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.826 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.827 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124275 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5824512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:30.827 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:30:30.827 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.847770095, 2024-09-14T05:30:30.827 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050665241999999999, 2024-09-14T05:30:30.827 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.828 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.828 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.828 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.828 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.828 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.828 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124275 2024-09-14T05:30:30.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.829 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:30:30.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:30:30.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:30:30.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1e-07 2024-09-14T05:30:30.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868053+0000", 2024-09-14T05:30:30.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.049e-06 2024-09-14T05:30:30.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868056+0000", 2024-09-14T05:30:30.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9519999999999999e-06 2024-09-14T05:30:30.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868058+0000", 2024-09-14T05:30:30.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4789999999999999e-06 2024-09-14T05:30:30.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918622+0000", 2024-09-14T05:30:30.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050563679 2024-09-14T05:30:30.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918666+0000", 2024-09-14T05:30:30.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3600000000000003e-05 2024-09-14T05:30:30.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918717+0000", 2024-09-14T05:30:30.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1273000000000002e-05 2024-09-14T05:30:30.836 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.836 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.836 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.836 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124276 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:30.837 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:30:30.837 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.847760551, 2024-09-14T05:30:30.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050860996999999998, 2024-09-14T05:30:30.837 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.837 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.837 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.837 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.837 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.837 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.837 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124276 2024-09-14T05:30:30.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.837 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:30:30.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:30:30.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:30:30.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1e-07 2024-09-14T05:30:30.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868062+0000", 2024-09-14T05:30:30.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2099999999999995e-07 2024-09-14T05:30:30.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868065+0000", 2024-09-14T05:30:30.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9500000000000001e-06 2024-09-14T05:30:30.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868068+0000", 2024-09-14T05:30:30.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4760000000000001e-06 2024-09-14T05:30:30.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918750+0000", 2024-09-14T05:30:30.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050681926000000002 2024-09-14T05:30:30.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918829+0000", 2024-09-14T05:30:30.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9264000000000005e-05 2024-09-14T05:30:30.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918922+0000", 2024-09-14T05:30:30.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3250000000000003e-05 2024-09-14T05:30:30.841 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.841 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.841 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.841 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124277 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 6037504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:30.841 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:30:30.841 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.847750942999999, 2024-09-14T05:30:30.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050980939000000003, 2024-09-14T05:30:30.842 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.842 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.842 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.842 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.842 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.842 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.842 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124277 2024-09-14T05:30:30.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.842 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:30:30.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:30:30.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:30:30.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.11e-07 2024-09-14T05:30:30.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868072+0000", 2024-09-14T05:30:30.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9999999999999995e-07 2024-09-14T05:30:30.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868075+0000", 2024-09-14T05:30:30.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9969999999999999e-06 2024-09-14T05:30:30.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868078+0000", 2024-09-14T05:30:30.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5349999999999999e-06 2024-09-14T05:30:30.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918956+0000", 2024-09-14T05:30:30.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050878067999999999 2024-09-14T05:30:30.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919000+0000", 2024-09-14T05:30:30.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3837000000000001e-05 2024-09-14T05:30:30.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919052+0000", 2024-09-14T05:30:30.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2290999999999998e-05 2024-09-14T05:30:30.846 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.846 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.846 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.847 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124278 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 6144000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:30.847 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868080+0000", 2024-09-14T05:30:30.847 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.847741505, 2024-09-14T05:30:30.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.051170446000000001, 2024-09-14T05:30:30.847 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.847 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.847 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.847 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.847 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.847 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.847 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124278 2024-09-14T05:30:30.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.848 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868080+0000", 2024-09-14T05:30:30.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.848 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868080+0000", 2024-09-14T05:30:30.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868081+0000", 2024-09-14T05:30:30.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1500000000000001e-07 2024-09-14T05:30:30.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868081+0000", 2024-09-14T05:30:30.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9400000000000004e-07 2024-09-14T05:30:30.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868084+0000", 2024-09-14T05:30:30.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.943e-06 2024-09-14T05:30:30.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868087+0000", 2024-09-14T05:30:30.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6929999999999999e-06 2024-09-14T05:30:30.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919084+0000", 2024-09-14T05:30:30.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050996581999999999 2024-09-14T05:30:30.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919163+0000", 2024-09-14T05:30:30.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8931999999999996e-05 2024-09-14T05:30:30.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919251+0000", 2024-09-14T05:30:30.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8187000000000001e-05 2024-09-14T05:30:30.851 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.852 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.852 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.852 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124279 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 6250496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:30.852 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:30:30.852 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 12.847731895000001, 2024-09-14T05:30:30.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.051284430999999998, 2024-09-14T05:30:30.852 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.853 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.853 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:30.853 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.853 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.853 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.853 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124279 2024-09-14T05:30:30.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.854 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:30:30.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:30:30.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:30:30.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.11e-07 2024-09-14T05:30:30.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868091+0000", 2024-09-14T05:30:30.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8299999999999995e-07 2024-09-14T05:30:30.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868094+0000", 2024-09-14T05:30:30.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8870000000000001e-06 2024-09-14T05:30:30.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868097+0000", 2024-09-14T05:30:30.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5330000000000001e-06 2024-09-14T05:30:30.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919276+0000", 2024-09-14T05:30:30.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.051179740000000001 2024-09-14T05:30:30.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:30.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919322+0000", 2024-09-14T05:30:30.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5475000000000001e-05 2024-09-14T05:30:30.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:30.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919374+0000", 2024-09-14T05:30:30.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2602000000000002e-05 2024-09-14T05:30:30.860 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.860 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.861 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.861 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.861 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.861 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:30.863 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:30:30.864 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:30.862+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:30.963 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:30.963+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:30.966 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:30.966 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:30.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.967 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:136816 3.7 3.c34a33a7 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-14T05:30:30.967 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:30.438431+0000", 2024-09-14T05:30:30.967 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.52821859800000004, 2024-09-14T05:30:30.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.52823636900000004, 2024-09-14T05:30:30.967 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:30.967 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:30.967 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "waiting for peered", 2024-09-14T05:30:30.967 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:30.967 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:30.967 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:30.967 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 136816 2024-09-14T05:30:30.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.968 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:30.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:30.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438431+0000", 2024-09-14T05:30:30.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:30.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438431+0000", 2024-09-14T05:30:30.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:30.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:30.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438432+0000", 2024-09-14T05:30:30.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0419999999999999e-06 2024-09-14T05:30:30.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:30.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438437+0000", 2024-09-14T05:30:30.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5000000000000001e-06 2024-09-14T05:30:30.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:30.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438446+0000", 2024-09-14T05:30:30.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5740000000000002e-06 2024-09-14T05:30:30.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:30.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438456+0000", 2024-09-14T05:30:30.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0056e-05 2024-09-14T05:30:30.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:30.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438500+0000", 2024-09-14T05:30:30.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3738e-05 2024-09-14T05:30:30.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:30.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:30.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:30:30.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438506+0000", 2024-09-14T05:30:30.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1659999999999999e-06 2024-09-14T05:30:30.971 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.971 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:30.971 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.971 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:30.971 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:30:30.971 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 1 2024-09-14T05:30:30.971 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:30.975 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:30:31.065 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:31.064+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:31.117 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:31.117 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:31.117 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:31.117 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:31.117 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:31.125 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:30:31.165 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:31.165+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:31.231 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73757 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2816000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:30:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.669955232, 2024-09-14T05:30:31.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044614540000000001, 2024-09-14T05:30:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73757 2024-09-14T05:30:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:30:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559678+0000", 2024-09-14T05:30:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559679+0000", 2024-09-14T05:30:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4000000000000002e-07 2024-09-14T05:30:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559681+0000", 2024-09-14T05:30:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1780000000000002e-06 2024-09-14T05:30:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559687+0000", 2024-09-14T05:30:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0270000000000001e-06 2024-09-14T05:30:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559692+0000", 2024-09-14T05:30:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1030000000000001e-06 2024-09-14T05:30:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604180+0000", 2024-09-14T05:30:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044487480000000003 2024-09-14T05:30:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604224+0000", 2024-09-14T05:30:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4357999999999998e-05 2024-09-14T05:30:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604293+0000", 2024-09-14T05:30:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8954000000000004e-05 2024-09-14T05:30:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73760 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2848768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.669935127000002, 2024-09-14T05:30:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.04469911, 2024-09-14T05:30:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73760 2024-09-14T05:30:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.237 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559701+0000", 2024-09-14T05:30:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1909999999999999e-06 2024-09-14T05:30:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559708+0000", 2024-09-14T05:30:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7279999999999998e-06 2024-09-14T05:30:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559713+0000", 2024-09-14T05:30:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8389999999999997e-06 2024-09-14T05:30:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604310+0000", 2024-09-14T05:30:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044597664000000002 2024-09-14T05:30:31.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604352+0000", 2024-09-14T05:30:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1236000000000003e-05 2024-09-14T05:30:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604398+0000", 2024-09-14T05:30:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6091000000000003e-05 2024-09-14T05:30:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73773 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2801664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.669914958, 2024-09-14T05:30:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941026000000002, 2024-09-14T05:30:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73773 2024-09-14T05:30:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8599999999999999e-07 2024-09-14T05:30:31.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559721+0000", 2024-09-14T05:30:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0540000000000002e-06 2024-09-14T05:30:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559729+0000", 2024-09-14T05:30:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1359999999999997e-06 2024-09-14T05:30:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559734+0000", 2024-09-14T05:30:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2140000000000004e-06 2024-09-14T05:30:31.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604441+0000", 2024-09-14T05:30:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044706828999999997 2024-09-14T05:30:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604522+0000", 2024-09-14T05:30:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0603000000000001e-05 2024-09-14T05:30:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604660+0000", 2024-09-14T05:30:31.244 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000137804 2024-09-14T05:30:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73777 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2834432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.669892556999997, 2024-09-14T05:30:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045116270999999999, 2024-09-14T05:30:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.245 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73777 2024-09-14T05:30:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559742+0000", 2024-09-14T05:30:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4099999999999999e-07 2024-09-14T05:30:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.247 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559744+0000", 2024-09-14T05:30:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0040000000000002e-06 2024-09-14T05:30:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559749+0000", 2024-09-14T05:30:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9739999999999999e-06 2024-09-14T05:30:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559754+0000", 2024-09-14T05:30:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9540000000000003e-06 2024-09-14T05:30:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604695+0000", 2024-09-14T05:30:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941003 2024-09-14T05:30:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604766+0000", 2024-09-14T05:30:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0296999999999995e-05 2024-09-14T05:30:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604857+0000", 2024-09-14T05:30:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1598e-05 2024-09-14T05:30:31.249 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73790 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.669872622, 2024-09-14T05:30:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316437000000001, 2024-09-14T05:30:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.250 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73790 2024-09-14T05:30:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7599999999999998e-07 2024-09-14T05:30:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559763+0000", 2024-09-14T05:30:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0880000000000002e-06 2024-09-14T05:30:31.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559769+0000", 2024-09-14T05:30:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6740000000000002e-06 2024-09-14T05:30:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559774+0000", 2024-09-14T05:30:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6110000000000004e-06 2024-09-14T05:30:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604905+0000", 2024-09-14T05:30:31.253 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045130827999999998 2024-09-14T05:30:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604982+0000", 2024-09-14T05:30:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7410999999999995e-05 2024-09-14T05:30:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.254 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605077+0000", 2024-09-14T05:30:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5449e-05 2024-09-14T05:30:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73793 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2836480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.255 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.669853418000002, 2024-09-14T05:30:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045441212000000002, 2024-09-14T05:30:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73793 2024-09-14T05:30:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.256 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.257 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559781+0000", 2024-09-14T05:30:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2899999999999999e-07 2024-09-14T05:30:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559783+0000", 2024-09-14T05:30:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.075e-06 2024-09-14T05:30:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559788+0000", 2024-09-14T05:30:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5369999999999997e-06 2024-09-14T05:30:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.258 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559793+0000", 2024-09-14T05:30:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:30:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605109+0000", 2024-09-14T05:30:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316229999999999 2024-09-14T05:30:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605168+0000", 2024-09-14T05:30:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8949000000000001e-05 2024-09-14T05:30:31.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.260 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.260 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605221+0000", 2024-09-14T05:30:31.260 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3260000000000002e-05 2024-09-14T05:30:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73806 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.669834792000003, 2024-09-14T05:30:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045526189000000002, 2024-09-14T05:30:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.261 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73806 2024-09-14T05:30:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.262 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:30:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559801+0000", 2024-09-14T05:30:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.995e-06 2024-09-14T05:30:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559807+0000", 2024-09-14T05:30:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5450000000000003e-06 2024-09-14T05:30:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559811+0000", 2024-09-14T05:30:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5090000000000004e-06 2024-09-14T05:30:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605237+0000", 2024-09-14T05:30:31.264 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045426133 2024-09-14T05:30:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605278+0000", 2024-09-14T05:30:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1084999999999997e-05 2024-09-14T05:30:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605325+0000", 2024-09-14T05:30:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6582999999999998e-05 2024-09-14T05:30:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.265 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73809 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2838528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.669811287000002, 2024-09-14T05:30:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045596062999999999, 2024-09-14T05:30:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73809 2024-09-14T05:30:31.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559823+0000", 2024-09-14T05:30:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.08e-07 2024-09-14T05:30:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559825+0000", 2024-09-14T05:30:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.26e-06 2024-09-14T05:30:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.268 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559831+0000", 2024-09-14T05:30:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6690000000000003e-06 2024-09-14T05:30:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559835+0000", 2024-09-14T05:30:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5120000000000002e-06 2024-09-14T05:30:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605342+0000", 2024-09-14T05:30:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045506957000000001 2024-09-14T05:30:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605375+0000", 2024-09-14T05:30:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3250999999999997e-05 2024-09-14T05:30:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605418+0000", 2024-09-14T05:30:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3006000000000001e-05 2024-09-14T05:30:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.270 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73822 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2807808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.669783633999998, 2024-09-14T05:30:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045661405000000002, 2024-09-14T05:30:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73822 2024-09-14T05:30:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.271 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559851+0000", 2024-09-14T05:30:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4000000000000002e-07 2024-09-14T05:30:31.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559853+0000", 2024-09-14T05:30:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5380000000000001e-06 2024-09-14T05:30:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559859+0000", 2024-09-14T05:30:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0619999999999997e-06 2024-09-14T05:30:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559864+0000", 2024-09-14T05:30:31.273 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.977e-06 2024-09-14T05:30:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605435+0000", 2024-09-14T05:30:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045570672 2024-09-14T05:30:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605467+0000", 2024-09-14T05:30:31.274 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2486e-05 2024-09-14T05:30:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605511+0000", 2024-09-14T05:30:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4129999999999999e-05 2024-09-14T05:30:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73825 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2840576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.669760269999998, 2024-09-14T05:30:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045728694, 2024-09-14T05:30:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73825 2024-09-14T05:30:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559874+0000", 2024-09-14T05:30:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5400000000000002e-07 2024-09-14T05:30:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.277 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559876+0000", 2024-09-14T05:30:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.131e-06 2024-09-14T05:30:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559888+0000", 2024-09-14T05:30:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2216e-05 2024-09-14T05:30:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559893+0000", 2024-09-14T05:30:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6539999999999998e-06 2024-09-14T05:30:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.278 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605526+0000", 2024-09-14T05:30:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045633119 2024-09-14T05:30:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605560+0000", 2024-09-14T05:30:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3825999999999998e-05 2024-09-14T05:30:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605602+0000", 2024-09-14T05:30:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2293999999999999e-05 2024-09-14T05:30:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.279 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74023 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2863104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.668574294999999, 2024-09-14T05:30:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044635385999999999, 2024-09-14T05:30:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.280 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74023 2024-09-14T05:30:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561060+0000", 2024-09-14T05:30:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0699999999999998e-07 2024-09-14T05:30:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561072+0000", 2024-09-14T05:30:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2261e-05 2024-09-14T05:30:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561079+0000", 2024-09-14T05:30:31.282 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8399999999999997e-06 2024-09-14T05:30:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561084+0000", 2024-09-14T05:30:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7380000000000002e-06 2024-09-14T05:30:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605620+0000", 2024-09-14T05:30:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044536146999999998 2024-09-14T05:30:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.283 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605653+0000", 2024-09-14T05:30:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2969000000000002e-05 2024-09-14T05:30:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605695+0000", 2024-09-14T05:30:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2024000000000003e-05 2024-09-14T05:30:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74026 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2895872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.284 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.668542832999997, 2024-09-14T05:30:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044692717, 2024-09-14T05:30:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74026 2024-09-14T05:30:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.285 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561093+0000", 2024-09-14T05:30:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-14T05:30:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561099+0000", 2024-09-14T05:30:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0399999999999998e-06 2024-09-14T05:30:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561104+0000", 2024-09-14T05:30:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8180000000000003e-06 2024-09-14T05:30:31.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.288 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.288 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605710+0000", 2024-09-14T05:30:31.288 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044605657 2024-09-14T05:30:31.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.288 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.288 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605742+0000", 2024-09-14T05:30:31.288 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1949000000000002e-05 2024-09-14T05:30:31.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.288 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.288 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605784+0000", 2024-09-14T05:30:31.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1884000000000001e-05 2024-09-14T05:30:31.289 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.289 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.289 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.289 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74039 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.289 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:31.289 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.668416235000002, 2024-09-14T05:30:31.289 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044654905000000002, 2024-09-14T05:30:31.289 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.289 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.289 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.289 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74039 2024-09-14T05:30:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561218+0000", 2024-09-14T05:30:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4000000000000001e-07 2024-09-14T05:30:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561221+0000", 2024-09-14T05:30:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9730000000000002e-06 2024-09-14T05:30:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561229+0000", 2024-09-14T05:30:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1759999999999993e-06 2024-09-14T05:30:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561235+0000", 2024-09-14T05:30:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0109999999999997e-06 2024-09-14T05:30:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605798+0000", 2024-09-14T05:30:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044563285000000001 2024-09-14T05:30:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605831+0000", 2024-09-14T05:30:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2054999999999997e-05 2024-09-14T05:30:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605872+0000", 2024-09-14T05:30:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1764999999999997e-05 2024-09-14T05:30:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74042 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.668367781999997, 2024-09-14T05:30:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044773234000000002, 2024-09-14T05:30:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74042 2024-09-14T05:30:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4099999999999998e-07 2024-09-14T05:30:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561269+0000", 2024-09-14T05:30:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7930000000000002e-06 2024-09-14T05:30:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561278+0000", 2024-09-14T05:30:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5150000000000006e-06 2024-09-14T05:30:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561284+0000", 2024-09-14T05:30:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2029999999999997e-06 2024-09-14T05:30:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605934+0000", 2024-09-14T05:30:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044650449000000002 2024-09-14T05:30:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605983+0000", 2024-09-14T05:30:31.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8943999999999998e-05 2024-09-14T05:30:31.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.298 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.298 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606039+0000", 2024-09-14T05:30:31.298 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5689000000000002e-05 2024-09-14T05:30:31.298 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.298 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.298 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74052 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.668340907000001, 2024-09-14T05:30:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044830474000000002, 2024-09-14T05:30:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74052 2024-09-14T05:30:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1099999999999996e-07 2024-09-14T05:30:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561295+0000", 2024-09-14T05:30:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7770000000000001e-06 2024-09-14T05:30:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561303+0000", 2024-09-14T05:30:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6860000000000006e-06 2024-09-14T05:30:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561308+0000", 2024-09-14T05:30:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.023e-06 2024-09-14T05:30:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605893+0000", 2024-09-14T05:30:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044585458000000001 2024-09-14T05:30:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605933+0000", 2024-09-14T05:30:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9551999999999997e-05 2024-09-14T05:30:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606123+0000", 2024-09-14T05:30:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019046699999999999 2024-09-14T05:30:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74054 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.668318880000001, 2024-09-14T05:30:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093610999999999, 2024-09-14T05:30:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74054 2024-09-14T05:30:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9699999999999996e-07 2024-09-14T05:30:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561317+0000", 2024-09-14T05:30:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9740000000000001e-06 2024-09-14T05:30:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561324+0000", 2024-09-14T05:30:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4590000000000003e-06 2024-09-14T05:30:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561329+0000", 2024-09-14T05:30:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5969999999999999e-06 2024-09-14T05:30:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606142+0000", 2024-09-14T05:30:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044812347000000002 2024-09-14T05:30:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606184+0000", 2024-09-14T05:30:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2441999999999998e-05 2024-09-14T05:30:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606408+0000", 2024-09-14T05:30:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022429500000000001 2024-09-14T05:30:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74056 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.668124177000003, 2024-09-14T05:30:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045191685000000002, 2024-09-14T05:30:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74056 2024-09-14T05:30:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561510+0000", 2024-09-14T05:30:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2399999999999998e-07 2024-09-14T05:30:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561513+0000", 2024-09-14T05:30:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7219999999999999e-06 2024-09-14T05:30:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561524+0000", 2024-09-14T05:30:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1357e-05 2024-09-14T05:30:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561530+0000", 2024-09-14T05:30:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2890000000000003e-06 2024-09-14T05:30:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606438+0000", 2024-09-14T05:30:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044907326999999997 2024-09-14T05:30:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606520+0000", 2024-09-14T05:30:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1924999999999999e-05 2024-09-14T05:30:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606701+0000", 2024-09-14T05:30:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00018144100000000001 2024-09-14T05:30:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74058 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.667645503999999, 2024-09-14T05:30:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093368000000002, 2024-09-14T05:30:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74058 2024-09-14T05:30:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561989+0000", 2024-09-14T05:30:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3300000000000001e-07 2024-09-14T05:30:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561993+0000", 2024-09-14T05:30:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1790000000000001e-06 2024-09-14T05:30:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562007+0000", 2024-09-14T05:30:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3434e-05 2024-09-14T05:30:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562014+0000", 2024-09-14T05:30:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.537e-06 2024-09-14T05:30:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606736+0000", 2024-09-14T05:30:31.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044722117999999998 2024-09-14T05:30:31.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.316 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.316 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606829+0000", 2024-09-14T05:30:31.316 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2410999999999993e-05 2024-09-14T05:30:31.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607082+0000", 2024-09-14T05:30:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025285599999999998 2024-09-14T05:30:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74059 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.667587513000001, 2024-09-14T05:30:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045479928000000003, 2024-09-14T05:30:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74059 2024-09-14T05:30:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562047+0000", 2024-09-14T05:30:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6899999999999997e-07 2024-09-14T05:30:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562050+0000", 2024-09-14T05:30:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.943e-06 2024-09-14T05:30:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562058+0000", 2024-09-14T05:30:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0120000000000005e-06 2024-09-14T05:30:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562063+0000", 2024-09-14T05:30:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:30:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607186+0000", 2024-09-14T05:30:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045122348999999999 2024-09-14T05:30:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607265+0000", 2024-09-14T05:30:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9462999999999995e-05 2024-09-14T05:30:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607526+0000", 2024-09-14T05:30:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026090400000000001 2024-09-14T05:30:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74060 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2850816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 41.667562136000001, 2024-09-14T05:30:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045200092999999997, 2024-09-14T05:30:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74060 2024-09-14T05:30:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5200000000000002e-07 2024-09-14T05:30:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562075+0000", 2024-09-14T05:30:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5200000000000002e-06 2024-09-14T05:30:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562082+0000", 2024-09-14T05:30:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8929999999999999e-06 2024-09-14T05:30:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562088+0000", 2024-09-14T05:30:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5570000000000003e-06 2024-09-14T05:30:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607111+0000", 2024-09-14T05:30:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045023464999999999 2024-09-14T05:30:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607190+0000", 2024-09-14T05:30:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8721999999999996e-05 2024-09-14T05:30:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607272+0000", 2024-09-14T05:30:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1483999999999994e-05 2024-09-14T05:30:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.326 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:31.328 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:30:31.328 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:31.327+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:31.384 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T05:30:31.455 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:31.454+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:31.485 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:31.485 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:31.485 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:31.485 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:31.494 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:30:31.530 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:31.529+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:31.630 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:31.630+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:31.640 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:31.640 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:31.640 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:31.640 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:31.640 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:31.649 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:30:31.732 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:31.731+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:31.794 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:31.794 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:31.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:31.794 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:31.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.794 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4235.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-14T05:30:31.794 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:30:31.794 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 77.952837506999998, 2024-09-14T05:30:31.794 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12887758399999999, 2024-09-14T05:30:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:30:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4235", 2024-09-14T05:30:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/3211559243", 2024-09-14T05:30:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 2 2024-09-14T05:30:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:30:31.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:30:31.796 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.796 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.796 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839833+0000", 2024-09-14T05:30:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:30:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839838+0000", 2024-09-14T05:30:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2440000000000001e-06 2024-09-14T05:30:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839848+0000", 2024-09-14T05:30:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0030000000000001e-05 2024-09-14T05:30:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839854+0000", 2024-09-14T05:30:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3750000000000002e-06 2024-09-14T05:30:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839908+0000", 2024-09-14T05:30:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3965999999999998e-05 2024-09-14T05:30:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:30:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839914+0000", 2024-09-14T05:30:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6139999999999997e-06 2024-09-14T05:30:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968558+0000", 2024-09-14T05:30:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12864361199999999 2024-09-14T05:30:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968710+0000", 2024-09-14T05:30:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015177300000000001 2024-09-14T05:30:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27764 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 188416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:30:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.581511124999999, 2024-09-14T05:30:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080826920999999996, 2024-09-14T05:30:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27764 2024-09-14T05:30:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:30:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:30:31.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211159+0000", 2024-09-14T05:30:31.802 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1800000000000001e-07 2024-09-14T05:30:31.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.802 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.802 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211161+0000", 2024-09-14T05:30:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.968e-06 2024-09-14T05:30:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211169+0000", 2024-09-14T05:30:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4800000000000001e-06 2024-09-14T05:30:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211174+0000", 2024-09-14T05:30:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1880000000000001e-06 2024-09-14T05:30:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291670+0000", 2024-09-14T05:30:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080495199000000003 2024-09-14T05:30:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291775+0000", 2024-09-14T05:30:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000105375 2024-09-14T05:30:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291985+0000", 2024-09-14T05:30:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021029299999999999 2024-09-14T05:30:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27765 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:30:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.581488374999999, 2024-09-14T05:30:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081115197999999999, 2024-09-14T05:30:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.806 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.806 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.806 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27765 2024-09-14T05:30:31.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.806 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.806 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.806 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:30:31.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.807 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.807 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:30:31.807 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211182+0000", 2024-09-14T05:30:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3599999999999999e-07 2024-09-14T05:30:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211184+0000", 2024-09-14T05:30:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.03e-06 2024-09-14T05:30:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211192+0000", 2024-09-14T05:30:31.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2309999999999996e-06 2024-09-14T05:30:31.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211197+0000", 2024-09-14T05:30:31.809 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4199999999999998e-06 2024-09-14T05:30:31.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.809 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.809 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292017+0000", 2024-09-14T05:30:31.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080819516999999994 2024-09-14T05:30:31.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.810 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292087+0000", 2024-09-14T05:30:31.810 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0056999999999997e-05 2024-09-14T05:30:31.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.810 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.811 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292296+0000", 2024-09-14T05:30:31.811 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000209507 2024-09-14T05:30:31.811 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.811 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.811 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.811 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.811 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27766 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:30:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.581465160999997, 2024-09-14T05:30:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081359077000000002, 2024-09-14T05:30:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27766 2024-09-14T05:30:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:30:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:30:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211205+0000", 2024-09-14T05:30:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7e-07 2024-09-14T05:30:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211207+0000", 2024-09-14T05:30:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4540000000000001e-06 2024-09-14T05:30:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211216+0000", 2024-09-14T05:30:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3680000000000002e-06 2024-09-14T05:30:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211221+0000", 2024-09-14T05:30:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3499999999999996e-06 2024-09-14T05:30:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292329+0000", 2024-09-14T05:30:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081108240999999998 2024-09-14T05:30:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292370+0000", 2024-09-14T05:30:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.121e-05 2024-09-14T05:30:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292563+0000", 2024-09-14T05:30:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019298399999999999 2024-09-14T05:30:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27767 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.581441646999998, 2024-09-14T05:30:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081623119999999993, 2024-09-14T05:30:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27767 2024-09-14T05:30:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0499999999999999e-07 2024-09-14T05:30:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211230+0000", 2024-09-14T05:30:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0700000000000001e-06 2024-09-14T05:30:31.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.819 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211236+0000", 2024-09-14T05:30:31.819 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8429999999999998e-06 2024-09-14T05:30:31.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.819 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211241+0000", 2024-09-14T05:30:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:30:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292594+0000", 2024-09-14T05:30:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081352868999999994 2024-09-14T05:30:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292631+0000", 2024-09-14T05:30:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7083999999999999e-05 2024-09-14T05:30:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292851+0000", 2024-09-14T05:30:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022011700000000001 2024-09-14T05:30:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27768 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 274432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:30:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.581423110999999, 2024-09-14T05:30:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081875032, 2024-09-14T05:30:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27768 2024-09-14T05:30:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:30:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:30:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211247+0000", 2024-09-14T05:30:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.46e-07 2024-09-14T05:30:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211248+0000", 2024-09-14T05:30:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7349999999999999e-06 2024-09-14T05:30:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211253+0000", 2024-09-14T05:30:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.899e-06 2024-09-14T05:30:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211258+0000", 2024-09-14T05:30:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2810000000000002e-06 2024-09-14T05:30:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292888+0000", 2024-09-14T05:30:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081629955000000004 2024-09-14T05:30:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292926+0000", 2024-09-14T05:30:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7947000000000001e-05 2024-09-14T05:30:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293121+0000", 2024-09-14T05:30:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019586899999999999 2024-09-14T05:30:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27769 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 294912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:30:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.581405021999998, 2024-09-14T05:30:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082181899000000003, 2024-09-14T05:30:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.827 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.827 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27769 2024-09-14T05:30:31.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.827 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.827 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.827 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:30:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:30:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211265+0000", 2024-09-14T05:30:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2199999999999999e-07 2024-09-14T05:30:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211267+0000", 2024-09-14T05:30:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2239999999999998e-06 2024-09-14T05:30:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211273+0000", 2024-09-14T05:30:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6540000000000004e-06 2024-09-14T05:30:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211277+0000", 2024-09-14T05:30:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6299999999999997e-06 2024-09-14T05:30:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293150+0000", 2024-09-14T05:30:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081872575000000003 2024-09-14T05:30:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293245+0000", 2024-09-14T05:30:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5327000000000004e-05 2024-09-14T05:30:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293446+0000", 2024-09-14T05:30:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000201067 2024-09-14T05:30:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27770 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.58138486, 2024-09-14T05:30:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082663707000000003, 2024-09-14T05:30:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.832 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.832 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.832 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.832 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.832 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.832 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27770 2024-09-14T05:30:31.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.832 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:31.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:31.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1199999999999998e-07 2024-09-14T05:30:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211292+0000", 2024-09-14T05:30:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1609999999999997e-06 2024-09-14T05:30:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211299+0000", 2024-09-14T05:30:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3300000000000004e-06 2024-09-14T05:30:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211303+0000", 2024-09-14T05:30:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7859999999999996e-06 2024-09-14T05:30:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293628+0000", 2024-09-14T05:30:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082324283999999998 2024-09-14T05:30:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293694+0000", 2024-09-14T05:30:31.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6277000000000003e-05 2024-09-14T05:30:31.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293948+0000", 2024-09-14T05:30:31.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025445699999999998 2024-09-14T05:30:31.837 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.837 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.837 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27790 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:30:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.581358455999997, 2024-09-14T05:30:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082461006000000003, 2024-09-14T05:30:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27790 2024-09-14T05:30:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:30:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:30:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211312+0000", 2024-09-14T05:30:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9299999999999998e-07 2024-09-14T05:30:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211314+0000", 2024-09-14T05:30:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000002e-06 2024-09-14T05:30:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211320+0000", 2024-09-14T05:30:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1589999999999998e-06 2024-09-14T05:30:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211325+0000", 2024-09-14T05:30:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9609999999999996e-06 2024-09-14T05:30:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293474+0000", 2024-09-14T05:30:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082149080999999999 2024-09-14T05:30:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293542+0000", 2024-09-14T05:30:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8497000000000005e-05 2024-09-14T05:30:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293772+0000", 2024-09-14T05:30:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022969900000000001 2024-09-14T05:30:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27791 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:30:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.581338142, 2024-09-14T05:30:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082700838999999998, 2024-09-14T05:30:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27791 2024-09-14T05:30:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:30:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:30:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211332+0000", 2024-09-14T05:30:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7399999999999999e-07 2024-09-14T05:30:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211334+0000", 2024-09-14T05:30:31.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8500000000000001e-06 2024-09-14T05:30:31.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211340+0000", 2024-09-14T05:30:31.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3010000000000003e-06 2024-09-14T05:30:31.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211345+0000", 2024-09-14T05:30:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.938e-06 2024-09-14T05:30:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293797+0000", 2024-09-14T05:30:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082452152000000001 2024-09-14T05:30:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293869+0000", 2024-09-14T05:30:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2236000000000004e-05 2024-09-14T05:30:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294032+0000", 2024-09-14T05:30:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016298800000000001 2024-09-14T05:30:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27792 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.581318611, 2024-09-14T05:30:31.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082897304000000005, 2024-09-14T05:30:31.848 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.848 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.848 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.848 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.848 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.848 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.848 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27792 2024-09-14T05:30:31.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.849 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:31.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:31.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2000000000000001e-07 2024-09-14T05:30:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211353+0000", 2024-09-14T05:30:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.761e-06 2024-09-14T05:30:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211359+0000", 2024-09-14T05:30:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2430000000000002e-06 2024-09-14T05:30:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211364+0000", 2024-09-14T05:30:31.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0239999999999997e-06 2024-09-14T05:30:31.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.852 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294059+0000", 2024-09-14T05:30:31.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082694834999999994 2024-09-14T05:30:31.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294114+0000", 2024-09-14T05:30:31.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4823e-05 2024-09-14T05:30:31.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294248+0000", 2024-09-14T05:30:31.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013429800000000001 2024-09-14T05:30:31.853 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27793 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.581298812, 2024-09-14T05:30:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083153362999999994, 2024-09-14T05:30:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.855 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.855 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.855 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27793 2024-09-14T05:30:31.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.855 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:31.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:31.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.857 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.857 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:31.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5100000000000001e-07 2024-09-14T05:30:31.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211373+0000", 2024-09-14T05:30:31.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.001e-06 2024-09-14T05:30:31.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211379+0000", 2024-09-14T05:30:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4559999999999997e-06 2024-09-14T05:30:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211384+0000", 2024-09-14T05:30:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9749999999999998e-06 2024-09-14T05:30:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294279+0000", 2024-09-14T05:30:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082894282999999999 2024-09-14T05:30:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294327+0000", 2024-09-14T05:30:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8208000000000003e-05 2024-09-14T05:30:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294524+0000", 2024-09-14T05:30:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019708900000000001 2024-09-14T05:30:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27794 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.581278488000002, 2024-09-14T05:30:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588590000000004, 2024-09-14T05:30:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.862 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.862 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.862 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.862 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.862 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.862 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27794 2024-09-14T05:30:31.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.862 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:31.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:31.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:31.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4499999999999997e-07 2024-09-14T05:30:31.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211393+0000", 2024-09-14T05:30:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.815e-06 2024-09-14T05:30:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211400+0000", 2024-09-14T05:30:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2899999999999999e-06 2024-09-14T05:30:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211404+0000", 2024-09-14T05:30:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:30:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294560+0000", 2024-09-14T05:30:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083155534000000003 2024-09-14T05:30:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294631+0000", 2024-09-14T05:30:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1378999999999995e-05 2024-09-14T05:30:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294980+0000", 2024-09-14T05:30:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00034844900000000002 2024-09-14T05:30:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27795 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:30:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.581259129999999, 2024-09-14T05:30:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083970367000000004, 2024-09-14T05:30:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27795 2024-09-14T05:30:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:30:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:30:31.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211411+0000", 2024-09-14T05:30:31.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5499999999999999e-07 2024-09-14T05:30:31.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211413+0000", 2024-09-14T05:30:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:30:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211419+0000", 2024-09-14T05:30:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1720000000000004e-06 2024-09-14T05:30:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211424+0000", 2024-09-14T05:30:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0690000000000001e-06 2024-09-14T05:30:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295013+0000", 2024-09-14T05:30:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588898999999994 2024-09-14T05:30:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295102+0000", 2024-09-14T05:30:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9462000000000005e-05 2024-09-14T05:30:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295381+0000", 2024-09-14T05:30:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000278566 2024-09-14T05:30:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27796 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.581239400999998, 2024-09-14T05:30:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084379107999999994, 2024-09-14T05:30:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27796 2024-09-14T05:30:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2800000000000002e-07 2024-09-14T05:30:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211432+0000", 2024-09-14T05:30:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:30:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211439+0000", 2024-09-14T05:30:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5239999999999997e-06 2024-09-14T05:30:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211444+0000", 2024-09-14T05:30:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.95e-06 2024-09-14T05:30:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295602+0000", 2024-09-14T05:30:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084158017000000002 2024-09-14T05:30:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295674+0000", 2024-09-14T05:30:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2385999999999994e-05 2024-09-14T05:30:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295809+0000", 2024-09-14T05:30:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013495900000000001 2024-09-14T05:30:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.877 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27797 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 266240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.877 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:31.877 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.581219419, 2024-09-14T05:30:31.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084898597000000006, 2024-09-14T05:30:31.878 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.878 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.878 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.878 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.878 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.878 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.878 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27797 2024-09-14T05:30:31.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.879 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:31.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:30:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211452+0000", 2024-09-14T05:30:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.708e-06 2024-09-14T05:30:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211458+0000", 2024-09-14T05:30:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3489999999999997e-06 2024-09-14T05:30:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211464+0000", 2024-09-14T05:30:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3820000000000003e-06 2024-09-14T05:30:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295839+0000", 2024-09-14T05:30:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084375055000000004 2024-09-14T05:30:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295918+0000", 2024-09-14T05:30:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9253999999999997e-05 2024-09-14T05:30:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296349+0000", 2024-09-14T05:30:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043051 2024-09-14T05:30:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27798 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.581199703999999, 2024-09-14T05:30:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085300275999999994, 2024-09-14T05:30:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27798 2024-09-14T05:30:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211472+0000", 2024-09-14T05:30:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7579999999999999e-06 2024-09-14T05:30:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211478+0000", 2024-09-14T05:30:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0310000000000004e-06 2024-09-14T05:30:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211483+0000", 2024-09-14T05:30:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8110000000000002e-06 2024-09-14T05:30:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296469+0000", 2024-09-14T05:30:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084986575999999994 2024-09-14T05:30:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296545+0000", 2024-09-14T05:30:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5749999999999998e-05 2024-09-14T05:30:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296770+0000", 2024-09-14T05:30:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022498900000000001 2024-09-14T05:30:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27799 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 307200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.581180791000001, 2024-09-14T05:30:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085718461999999995, 2024-09-14T05:30:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27799 2024-09-14T05:30:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:31.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8000000000000001e-07 2024-09-14T05:30:31.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211491+0000", 2024-09-14T05:30:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-14T05:30:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211497+0000", 2024-09-14T05:30:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.857e-06 2024-09-14T05:30:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211502+0000", 2024-09-14T05:30:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9790000000000001e-06 2024-09-14T05:30:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296800+0000", 2024-09-14T05:30:31.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085298099000000002 2024-09-14T05:30:31.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296871+0000", 2024-09-14T05:30:31.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0879999999999999e-05 2024-09-14T05:30:31.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297207+0000", 2024-09-14T05:30:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033651699999999999 2024-09-14T05:30:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27826 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:30:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.579465669999998, 2024-09-14T05:30:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084306729999999996, 2024-09-14T05:30:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27826 2024-09-14T05:30:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:30:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:30:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213205+0000", 2024-09-14T05:30:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.277e-06 2024-09-14T05:30:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213210+0000", 2024-09-14T05:30:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3120000000000001e-06 2024-09-14T05:30:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213226+0000", 2024-09-14T05:30:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5279000000000001e-05 2024-09-14T05:30:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213234+0000", 2024-09-14T05:30:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7020000000000007e-06 2024-09-14T05:30:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297239+0000", 2024-09-14T05:30:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084004876000000006 2024-09-14T05:30:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297311+0000", 2024-09-14T05:30:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1336999999999998e-05 2024-09-14T05:30:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297511+0000", 2024-09-14T05:30:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019994699999999999 2024-09-14T05:30:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27827 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:30:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.579420483, 2024-09-14T05:30:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084520537000000007, 2024-09-14T05:30:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27827 2024-09-14T05:30:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:30:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:31.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:30:31.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:31.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:31.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213250+0000", 2024-09-14T05:30:31.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1099999999999995e-07 2024-09-14T05:30:31.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:31.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213252+0000", 2024-09-14T05:30:31.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6699999999999998e-06 2024-09-14T05:30:31.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:31.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213259+0000", 2024-09-14T05:30:31.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8469999999999998e-06 2024-09-14T05:30:31.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:31.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213265+0000", 2024-09-14T05:30:31.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4940000000000003e-06 2024-09-14T05:30:31.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.903 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:31.903 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297541+0000", 2024-09-14T05:30:31.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084276731999999993 2024-09-14T05:30:31.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.903 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297579+0000", 2024-09-14T05:30:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8108999999999997e-05 2024-09-14T05:30:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297770+0000", 2024-09-14T05:30:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019007400000000001 2024-09-14T05:30:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:31.905 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:31.906 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:30:31.907 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:31.905+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:32.006 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:32.006+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:32.022 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:32.022 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:32.031 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:30:32.107 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:32.107+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:32.177 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:32.177 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:32.177 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:32.177 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:32.177 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:32.181 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:30:32.185 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:30:32.226 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:32.225+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:32.291 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3523 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6062080~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.775164504000003, 2024-09-14T05:30:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096911030999999995, 2024-09-14T05:30:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3523 2024-09-14T05:30:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514359+0000", 2024-09-14T05:30:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9299999999999999e-07 2024-09-14T05:30:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514361+0000", 2024-09-14T05:30:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514366+0000", 2024-09-14T05:30:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1660000000000002e-06 2024-09-14T05:30:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514371+0000", 2024-09-14T05:30:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.648e-06 2024-09-14T05:30:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.610954+0000", 2024-09-14T05:30:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096582170999999994 2024-09-14T05:30:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611002+0000", 2024-09-14T05:30:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7923999999999997e-05 2024-09-14T05:30:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611269+0000", 2024-09-14T05:30:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026789499999999998 2024-09-14T05:30:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3524 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6553600~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.775145311999999, 2024-09-14T05:30:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097263347999999999, 2024-09-14T05:30:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3524 2024-09-14T05:30:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514380+0000", 2024-09-14T05:30:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6500000000000001e-06 2024-09-14T05:30:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514386+0000", 2024-09-14T05:30:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7360000000000004e-06 2024-09-14T05:30:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514391+0000", 2024-09-14T05:30:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2830000000000003e-06 2024-09-14T05:30:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611295+0000", 2024-09-14T05:30:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096904814000000006 2024-09-14T05:30:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611343+0000", 2024-09-14T05:30:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7506999999999998e-05 2024-09-14T05:30:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611641+0000", 2024-09-14T05:30:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029798800000000001 2024-09-14T05:30:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3525 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7045120~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.775123424999997, 2024-09-14T05:30:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097644422999999994, 2024-09-14T05:30:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3525 2024-09-14T05:30:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514400+0000", 2024-09-14T05:30:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.77e-07 2024-09-14T05:30:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514402+0000", 2024-09-14T05:30:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6530000000000001e-06 2024-09-14T05:30:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514411+0000", 2024-09-14T05:30:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7429999999999997e-06 2024-09-14T05:30:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514417+0000", 2024-09-14T05:30:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2249999999999997e-06 2024-09-14T05:30:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611668+0000", 2024-09-14T05:30:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097250681000000005 2024-09-14T05:30:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611716+0000", 2024-09-14T05:30:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7336000000000002e-05 2024-09-14T05:30:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612044+0000", 2024-09-14T05:30:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032840799999999998 2024-09-14T05:30:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3526 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7536640~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.775098941000003, 2024-09-14T05:30:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097943884999999994, 2024-09-14T05:30:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3526 2024-09-14T05:30:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0400000000000002e-07 2024-09-14T05:30:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514426+0000", 2024-09-14T05:30:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514432+0000", 2024-09-14T05:30:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.105e-06 2024-09-14T05:30:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514438+0000", 2024-09-14T05:30:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5149999999999997e-06 2024-09-14T05:30:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612072+0000", 2024-09-14T05:30:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097633947999999998 2024-09-14T05:30:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612120+0000", 2024-09-14T05:30:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8458999999999998e-05 2024-09-14T05:30:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612368+0000", 2024-09-14T05:30:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024761999999999999 2024-09-14T05:30:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3527 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 8028160~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.775078227999998, 2024-09-14T05:30:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098244955999999994, 2024-09-14T05:30:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3527 2024-09-14T05:30:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519842+0000", 2024-09-14T05:30:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053975230000000004 2024-09-14T05:30:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519851+0000", 2024-09-14T05:30:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5599999999999994e-06 2024-09-14T05:30:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519858+0000", 2024-09-14T05:30:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2409999999999998e-06 2024-09-14T05:30:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519863+0000", 2024-09-14T05:30:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:30:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612395+0000", 2024-09-14T05:30:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092532552000000004 2024-09-14T05:30:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612442+0000", 2024-09-14T05:30:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6486000000000002e-05 2024-09-14T05:30:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612690+0000", 2024-09-14T05:30:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024791599999999999 2024-09-14T05:30:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3603 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5177344~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.769525782999999, 2024-09-14T05:30:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096796436, 2024-09-14T05:30:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3603 2024-09-14T05:30:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1899999999999999e-07 2024-09-14T05:30:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519998+0000", 2024-09-14T05:30:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5000000000000001e-07 2024-09-14T05:30:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520002+0000", 2024-09-14T05:30:32.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7299999999999999e-06 2024-09-14T05:30:32.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520004+0000", 2024-09-14T05:30:32.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.429e-06 2024-09-14T05:30:32.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616493+0000", 2024-09-14T05:30:32.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096488345000000003 2024-09-14T05:30:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616544+0000", 2024-09-14T05:30:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1724000000000001e-05 2024-09-14T05:30:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616794+0000", 2024-09-14T05:30:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000249139 2024-09-14T05:30:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3604 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5668864~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.769511145000003, 2024-09-14T05:30:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108222999999994, 2024-09-14T05:30:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3604 2024-09-14T05:30:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7000000000000001e-07 2024-09-14T05:30:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520013+0000", 2024-09-14T05:30:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.389e-06 2024-09-14T05:30:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520018+0000", 2024-09-14T05:30:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3019999999999996e-06 2024-09-14T05:30:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520026+0000", 2024-09-14T05:30:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6170000000000007e-06 2024-09-14T05:30:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616816+0000", 2024-09-14T05:30:32.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096789766999999999 2024-09-14T05:30:32.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616871+0000", 2024-09-14T05:30:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4811000000000001e-05 2024-09-14T05:30:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617120+0000", 2024-09-14T05:30:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024906700000000001 2024-09-14T05:30:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3605 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6160384~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.769492685000003, 2024-09-14T05:30:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097423945999999997, 2024-09-14T05:30:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3605 2024-09-14T05:30:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3999999999999998e-07 2024-09-14T05:30:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520032+0000", 2024-09-14T05:30:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.083e-06 2024-09-14T05:30:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520036+0000", 2024-09-14T05:30:32.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.019e-06 2024-09-14T05:30:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520038+0000", 2024-09-14T05:30:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4849999999999999e-06 2024-09-14T05:30:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617147+0000", 2024-09-14T05:30:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108665999999996 2024-09-14T05:30:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617199+0000", 2024-09-14T05:30:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2558000000000001e-05 2024-09-14T05:30:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617454+0000", 2024-09-14T05:30:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025489499999999999 2024-09-14T05:30:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3606 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6651904~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.769481962999997, 2024-09-14T05:30:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097745705000000002, 2024-09-14T05:30:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3606 2024-09-14T05:30:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-14T05:30:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520042+0000", 2024-09-14T05:30:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4799999999999997e-07 2024-09-14T05:30:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520046+0000", 2024-09-14T05:30:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8149999999999999e-06 2024-09-14T05:30:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520049+0000", 2024-09-14T05:30:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.576e-06 2024-09-14T05:30:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617476+0000", 2024-09-14T05:30:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097427558999999997 2024-09-14T05:30:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617522+0000", 2024-09-14T05:30:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5927999999999998e-05 2024-09-14T05:30:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617787+0000", 2024-09-14T05:30:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000264663 2024-09-14T05:30:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3607 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7143424~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.769471436000003, 2024-09-14T05:30:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097871125000000003, 2024-09-14T05:30:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3607 2024-09-14T05:30:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520052+0000", 2024-09-14T05:30:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.03e-07 2024-09-14T05:30:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520053+0000", 2024-09-14T05:30:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.85e-07 2024-09-14T05:30:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520056+0000", 2024-09-14T05:30:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9140000000000001e-06 2024-09-14T05:30:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520059+0000", 2024-09-14T05:30:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4600000000000002e-06 2024-09-14T05:30:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617809+0000", 2024-09-14T05:30:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097750216000000001 2024-09-14T05:30:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617864+0000", 2024-09-14T05:30:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5211999999999999e-05 2024-09-14T05:30:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617923+0000", 2024-09-14T05:30:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8235000000000001e-05 2024-09-14T05:30:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3608 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7634944~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.769461128000003, 2024-09-14T05:30:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098199212999999994, 2024-09-14T05:30:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.355 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.355 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3608 2024-09-14T05:30:32.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.355 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:32.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:32.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525953+0000", 2024-09-14T05:30:32.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058911850000000002 2024-09-14T05:30:32.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525957+0000", 2024-09-14T05:30:32.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2379999999999999e-06 2024-09-14T05:30:32.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525967+0000", 2024-09-14T05:30:32.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5389999999999996e-06 2024-09-14T05:30:32.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525972+0000", 2024-09-14T05:30:32.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.541e-06 2024-09-14T05:30:32.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617945+0000", 2024-09-14T05:30:32.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091972597000000003 2024-09-14T05:30:32.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617992+0000", 2024-09-14T05:30:32.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7397e-05 2024-09-14T05:30:32.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.618261+0000", 2024-09-14T05:30:32.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026871600000000001 2024-09-14T05:30:32.359 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.360 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.360 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.360 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3640 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6586368~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.360 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:32.360 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.763353285999997, 2024-09-14T05:30:32.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097068580000000002, 2024-09-14T05:30:32.360 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.360 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.360 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.361 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.361 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.361 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3640 2024-09-14T05:30:32.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.361 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:32.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:32.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:32.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.23e-07 2024-09-14T05:30:32.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526171+0000", 2024-09-14T05:30:32.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:30:32.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526175+0000", 2024-09-14T05:30:32.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8369999999999999e-06 2024-09-14T05:30:32.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526177+0000", 2024-09-14T05:30:32.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4959999999999999e-06 2024-09-14T05:30:32.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622847+0000", 2024-09-14T05:30:32.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096669790000000005 2024-09-14T05:30:32.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622902+0000", 2024-09-14T05:30:32.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5157000000000001e-05 2024-09-14T05:30:32.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623238+0000", 2024-09-14T05:30:32.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033603399999999997 2024-09-14T05:30:32.365 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.365 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.365 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.365 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3641 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7077888~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.365 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:32.365 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.763342776000002, 2024-09-14T05:30:32.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097371141999999994, 2024-09-14T05:30:32.366 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.366 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.366 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.366 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3641 2024-09-14T05:30:32.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.366 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:32.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:32.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531851+0000", 2024-09-14T05:30:32.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005671265 2024-09-14T05:30:32.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531855+0000", 2024-09-14T05:30:32.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0849999999999999e-06 2024-09-14T05:30:32.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531867+0000", 2024-09-14T05:30:32.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1028e-05 2024-09-14T05:30:32.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531872+0000", 2024-09-14T05:30:32.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000003e-06 2024-09-14T05:30:32.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623251+0000", 2024-09-14T05:30:32.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091378980999999998 2024-09-14T05:30:32.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623295+0000", 2024-09-14T05:30:32.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3881999999999998e-05 2024-09-14T05:30:32.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623551+0000", 2024-09-14T05:30:32.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025636099999999998 2024-09-14T05:30:32.370 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.370 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.370 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.370 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3721 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.370 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:32.370 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.757484501, 2024-09-14T05:30:32.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096806472000000005, 2024-09-14T05:30:32.371 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.371 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.371 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.371 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.371 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.371 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.371 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3721 2024-09-14T05:30:32.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.371 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:32.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:32.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532039+0000", 2024-09-14T05:30:32.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.05e-07 2024-09-14T05:30:32.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532040+0000", 2024-09-14T05:30:32.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9099999999999991e-07 2024-09-14T05:30:32.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532043+0000", 2024-09-14T05:30:32.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6610000000000002e-06 2024-09-14T05:30:32.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532046+0000", 2024-09-14T05:30:32.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4080000000000001e-06 2024-09-14T05:30:32.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628545+0000", 2024-09-14T05:30:32.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096499615999999996 2024-09-14T05:30:32.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628592+0000", 2024-09-14T05:30:32.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6907000000000003e-05 2024-09-14T05:30:32.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628845+0000", 2024-09-14T05:30:32.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025268399999999998 2024-09-14T05:30:32.375 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.375 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.375 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.376 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3722 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6193152~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.376 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:32.376 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.757474420999998, 2024-09-14T05:30:32.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097146587000000006, 2024-09-14T05:30:32.376 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.376 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.376 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.376 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3722 2024-09-14T05:30:32.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.376 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:32.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:32.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538191+0000", 2024-09-14T05:30:32.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061429969999999999 2024-09-14T05:30:32.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538200+0000", 2024-09-14T05:30:32.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7069999999999998e-06 2024-09-14T05:30:32.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538212+0000", 2024-09-14T05:30:32.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1504e-05 2024-09-14T05:30:32.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538219+0000", 2024-09-14T05:30:32.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7500000000000003e-06 2024-09-14T05:30:32.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628869+0000", 2024-09-14T05:30:32.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090649224 2024-09-14T05:30:32.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628919+0000", 2024-09-14T05:30:32.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0144e-05 2024-09-14T05:30:32.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.629195+0000", 2024-09-14T05:30:32.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027626099999999998 2024-09-14T05:30:32.380 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.381 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.381 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.381 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3758 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7110656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.381 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:32.381 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.750880205000001, 2024-09-14T05:30:32.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096977544999999998, 2024-09-14T05:30:32.381 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.381 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.381 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.382 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.382 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3758 2024-09-14T05:30:32.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.382 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:32.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:32.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:32.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:32.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538645+0000", 2024-09-14T05:30:32.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.843e-06 2024-09-14T05:30:32.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538651+0000", 2024-09-14T05:30:32.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6169999999999998e-06 2024-09-14T05:30:32.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538655+0000", 2024-09-14T05:30:32.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5399999999999997e-06 2024-09-14T05:30:32.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635225+0000", 2024-09-14T05:30:32.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096570149999999993 2024-09-14T05:30:32.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635272+0000", 2024-09-14T05:30:32.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7199e-05 2024-09-14T05:30:32.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635620+0000", 2024-09-14T05:30:32.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034782600000000003 2024-09-14T05:30:32.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.386 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.386 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3759 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.386 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:32.386 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.750862294000001, 2024-09-14T05:30:32.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097335331999999997, 2024-09-14T05:30:32.386 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.386 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.387 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.387 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3759 2024-09-14T05:30:32.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.387 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:32.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:32.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544013+0000", 2024-09-14T05:30:32.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053520829999999997 2024-09-14T05:30:32.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544016+0000", 2024-09-14T05:30:32.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6660000000000001e-06 2024-09-14T05:30:32.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544026+0000", 2024-09-14T05:30:32.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7089999999999997e-06 2024-09-14T05:30:32.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544043+0000", 2024-09-14T05:30:32.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833999999999998e-05 2024-09-14T05:30:32.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635646+0000", 2024-09-14T05:30:32.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091603544999999995 2024-09-14T05:30:32.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635696+0000", 2024-09-14T05:30:32.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9307000000000001e-05 2024-09-14T05:30:32.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635996+0000", 2024-09-14T05:30:32.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030018800000000001 2024-09-14T05:30:32.391 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.391 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.391 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.391 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3838 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6225920~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.391 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:32.391 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.745200613999998, 2024-09-14T05:30:32.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096894235999999995, 2024-09-14T05:30:32.391 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.391 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.391 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.392 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.392 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.392 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.392 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3838 2024-09-14T05:30:32.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.392 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:32.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:32.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544323+0000", 2024-09-14T05:30:32.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-14T05:30:32.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544325+0000", 2024-09-14T05:30:32.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.75e-06 2024-09-14T05:30:32.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544331+0000", 2024-09-14T05:30:32.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5019999999999998e-06 2024-09-14T05:30:32.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544335+0000", 2024-09-14T05:30:32.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2989999999999998e-06 2024-09-14T05:30:32.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640869+0000", 2024-09-14T05:30:32.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096533351000000003 2024-09-14T05:30:32.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640919+0000", 2024-09-14T05:30:32.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999999e-05 2024-09-14T05:30:32.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641217+0000", 2024-09-14T05:30:32.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029717600000000002 2024-09-14T05:30:32.396 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.396 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.396 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.396 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3839 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.396 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:32.396 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.745181676000001, 2024-09-14T05:30:32.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097240339999999995, 2024-09-14T05:30:32.397 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.397 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.397 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.397 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.397 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.397 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.397 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3839 2024-09-14T05:30:32.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.397 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:32.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:32.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549834+0000", 2024-09-14T05:30:32.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054927819999999999 2024-09-14T05:30:32.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549838+0000", 2024-09-14T05:30:32.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8689999999999997e-06 2024-09-14T05:30:32.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549848+0000", 2024-09-14T05:30:32.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0154e-05 2024-09-14T05:30:32.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549853+0000", 2024-09-14T05:30:32.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9740000000000001e-06 2024-09-14T05:30:32.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641245+0000", 2024-09-14T05:30:32.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091392245999999996 2024-09-14T05:30:32.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641295+0000", 2024-09-14T05:30:32.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0018999999999997e-05 2024-09-14T05:30:32.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641582+0000", 2024-09-14T05:30:32.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028629600000000001 2024-09-14T05:30:32.401 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.401 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.401 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.401 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3868 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:32.401 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:32.402 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.739519098000002, 2024-09-14T05:30:32.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096859555999999999, 2024-09-14T05:30:32.402 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:32.402 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:32.402 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:32.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:32.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:32.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:32.402 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3868 2024-09-14T05:30:32.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.402 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:32.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:32.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:32.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:32.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:32.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:32.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:32.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556173+0000", 2024-09-14T05:30:32.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006169038 2024-09-14T05:30:32.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:32.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556177+0000", 2024-09-14T05:30:32.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0010000000000003e-06 2024-09-14T05:30:32.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:32.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556188+0000", 2024-09-14T05:30:32.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0675e-05 2024-09-14T05:30:32.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:32.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556199+0000", 2024-09-14T05:30:32.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.155e-05 2024-09-14T05:30:32.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:32.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646511+0000", 2024-09-14T05:30:32.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090312187000000002 2024-09-14T05:30:32.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:32.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646563+0000", 2024-09-14T05:30:32.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2219000000000003e-05 2024-09-14T05:30:32.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:32.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:32.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:32.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646863+0000", 2024-09-14T05:30:32.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029988600000000002 2024-09-14T05:30:32.406 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.406 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.406 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.406 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:32.406 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:32.406 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:32.407 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T05:30:32.407 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T05:30:32.407 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T05:30:32.407 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T05:30:32.408 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:32.409 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:32.409 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:32.409 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:32.410 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:30:32.410 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:32.411 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:32.411 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:32.412 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:32.412 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:32.412 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:32.412 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:32.413 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:32.413 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-14T05:30:32.414 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:32.407+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:32.414 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:32.414 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:32.429 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:32.429 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:32.508 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:32.508+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:32.526 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:32.526 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:32.535 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:30:32.609 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:32.608+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:32.639 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:32.639 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:32.647 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:30:32.708 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:30:32.708 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:32.708 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 83, 2024-09-14T05:30:32.708 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "1127be44-5f14-4e7d-ad23-f608afe154f5", 2024-09-14T05:30:32.708 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2024-09-14T05:28:54.203072+0000", 2024-09-14T05:30:32.708 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-14T05:30:32.391857+0000", 2024-09-14T05:30:32.708 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2024-09-14T05:29:06.755531+0000", 2024-09-14T05:30:32.708 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2024-09-14T05:28:56.535048+0000", 2024-09-14T05:30:32.708 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T05:30:32.709 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2024-09-14T05:30:32.709 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2024-09-14T05:30:32.709 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2024-09-14T05:30:32.709 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2024-09-14T05:30:32.709 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2024-09-14T05:30:32.709 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2024-09-14T05:30:32.709 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:30:32.710 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 6, 2024-09-14T05:30:32.710 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T05:30:32.710 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T05:30:32.710 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T05:30:32.710 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2024-09-14T05:30:32.710 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 3, 2024-09-14T05:30:32.710 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2024-09-14T05:30:32.710 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2024-09-14T05:30:32.710 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "luminous", 2024-09-14T05:30:32.710 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "squid", 2024-09-14T05:30:32.711 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2024-09-14T05:30:32.711 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2024-09-14T05:30:32.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.711 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2024-09-14T05:30:32.711 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2024-09-14T05:30:32.711 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-14T05:29:06.452744+0000", 2024-09-14T05:30:32.711 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-14T05:30:32.711 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2024-09-14T05:30:32.711 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-14T05:30:32.711 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-14T05:30:32.711 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-14T05:30:32.711 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-14T05:30:32.712 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-14T05:30:32.712 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-14T05:30:32.712 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T05:30:32.712 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T05:30:32.712 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-14T05:30:32.712 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-14T05:30:32.712 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-14T05:30:32.712 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2024-09-14T05:30:32.712 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2024-09-14T05:30:32.713 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2024-09-14T05:30:32.713 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2024-09-14T05:30:32.713 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2024-09-14T05:30:32.713 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-14T05:30:32.713 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-14T05:30:32.713 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-14T05:30:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-14T05:30:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-14T05:30:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-14T05:30:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-14T05:30:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "18", 2024-09-14T05:30:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-14T05:30:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T05:30:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T05:30:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-14T05:30:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-14T05:30:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2024-09-14T05:30:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2024-09-14T05:30:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-14T05:30:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-14T05:30:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-14T05:30:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-14T05:30:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-14T05:30:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-14T05:30:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-14T05:30:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-14T05:30:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-14T05:30:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-14T05:30:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-14T05:30:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T05:30:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T05:30:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T05:30:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-14T05:30:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-14T05:30:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-14T05:30:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-14T05:30:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-14T05:30:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-14T05:30:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-14T05:30:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-14T05:30:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-14T05:30:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-14T05:30:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T05:30:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-14T05:30:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-14T05:30:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-14T05:30:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-14T05:30:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-14T05:30:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2024-09-14T05:30:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2024-09-14T05:30:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2024-09-14T05:30:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.718 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-14T05:30:32.718 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2024-09-14T05:30:32.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.718 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-14T05:30:32.718 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-14T05:30:32.718 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2024-09-14T05:30:32.718 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2024-09-14T05:30:32.718 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2024-09-14T05:30:32.718 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2024-09-14T05:30:32.718 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2024-09-14T05:30:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.87708091735839844, 2024-09-14T05:30:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T05:30:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T05:30:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 2, 2024-09-14T05:30:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "rbd", 2024-09-14T05:30:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-14T05:29:12.380843+0000", 2024-09-14T05:30:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 8193, 2024-09-14T05:30:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T05:30:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-14T05:30:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-14T05:30:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-14T05:30:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-14T05:30:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-14T05:30:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-14T05:30:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T05:30:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T05:30:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-14T05:30:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-14T05:30:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-14T05:30:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 8, 2024-09-14T05:30:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 8, 2024-09-14T05:30:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 8, 2024-09-14T05:30:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 8, 2024-09-14T05:30:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 8, 2024-09-14T05:30:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-14T05:30:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-14T05:30:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-14T05:30:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-14T05:30:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-14T05:30:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-14T05:30:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-14T05:30:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "22", 2024-09-14T05:30:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-14T05:30:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T05:30:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T05:30:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-14T05:30:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-14T05:30:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 2, 2024-09-14T05:30:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 22, 2024-09-14T05:30:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-14T05:30:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-14T05:30:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-14T05:30:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-14T05:30:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-14T05:30:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-14T05:30:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-14T05:30:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-14T05:30:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-14T05:30:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-14T05:30:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-14T05:30:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T05:30:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T05:30:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T05:30:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-14T05:30:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-14T05:30:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-14T05:30:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-14T05:30:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-14T05:30:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-14T05:30:32.724 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-14T05:30:32.724 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-14T05:30:32.724 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-14T05:30:32.724 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-14T05:30:32.724 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T05:30:32.724 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-14T05:30:32.724 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-14T05:30:32.724 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-14T05:30:32.725 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-14T05:30:32.725 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-14T05:30:32.725 INFO:teuthology.orchestra.run.smithi071.stdout: "options": {}, 2024-09-14T05:30:32.725 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-14T05:30:32.725 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd": {} 2024-09-14T05:30:32.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.726 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-14T05:30:32.726 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-14T05:30:32.726 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 4.9800000190734863, 2024-09-14T05:30:32.726 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 4.9800000190734863, 2024-09-14T05:30:32.726 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T05:30:32.726 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 4.380000114440918, 2024-09-14T05:30:32.726 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 4.380000114440918, 2024-09-14T05:30:32.726 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.87708091735839844, 2024-09-14T05:30:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T05:30:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T05:30:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 3, 2024-09-14T05:30:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "pool_client.0", 2024-09-14T05:30:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2024-09-14T05:29:22.335419+0000", 2024-09-14T05:30:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 8193, 2024-09-14T05:30:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T05:30:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2024-09-14T05:30:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2024-09-14T05:30:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2024-09-14T05:30:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2024-09-14T05:30:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2024-09-14T05:30:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2024-09-14T05:30:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T05:30:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T05:30:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "is_stretch_pool": false, 2024-09-14T05:30:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2024-09-14T05:30:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2024-09-14T05:30:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2024-09-14T05:30:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2024-09-14T05:30:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2024-09-14T05:30:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2024-09-14T05:30:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2024-09-14T05:30:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2024-09-14T05:30:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2024-09-14T05:30:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2024-09-14T05:30:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2024-09-14T05:30:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2024-09-14T05:30:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2024-09-14T05:30:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2024-09-14T05:30:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "83", 2024-09-14T05:30:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2024-09-14T05:30:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T05:30:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T05:30:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2024-09-14T05:30:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2024-09-14T05:30:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 40, 2024-09-14T05:30:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 83, 2024-09-14T05:30:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2024-09-14T05:30:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2024-09-14T05:30:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2024-09-14T05:30:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2024-09-14T05:30:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2024-09-14T05:30:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2024-09-14T05:30:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2024-09-14T05:30:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2024-09-14T05:30:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2024-09-14T05:30:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2024-09-14T05:30:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2024-09-14T05:30:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T05:30:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T05:30:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T05:30:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2024-09-14T05:30:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2024-09-14T05:30:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2024-09-14T05:30:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2024-09-14T05:30:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2024-09-14T05:30:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2024-09-14T05:30:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2024-09-14T05:30:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2024-09-14T05:30:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2024-09-14T05:30:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2024-09-14T05:30:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T05:30:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2024-09-14T05:30:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2024-09-14T05:30:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2024-09-14T05:30:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2024-09-14T05:30:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2024-09-14T05:30:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "options": {}, 2024-09-14T05:30:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2024-09-14T05:30:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd": {} 2024-09-14T05:30:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2024-09-14T05:30:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: "score_type": "Fair distribution", 2024-09-14T05:30:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2024-09-14T05:30:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2024-09-14T05:30:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2024-09-14T05:30:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2024-09-14T05:30:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2024-09-14T05:30:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 0.87708091735839844, 2024-09-14T05:30:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T05:30:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T05:30:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:30:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2024-09-14T05:30:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2024-09-14T05:30:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "e7c7817f-4a65-4f66-9af5-d31d0a92cf30", 2024-09-14T05:30:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:30:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:30:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:30:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:30:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:30:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:30:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 16, 2024-09-14T05:30:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:30:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:30:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:30:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:30:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2024-09-14T05:30:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:30:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2024-09-14T05:30:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:30:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:30:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2024-09-14T05:30:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:30:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2024-09-14T05:30:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:30:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:30:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2024-09-14T05:30:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:30:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2024-09-14T05:30:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:30:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:30:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2024-09-14T05:30:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:30:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2024-09-14T05:30:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2926109885 2024-09-14T05:30:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6819/2926109885", 2024-09-14T05:30:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6821/2926109885", 2024-09-14T05:30:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6825/2926109885", 2024-09-14T05:30:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6823/2926109885", 2024-09-14T05:30:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:30:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:30:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:30:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2024-09-14T05:30:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "e184604e-b973-4b7a-bba2-6f471753a0fd", 2024-09-14T05:30:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:30:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:30:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:30:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:30:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:30:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:30:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:30:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:30:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:30:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:30:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:30:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2024-09-14T05:30:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:30:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2024-09-14T05:30:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:30:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:30:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2024-09-14T05:30:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:30:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2024-09-14T05:30:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:30:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:30:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6832", 2024-09-14T05:30:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:30:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6833", 2024-09-14T05:30:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:30:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:30:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2024-09-14T05:30:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:30:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2024-09-14T05:30:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 345649648 2024-09-14T05:30:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6827/345649648", 2024-09-14T05:30:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6829/345649648", 2024-09-14T05:30:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6833/345649648", 2024-09-14T05:30:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6831/345649648", 2024-09-14T05:30:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:30:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:30:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:30:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2024-09-14T05:30:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "d8b70f09-5afb-4296-b8c9-719c172e0e70", 2024-09-14T05:30:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:30:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:30:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:30:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:30:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:30:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:30:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:30:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 81, 2024-09-14T05:30:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:30:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:30:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:30:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2024-09-14T05:30:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:30:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2024-09-14T05:30:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:30:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:30:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.749 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.749 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2024-09-14T05:30:32.749 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:30:32.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.749 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.749 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2024-09-14T05:30:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:30:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:30:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2024-09-14T05:30:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:30:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2024-09-14T05:30:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:30:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:30:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2024-09-14T05:30:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:30:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2024-09-14T05:30:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2721860691 2024-09-14T05:30:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/2721860691", 2024-09-14T05:30:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/2721860691", 2024-09-14T05:30:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/2721860691", 2024-09-14T05:30:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/2721860691", 2024-09-14T05:30:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:30:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:30:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:30:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2024-09-14T05:30:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c", 2024-09-14T05:30:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:30:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:30:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:30:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:30:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:30:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:30:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:30:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 39, 2024-09-14T05:30:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:30:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:30:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:30:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2024-09-14T05:30:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:30:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2024-09-14T05:30:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:30:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:30:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2024-09-14T05:30:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:30:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2024-09-14T05:30:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:30:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:30:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2024-09-14T05:30:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:30:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2024-09-14T05:30:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:30:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:30:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2024-09-14T05:30:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:30:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2024-09-14T05:30:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 494607463 2024-09-14T05:30:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/494607463", 2024-09-14T05:30:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6811/494607463", 2024-09-14T05:30:32.759 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6815/494607463", 2024-09-14T05:30:32.759 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6813/494607463", 2024-09-14T05:30:32.759 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:30:32.759 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:30:32.759 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:30:32.759 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.759 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2024-09-14T05:30:32.759 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "515cb7d6-16eb-4aff-973a-7f7ed49325ba", 2024-09-14T05:30:32.760 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:30:32.760 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:30:32.760 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:30:32.760 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:30:32.760 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:30:32.760 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:30:32.760 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:30:32.760 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 80, 2024-09-14T05:30:32.760 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:30:32.760 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:30:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:30:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6816", 2024-09-14T05:30:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:30:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6817", 2024-09-14T05:30:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:30:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:30:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6818", 2024-09-14T05:30:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:30:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6819", 2024-09-14T05:30:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:30:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:30:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6822", 2024-09-14T05:30:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:30:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6823", 2024-09-14T05:30:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:30:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:30:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6820", 2024-09-14T05:30:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:30:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6821", 2024-09-14T05:30:32.765 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2240133417 2024-09-14T05:30:32.765 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.765 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.765 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6817/2240133417", 2024-09-14T05:30:32.765 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6819/2240133417", 2024-09-14T05:30:32.765 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6823/2240133417", 2024-09-14T05:30:32.765 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6821/2240133417", 2024-09-14T05:30:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:30:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:30:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:30:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2024-09-14T05:30:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "c434b53a-04be-45b9-b89a-193654133950", 2024-09-14T05:30:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:30:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:30:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:30:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:30:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:30:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:30:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:30:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 39, 2024-09-14T05:30:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:30:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:30:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:30:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6808", 2024-09-14T05:30:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:30:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6809", 2024-09-14T05:30:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:30:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:30:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6810", 2024-09-14T05:30:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:30:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6811", 2024-09-14T05:30:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:30:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:30:32.770 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.770 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.770 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6814", 2024-09-14T05:30:32.770 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:30:32.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.770 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.770 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6815", 2024-09-14T05:30:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:30:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:30:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6812", 2024-09-14T05:30:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:30:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6813", 2024-09-14T05:30:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2393825223 2024-09-14T05:30:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6809/2393825223", 2024-09-14T05:30:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6811/2393825223", 2024-09-14T05:30:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6815/2393825223", 2024-09-14T05:30:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6813/2393825223", 2024-09-14T05:30:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:30:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:30:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:30:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2024-09-14T05:30:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "3d2f8405-9286-4b40-8acd-036f67373ac6", 2024-09-14T05:30:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:30:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:30:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:30:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 0.0166473388671875, 2024-09-14T05:30:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:30:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:30:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:30:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:30:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:30:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:30:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:30:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6800", 2024-09-14T05:30:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:30:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6801", 2024-09-14T05:30:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:30:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:30:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6802", 2024-09-14T05:30:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:30:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6803", 2024-09-14T05:30:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:30:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:30:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6806", 2024-09-14T05:30:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:30:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6807", 2024-09-14T05:30:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:30:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:30:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6804", 2024-09-14T05:30:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:30:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6805", 2024-09-14T05:30:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3235729544 2024-09-14T05:30:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6801/3235729544", 2024-09-14T05:30:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6803/3235729544", 2024-09-14T05:30:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6807/3235729544", 2024-09-14T05:30:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6805/3235729544", 2024-09-14T05:30:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:30:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:30:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:30:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2024-09-14T05:30:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "6724f080-4dc3-465a-939f-b023985bf998", 2024-09-14T05:30:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-14T05:30:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-14T05:30:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2024-09-14T05:30:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2024-09-14T05:30:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2024-09-14T05:30:32.780 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2024-09-14T05:30:32.780 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 15, 2024-09-14T05:30:32.780 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 24, 2024-09-14T05:30:32.780 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2024-09-14T05:30:32.780 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2024-09-14T05:30:32.780 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2024-09-14T05:30:32.780 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.780 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6824", 2024-09-14T05:30:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:30:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6825", 2024-09-14T05:30:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:30:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2024-09-14T05:30:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6826", 2024-09-14T05:30:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:30:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6827", 2024-09-14T05:30:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:30:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2024-09-14T05:30:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6830", 2024-09-14T05:30:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:30:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6831", 2024-09-14T05:30:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:30:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2024-09-14T05:30:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-14T05:30:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-14T05:30:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6828", 2024-09-14T05:30:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:30:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-14T05:30:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.150:6829", 2024-09-14T05:30:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2381001355 2024-09-14T05:30:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.150:6825/2381001355", 2024-09-14T05:30:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.150:6827/2381001355", 2024-09-14T05:30:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.150:6831/2381001355", 2024-09-14T05:30:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.150:6829/2381001355", 2024-09-14T05:30:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2024-09-14T05:30:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2024-09-14T05:30:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2024-09-14T05:30:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:30:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2024-09-14T05:30:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2024-09-14T05:30:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:30:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:30:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:30:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:30:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:30:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.548803+0000", 2024-09-14T05:30:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:30:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2024-09-14T05:30:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:30:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:30:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:30:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:30:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:30:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.075023+0000", 2024-09-14T05:30:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:30:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2024-09-14T05:30:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:30:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:30:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:30:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:30:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:30:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.117036+0000", 2024-09-14T05:30:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:30:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2024-09-14T05:30:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:30:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:30:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:30:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:30:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:30:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.332113+0000", 2024-09-14T05:30:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:30:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2024-09-14T05:30:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:30:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:30:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:30:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:30:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:30:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.330371+0000", 2024-09-14T05:30:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:30:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2024-09-14T05:30:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:30:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:30:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:30:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:30:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:30:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:02.753946+0000", 2024-09-14T05:30:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:30:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.791 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2024-09-14T05:30:32.791 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:30:32.791 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:30:32.791 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:30:32.791 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:30:32.791 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:30:32.791 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:02.071186+0000", 2024-09-14T05:30:32.791 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:30:32.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2024-09-14T05:30:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2024-09-14T05:30:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2024-09-14T05:30:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2024-09-14T05:30:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540719139924082687, 2024-09-14T05:30:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2024-09-14T05:30:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2024-09-14T05:29:03.378781+0000", 2024-09-14T05:30:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2024-09-14T05:30:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:30:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [ 2024-09-14T05:30:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "3.7", 2024-09-14T05:30:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2024-09-14T05:30:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: 2, 2024-09-14T05:30:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: 4 2024-09-14T05:30:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:30:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2024-09-14T05:30:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2024-09-14T05:30:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2024-09-14T05:30:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2024-09-14T05:30:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2024-09-14T05:30:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2024-09-14T05:30:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2024-09-14T05:30:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2024-09-14T05:30:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2024-09-14T05:30:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2024-09-14T05:30:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2024-09-14T05:30:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2024-09-14T05:30:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2024-09-14T05:30:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [ 2024-09-14T05:30:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 3, 2024-09-14T05:30:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: "snaps": [ 2024-09-14T05:30:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: "begin": 33, 2024-09-14T05:30:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: "length": 8 2024-09-14T05:30:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:30:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [ 2024-09-14T05:30:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 3, 2024-09-14T05:30:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: "snaps": [ 2024-09-14T05:30:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: "begin": 33, 2024-09-14T05:30:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: "length": 1 2024-09-14T05:30:32.796 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.796 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:32.796 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.796 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-14T05:30:32.796 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2024-09-14T05:30:32.796 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2024-09-14T05:30:32.796 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2024-09-14T05:30:32.797 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2024-09-14T05:30:32.797 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2024-09-14T05:30:32.797 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2024-09-14T05:30:32.797 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2024-09-14T05:30:32.797 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2024-09-14T05:30:32.797 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2024-09-14T05:30:32.797 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:32.797 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:32.799 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:32.799 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:32.800 INFO:tasks.thrashosds.thrasher:j is {'epoch': 83, 'fsid': '1127be44-5f14-4e7d-ad23-f608afe154f5', 'created': '2024-09-14T05:28:54.203072+0000', 'modified': '2024-09-14T05:30:32.391857+0000', 'last_up_change': '2024-09-14T05:29:06.755531+0000', 'last_in_change': '2024-09-14T05:28:56.535048+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T05:29:06.452744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8770809173583984, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T05:29:12.380843+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.8770809173583984, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T05:29:22.335419+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 40, 'snap_epoch': 83, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8770809173583984, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'e7c7817f-4a65-4f66-9af5-d31d0a92cf30', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6818', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6819', 'nonce': 2926109885}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6820', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6821', 'nonce': 2926109885}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6824', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6825', 'nonce': 2926109885}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6822', 'nonce': 2926109885}, {'type': 'v1', 'addr': '172.21.15.71:6823', 'nonce': 2926109885}]}, 'public_addr': '172.21.15.71:6819/2926109885', 'cluster_addr': '172.21.15.71:6821/2926109885', 'heartbeat_back_addr': '172.21.15.71:6825/2926109885', 'heartbeat_front_addr': '172.21.15.71:6823/2926109885', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e184604e-b973-4b7a-bba2-6f471753a0fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6826', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6827', 'nonce': 345649648}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6828', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6829', 'nonce': 345649648}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6832', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6833', 'nonce': 345649648}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6830', 'nonce': 345649648}, {'type': 'v1', 'addr': '172.21.15.71:6831', 'nonce': 345649648}]}, 'public_addr': '172.21.15.71:6827/345649648', 'cluster_addr': '172.21.15.71:6829/345649648', 'heartbeat_back_addr': '172.21.15.71:6833/345649648', 'heartbeat_front_addr': '172.21.15.71:6831/345649648', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd8b70f09-5afb-4296-b8c9-719c172e0e70', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6800', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6801', 'nonce': 2721860691}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6802', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6803', 'nonce': 2721860691}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6806', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6807', 'nonce': 2721860691}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6804', 'nonce': 2721860691}, {'type': 'v1', 'addr': '172.21.15.71:6805', 'nonce': 2721860691}]}, 'public_addr': '172.21.15.71:6801/2721860691', 'cluster_addr': '172.21.15.71:6803/2721860691', 'heartbeat_back_addr': '172.21.15.71:6807/2721860691', 'heartbeat_front_addr': '172.21.15.71:6805/2721860691', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6808', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6809', 'nonce': 494607463}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6810', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6811', 'nonce': 494607463}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6814', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6815', 'nonce': 494607463}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6812', 'nonce': 494607463}, {'type': 'v1', 'addr': '172.21.15.71:6813', 'nonce': 494607463}]}, 'public_addr': '172.21.15.71:6809/494607463', 'cluster_addr': '172.21.15.71:6811/494607463', 'heartbeat_back_addr': '172.21.15.71:6815/494607463', 'heartbeat_front_addr': '172.21.15.71:6813/494607463', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '515cb7d6-16eb-4aff-973a-7f7ed49325ba', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 80, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6816', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6817', 'nonce': 2240133417}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6818', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6819', 'nonce': 2240133417}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6822', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6823', 'nonce': 2240133417}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6820', 'nonce': 2240133417}, {'type': 'v1', 'addr': '172.21.15.150:6821', 'nonce': 2240133417}]}, 'public_addr': '172.21.15.150:6817/2240133417', 'cluster_addr': '172.21.15.150:6819/2240133417', 'heartbeat_back_addr': '172.21.15.150:6823/2240133417', 'heartbeat_front_addr': '172.21.15.150:6821/2240133417', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'c434b53a-04be-45b9-b89a-193654133950', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6808', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6809', 'nonce': 2393825223}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6810', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6811', 'nonce': 2393825223}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6814', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6815', 'nonce': 2393825223}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6812', 'nonce': 2393825223}, {'type': 'v1', 'addr': '172.21.15.150:6813', 'nonce': 2393825223}]}, 'public_addr': '172.21.15.150:6809/2393825223', 'cluster_addr': '172.21.15.150:6811/2393825223', 'heartbeat_back_addr': '172.21.15.150:6815/2393825223', 'heartbeat_front_addr': '172.21.15.150:6813/2393825223', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '3d2f8405-9286-4b40-8acd-036f67373ac6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0166473388671875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6800', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6801', 'nonce': 3235729544}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6802', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6803', 'nonce': 3235729544}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6806', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6807', 'nonce': 3235729544}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6804', 'nonce': 3235729544}, {'type': 'v1', 'addr': '172.21.15.150:6805', 'nonce': 3235729544}]}, 'public_addr': '172.21.15.150:6801/3235729544', 'cluster_addr': '172.21.15.150:6803/3235729544', 'heartbeat_back_addr': '172.21.15.150:6807/3235729544', 'heartbeat_front_addr': '172.21.15.150:6805/3235729544', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '6724f080-4dc3-465a-939f-b023985bf998', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6824', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6825', 'nonce': 2381001355}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6826', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6827', 'nonce': 2381001355}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6830', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6831', 'nonce': 2381001355}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6828', 'nonce': 2381001355}, {'type': 'v1', 'addr': '172.21.15.150:6829', 'nonce': 2381001355}]}, 'public_addr': '172.21.15.150:6825/2381001355', 'cluster_addr': '172.21.15.150:6827/2381001355', 'heartbeat_back_addr': '172.21.15.150:6831/2381001355', 'heartbeat_front_addr': '172.21.15.150:6829/2381001355', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.548803+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.075023+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.117036+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.332113+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.330371+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:02.753946+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:02.071186+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T05:29:03.378781+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.7', 'osds': [2, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 33, 'length': 8}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 33, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T05:30:32.800 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:30:32.801 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:30:32.801 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:32.798+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:32.899 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:32.899+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:32.944 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:32.945 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:32.953 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:30:33.000 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:33.000+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:33.095 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:33.095 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:33.101 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:33.101+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:33.103 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:30:33.120 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:30:33.120 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:30:33.131 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-14T05:30:32.449537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1061450959,"num_objects":1872,"num_object_clones":481,"num_object_copies":3744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1872,"num_whiteouts":1001,"num_read":174580,"num_read_kb":558781,"num_write":19462,"num_write_kb":282055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":648894,"num_keys_recovered":33,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16787,"ondisk_log_size":16787,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":590948,"kb_used_data":270620,"kb_used_omap":12,"kb_used_meta":320115,"kb_avail":754383772,"statfs":{"total":773094113280,"available":772488982528,"internally_reserved":0,"allocated":277114880,"data_stored":282176515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":327798320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8444391,"num_objects":-34,"num_object_clones":-11,"num_object_copies":-68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-34,"num_whiteouts":-9,"num_read":602,"num_read_kb":508,"num_write":83,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":648894,"num_keys_recovered":33,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000563"},"pg_stats":[{"pgid":"3.1f","version":"77'679","reported_seq":10530,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.422124+0000","last_change":"2024-09-14T05:30:30.395624+0000","last_active":"2024-09-14T05:30:31.422124+0000","last_peered":"2024-09-14T05:30:31.422124+0000","last_clean":"2024-09-14T05:30:31.422124+0000","last_became_active":"2024-09-14T05:29:24.018892+0000","last_became_peered":"2024-09-14T05:29:24.018892+0000","last_unstale":"2024-09-14T05:30:31.422124+0000","last_undegraded":"2024-09-14T05:30:31.422124+0000","last_fullsized":"2024-09-14T05:30:31.422124+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:53:39.919084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042100999999999998,"stat_sum":{"num_bytes":42568241,"num_objects":62,"num_object_clones":15,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":32,"num_read":11934,"num_read_kb":37115,"num_write":776,"num_write_kb":31140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.1e","version":"78'362","reported_seq":3001,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.422091+0000","last_change":"2024-09-14T05:30:30.395894+0000","last_active":"2024-09-14T05:30:31.422091+0000","last_peered":"2024-09-14T05:30:31.422091+0000","last_clean":"2024-09-14T05:30:31.422091+0000","last_became_active":"2024-09-14T05:29:24.014468+0000","last_became_peered":"2024-09-14T05:29:24.014468+0000","last_unstale":"2024-09-14T05:30:31.422091+0000","last_undegraded":"2024-09-14T05:30:31.422091+0000","last_fullsized":"2024-09-14T05:30:31.422091+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:28:12.408675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069726800000000002,"stat_sum":{"num_bytes":53702062,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":27,"num_read":2764,"num_read_kb":32470,"num_write":381,"num_write_kb":21093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.1d","version":"77'998","reported_seq":9584,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.393632+0000","last_change":"2024-09-14T05:30:30.390285+0000","last_active":"2024-09-14T05:30:31.393632+0000","last_peered":"2024-09-14T05:30:31.393632+0000","last_clean":"2024-09-14T05:30:31.393632+0000","last_became_active":"2024-09-14T05:29:24.014251+0000","last_became_peered":"2024-09-14T05:29:24.014251+0000","last_unstale":"2024-09-14T05:30:31.393632+0000","last_undegraded":"2024-09-14T05:30:31.393632+0000","last_fullsized":"2024-09-14T05:30:31.393632+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":998,"log_dups_size":0,"ondisk_log_size":998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:44:11.057499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029717300000000001,"stat_sum":{"num_bytes":53832863,"num_objects":60,"num_object_clones":20,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":31,"num_read":12251,"num_read_kb":23813,"num_write":1544,"num_write_kb":7077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.1c","version":"82'466","reported_seq":9918,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.403712+0000","last_change":"2024-09-14T05:30:30.396284+0000","last_active":"2024-09-14T05:30:31.403712+0000","last_peered":"2024-09-14T05:30:31.403712+0000","last_clean":"2024-09-14T05:30:31.403712+0000","last_became_active":"2024-09-14T05:29:24.012427+0000","last_became_peered":"2024-09-14T05:29:24.012427+0000","last_unstale":"2024-09-14T05:30:31.403712+0000","last_undegraded":"2024-09-14T05:30:31.403712+0000","last_fullsized":"2024-09-14T05:30:31.403712+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:46:20.280378+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0037953779999999999,"stat_sum":{"num_bytes":62039655,"num_objects":71,"num_object_clones":19,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":37,"num_read":10380,"num_read_kb":34695,"num_write":448,"num_write_kb":4621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.1b","version":"82'496","reported_seq":3141,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.463752+0000","last_change":"2024-09-14T05:30:30.388326+0000","last_active":"2024-09-14T05:30:31.463752+0000","last_peered":"2024-09-14T05:30:31.463752+0000","last_clean":"2024-09-14T05:30:31.463752+0000","last_became_active":"2024-09-14T05:29:24.012079+0000","last_became_peered":"2024-09-14T05:29:24.012079+0000","last_unstale":"2024-09-14T05:30:31.463752+0000","last_undegraded":"2024-09-14T05:30:31.463752+0000","last_fullsized":"2024-09-14T05:30:31.463752+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:39:35.851530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087073300000000005,"stat_sum":{"num_bytes":37003506,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":34,"num_read":3905,"num_read_kb":25782,"num_write":524,"num_write_kb":21851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.1a","version":"82'916","reported_seq":5861,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.452281+0000","last_change":"2024-09-14T05:30:30.395783+0000","last_active":"2024-09-14T05:30:31.452281+0000","last_peered":"2024-09-14T05:30:31.452281+0000","last_clean":"2024-09-14T05:30:31.452281+0000","last_became_active":"2024-09-14T05:29:24.012461+0000","last_became_peered":"2024-09-14T05:29:24.012461+0000","last_unstale":"2024-09-14T05:30:31.452281+0000","last_undegraded":"2024-09-14T05:30:31.452281+0000","last_fullsized":"2024-09-14T05:30:31.452281+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":916,"log_dups_size":0,"ondisk_log_size":916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:03:01.312944+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.002969789,"stat_sum":{"num_bytes":26774382,"num_objects":67,"num_object_clones":17,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":35,"num_read":10073,"num_read_kb":34074,"num_write":1223,"num_write_kb":8448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.19","version":"75'514","reported_seq":1914,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.397152+0000","last_change":"2024-09-14T05:30:30.387956+0000","last_active":"2024-09-14T05:30:31.397152+0000","last_peered":"2024-09-14T05:30:31.397152+0000","last_clean":"2024-09-14T05:30:31.397152+0000","last_became_active":"2024-09-14T05:29:24.018753+0000","last_became_peered":"2024-09-14T05:29:24.018753+0000","last_unstale":"2024-09-14T05:30:31.397152+0000","last_undegraded":"2024-09-14T05:30:31.397152+0000","last_fullsized":"2024-09-14T05:30:31.397152+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:05:26.487814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063216900000000003,"stat_sum":{"num_bytes":40471085,"num_objects":70,"num_object_clones":19,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":31,"num_read":2562,"num_read_kb":13567,"num_write":660,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.18","version":"77'575","reported_seq":2316,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.422229+0000","last_change":"2024-09-14T05:30:30.396209+0000","last_active":"2024-09-14T05:30:31.422229+0000","last_peered":"2024-09-14T05:30:31.422229+0000","last_clean":"2024-09-14T05:30:31.422229+0000","last_became_active":"2024-09-14T05:29:24.011176+0000","last_became_peered":"2024-09-14T05:29:24.011176+0000","last_unstale":"2024-09-14T05:30:31.422229+0000","last_undegraded":"2024-09-14T05:30:31.422229+0000","last_fullsized":"2024-09-14T05:30:31.422229+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:02:20.633951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074608699999999999,"stat_sum":{"num_bytes":12323793,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":25,"num_read":4031,"num_read_kb":6915,"num_write":615,"num_write_kb":3400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.17","version":"75'302","reported_seq":2799,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.422134+0000","last_change":"2024-09-14T05:30:30.395946+0000","last_active":"2024-09-14T05:30:31.422134+0000","last_peered":"2024-09-14T05:30:31.422134+0000","last_clean":"2024-09-14T05:30:31.422134+0000","last_became_active":"2024-09-14T05:29:24.014005+0000","last_became_peered":"2024-09-14T05:29:24.014005+0000","last_unstale":"2024-09-14T05:30:31.422134+0000","last_undegraded":"2024-09-14T05:30:31.422134+0000","last_fullsized":"2024-09-14T05:30:31.422134+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:35:54.301580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073028199999999996,"stat_sum":{"num_bytes":7394881,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":22,"num_read":2969,"num_read_kb":11190,"num_write":288,"num_write_kb":5639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.16","version":"77'296","reported_seq":9629,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.395640+0000","last_change":"2024-09-14T05:30:30.390615+0000","last_active":"2024-09-14T05:30:31.395640+0000","last_peered":"2024-09-14T05:30:31.395640+0000","last_clean":"2024-09-14T05:30:31.395640+0000","last_became_active":"2024-09-14T05:29:24.013434+0000","last_became_peered":"2024-09-14T05:29:24.013434+0000","last_unstale":"2024-09-14T05:30:31.395640+0000","last_undegraded":"2024-09-14T05:30:31.395640+0000","last_fullsized":"2024-09-14T05:30:31.395640+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:09:06.913024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094596800000000005,"stat_sum":{"num_bytes":53492260,"num_objects":63,"num_object_clones":18,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":37,"num_read":9226,"num_read_kb":22920,"num_write":288,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.15","version":"77'369","reported_seq":2330,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T05:30:27.371249+0000","last_change":"2024-09-14T05:30:27.371577+0000","last_active":"2024-09-14T05:30:27.371249+0000","last_peered":"2024-09-14T05:30:27.371249+0000","last_clean":"2024-09-14T05:30:27.371249+0000","last_became_active":"2024-09-14T05:29:24.014457+0000","last_became_peered":"2024-09-14T05:29:24.014457+0000","last_unstale":"2024-09-14T05:30:27.371249+0000","last_undegraded":"2024-09-14T05:30:27.371249+0000","last_fullsized":"2024-09-14T05:30:27.371249+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:19:02.244545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086303200000000001,"stat_sum":{"num_bytes":71540267,"num_objects":60,"num_object_clones":19,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":29,"num_read":2400,"num_read_kb":9169,"num_write":388,"num_write_kb":12869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"23","length":"6"}]},{"pgid":"3.14","version":"81'785","reported_seq":2204,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.393753+0000","last_change":"2024-09-14T05:30:30.394471+0000","last_active":"2024-09-14T05:30:31.393753+0000","last_peered":"2024-09-14T05:30:31.393753+0000","last_clean":"2024-09-14T05:30:31.393753+0000","last_became_active":"2024-09-14T05:29:24.013266+0000","last_became_peered":"2024-09-14T05:29:24.013266+0000","last_unstale":"2024-09-14T05:30:31.393753+0000","last_undegraded":"2024-09-14T05:30:31.393753+0000","last_fullsized":"2024-09-14T05:30:31.393753+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:40:29.896273+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074387239999999999,"stat_sum":{"num_bytes":14862751,"num_objects":51,"num_object_clones":8,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":33,"num_read":4166,"num_read_kb":5461,"num_write":1036,"num_write_kb":4015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"2.7","version":"0'0","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.393979+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:30:31.393979+0000","last_peered":"2024-09-14T05:30:31.393979+0000","last_clean":"2024-09-14T05:30:31.393979+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:30:31.393979+0000","last_undegraded":"2024-09-14T05:30:31.393979+0000","last_fullsized":"2024-09-14T05:30:31.393979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"82'290","reported_seq":1903,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.397330+0000","last_change":"2024-09-14T05:30:30.389597+0000","last_active":"2024-09-14T05:30:31.397330+0000","last_peered":"2024-09-14T05:30:31.397330+0000","last_clean":"2024-09-14T05:30:31.397330+0000","last_became_active":"2024-09-14T05:29:24.012099+0000","last_became_peered":"2024-09-14T05:29:24.012099+0000","last_unstale":"2024-09-14T05:30:31.397330+0000","last_undegraded":"2024-09-14T05:30:31.397330+0000","last_fullsized":"2024-09-14T05:30:31.397330+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:34:56.044434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020561020000000002,"stat_sum":{"num_bytes":7263431,"num_objects":46,"num_object_clones":14,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":27,"num_read":1938,"num_read_kb":3548,"num_write":263,"num_write_kb":5452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"2.6","version":"0'0","reported_seq":138,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.397252+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:30:31.397252+0000","last_peered":"2024-09-14T05:30:31.397252+0000","last_clean":"2024-09-14T05:30:31.397252+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:30:31.397252+0000","last_undegraded":"2024-09-14T05:30:31.397252+0000","last_fullsized":"2024-09-14T05:30:31.397252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"82'814","reported_seq":3412,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.401282+0000","last_change":"2024-09-14T05:30:31.401282+0000","last_active":"2024-09-14T05:30:31.401282+0000","last_peered":"2024-09-14T05:30:31.401282+0000","last_clean":"2024-09-14T05:30:31.401282+0000","last_became_active":"2024-09-14T05:30:31.398064+0000","last_became_peered":"2024-09-14T05:30:31.398064+0000","last_unstale":"2024-09-14T05:30:31.401282+0000","last_undegraded":"2024-09-14T05:30:31.401282+0000","last_fullsized":"2024-09-14T05:30:31.401282+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:17:19.456468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010325449999999999,"stat_sum":{"num_bytes":44366422,"num_objects":57,"num_object_clones":12,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":28,"num_read":6150,"num_read_kb":7881,"num_write":993,"num_write_kb":5135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":648894,"num_keys_recovered":33,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"2.5","version":"0'0","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.394012+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:30:31.394012+0000","last_peered":"2024-09-14T05:30:31.394012+0000","last_clean":"2024-09-14T05:30:31.394012+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:30:31.394012+0000","last_undegraded":"2024-09-14T05:30:31.394012+0000","last_fullsized":"2024-09-14T05:30:31.394012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"82'502","reported_seq":2384,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.466080+0000","last_change":"2024-09-14T05:30:30.395646+0000","last_active":"2024-09-14T05:30:31.466080+0000","last_peered":"2024-09-14T05:30:31.466080+0000","last_clean":"2024-09-14T05:30:31.466080+0000","last_became_active":"2024-09-14T05:29:40.095441+0000","last_became_peered":"2024-09-14T05:29:40.095441+0000","last_unstale":"2024-09-14T05:30:31.466080+0000","last_undegraded":"2024-09-14T05:30:31.466080+0000","last_fullsized":"2024-09-14T05:30:31.466080+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:20:41.187799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045598500000000001,"stat_sum":{"num_bytes":20088036,"num_objects":61,"num_object_clones":15,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":36,"num_read":3356,"num_read_kb":12341,"num_write":581,"num_write_kb":11383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"2.4","version":"0'0","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.422146+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:30:31.422146+0000","last_peered":"2024-09-14T05:30:31.422146+0000","last_clean":"2024-09-14T05:30:31.422146+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:30:31.422146+0000","last_undegraded":"2024-09-14T05:30:31.422146+0000","last_fullsized":"2024-09-14T05:30:31.422146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"73'289","reported_seq":1627,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.395604+0000","last_change":"2024-09-14T05:30:30.390035+0000","last_active":"2024-09-14T05:30:31.395604+0000","last_peered":"2024-09-14T05:30:31.395604+0000","last_clean":"2024-09-14T05:30:31.395604+0000","last_became_active":"2024-09-14T05:29:24.016940+0000","last_became_peered":"2024-09-14T05:29:24.016940+0000","last_unstale":"2024-09-14T05:30:31.395604+0000","last_undegraded":"2024-09-14T05:30:31.395604+0000","last_fullsized":"2024-09-14T05:30:31.395604+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:12:59.471081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045225000000000002,"stat_sum":{"num_bytes":11629336,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":27,"num_read":1526,"num_read_kb":4357,"num_write":295,"num_write_kb":3991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.3","version":"78'455","reported_seq":2442,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T05:30:27.459000+0000","last_change":"2024-09-14T05:30:27.371564+0000","last_active":"2024-09-14T05:30:27.459000+0000","last_peered":"2024-09-14T05:30:27.459000+0000","last_clean":"2024-09-14T05:30:27.459000+0000","last_became_active":"2024-09-14T05:29:24.015905+0000","last_became_peered":"2024-09-14T05:29:24.015905+0000","last_unstale":"2024-09-14T05:30:27.459000+0000","last_undegraded":"2024-09-14T05:30:27.459000+0000","last_fullsized":"2024-09-14T05:30:27.459000+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:21:05.286401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001503554,"stat_sum":{"num_bytes":12433279,"num_objects":72,"num_object_clones":21,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":42,"num_read":2319,"num_read_kb":9502,"num_write":493,"num_write_kb":7676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"23","length":"6"}]},{"pgid":"2.2","version":"22'2","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.393775+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:30:31.393775+0000","last_peered":"2024-09-14T05:30:31.393775+0000","last_clean":"2024-09-14T05:30:31.393775+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:30:31.393775+0000","last_undegraded":"2024-09-14T05:30:31.393775+0000","last_fullsized":"2024-09-14T05:30:31.393775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"77'389","reported_seq":10415,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.421973+0000","last_change":"2024-09-14T05:30:30.396142+0000","last_active":"2024-09-14T05:30:31.421973+0000","last_peered":"2024-09-14T05:30:31.421973+0000","last_clean":"2024-09-14T05:30:31.421973+0000","last_became_active":"2024-09-14T05:29:24.011764+0000","last_became_peered":"2024-09-14T05:29:24.011764+0000","last_unstale":"2024-09-14T05:30:31.421973+0000","last_undegraded":"2024-09-14T05:30:31.421973+0000","last_fullsized":"2024-09-14T05:30:31.421973+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:50:30.962508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000924522,"stat_sum":{"num_bytes":42046776,"num_objects":60,"num_object_clones":14,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":31,"num_read":11248,"num_read_kb":38521,"num_write":354,"num_write_kb":4943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"2.1","version":"0'0","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.394339+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:30:31.394339+0000","last_peered":"2024-09-14T05:30:31.394339+0000","last_clean":"2024-09-14T05:30:31.394339+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:30:31.394339+0000","last_undegraded":"2024-09-14T05:30:31.394339+0000","last_fullsized":"2024-09-14T05:30:31.394339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'632","reported_seq":1627,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.395494+0000","last_change":"2024-09-14T05:30:30.393886+0000","last_active":"2024-09-14T05:30:31.395494+0000","last_peered":"2024-09-14T05:30:31.395494+0000","last_clean":"2024-09-14T05:30:31.395494+0000","last_became_active":"2024-09-14T05:29:40.095255+0000","last_became_peered":"2024-09-14T05:29:40.095255+0000","last_unstale":"2024-09-14T05:30:31.395494+0000","last_undegraded":"2024-09-14T05:30:31.395494+0000","last_fullsized":"2024-09-14T05:30:31.395494+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:53:25.518619+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0047185430000000004,"stat_sum":{"num_bytes":49125355,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":24,"num_read":2916,"num_read_kb":7079,"num_write":879,"num_write_kb":10022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"2.0","version":"0'0","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.393560+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:30:31.393560+0000","last_peered":"2024-09-14T05:30:31.393560+0000","last_clean":"2024-09-14T05:30:31.393560+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:30:31.393560+0000","last_undegraded":"2024-09-14T05:30:31.393560+0000","last_fullsized":"2024-09-14T05:30:31.393560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"76'510","reported_seq":2362,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.397143+0000","last_change":"2024-09-14T05:30:30.388125+0000","last_active":"2024-09-14T05:30:31.397143+0000","last_peered":"2024-09-14T05:30:31.397143+0000","last_clean":"2024-09-14T05:30:31.397143+0000","last_became_active":"2024-09-14T05:29:24.015792+0000","last_became_peered":"2024-09-14T05:29:24.015792+0000","last_unstale":"2024-09-14T05:30:31.397143+0000","last_undegraded":"2024-09-14T05:30:31.397143+0000","last_fullsized":"2024-09-14T05:30:31.397143+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:19:16.056124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077776799999999997,"stat_sum":{"num_bytes":16760193,"num_objects":67,"num_object_clones":16,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":40,"num_read":3312,"num_read_kb":9280,"num_write":580,"num_write_kb":5620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"2.3","version":"20'1","reported_seq":141,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.393654+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:30:31.393654+0000","last_peered":"2024-09-14T05:30:31.393654+0000","last_clean":"2024-09-14T05:30:31.393654+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:30:31.393654+0000","last_undegraded":"2024-09-14T05:30:31.393654+0000","last_fullsized":"2024-09-14T05:30:31.393654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":231,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.402356+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:30:31.402356+0000","last_peered":"2024-09-14T05:30:31.402356+0000","last_clean":"2024-09-14T05:30:31.402356+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:30:31.402356+0000","last_undegraded":"2024-09-14T05:30:31.402356+0000","last_fullsized":"2024-09-14T05:30:31.402356+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"81'357","reported_seq":1993,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.397137+0000","last_change":"2024-09-14T05:30:30.391494+0000","last_active":"2024-09-14T05:30:31.397137+0000","last_peered":"2024-09-14T05:30:31.397137+0000","last_clean":"2024-09-14T05:30:31.397137+0000","last_became_active":"2024-09-14T05:29:24.012222+0000","last_became_peered":"2024-09-14T05:29:24.012222+0000","last_unstale":"2024-09-14T05:30:31.397137+0000","last_undegraded":"2024-09-14T05:30:31.397137+0000","last_fullsized":"2024-09-14T05:30:31.397137+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:25:00.645164+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0041534240000000002,"stat_sum":{"num_bytes":26536058,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":26,"num_read":2820,"num_read_kb":6024,"num_write":313,"num_write_kb":3878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.9","version":"82'571","reported_seq":2143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.466639+0000","last_change":"2024-09-14T05:30:30.390508+0000","last_active":"2024-09-14T05:30:31.466639+0000","last_peered":"2024-09-14T05:30:31.466639+0000","last_clean":"2024-09-14T05:30:31.466639+0000","last_became_active":"2024-09-14T05:29:24.012041+0000","last_became_peered":"2024-09-14T05:29:24.012041+0000","last_unstale":"2024-09-14T05:30:31.466639+0000","last_undegraded":"2024-09-14T05:30:31.466639+0000","last_fullsized":"2024-09-14T05:30:31.466639+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:39:18.283904+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0039771950000000002,"stat_sum":{"num_bytes":12460018,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":24,"num_read":2877,"num_read_kb":7328,"num_write":728,"num_write_kb":6418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.a","version":"82'540","reported_seq":2222,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.431957+0000","last_change":"2024-09-14T05:30:30.388894+0000","last_active":"2024-09-14T05:30:31.431957+0000","last_peered":"2024-09-14T05:30:31.431957+0000","last_clean":"2024-09-14T05:30:31.431957+0000","last_became_active":"2024-09-14T05:29:24.011403+0000","last_became_peered":"2024-09-14T05:29:24.011403+0000","last_unstale":"2024-09-14T05:30:31.431957+0000","last_undegraded":"2024-09-14T05:30:31.431957+0000","last_fullsized":"2024-09-14T05:30:31.431957+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:30:45.384954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022304999999999998,"stat_sum":{"num_bytes":18545837,"num_objects":63,"num_object_clones":18,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":30,"num_read":3620,"num_read_kb":7752,"num_write":637,"num_write_kb":4965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.b","version":"82'507","reported_seq":9582,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.421872+0000","last_change":"2024-09-14T05:30:30.390558+0000","last_active":"2024-09-14T05:30:31.421872+0000","last_peered":"2024-09-14T05:30:31.421872+0000","last_clean":"2024-09-14T05:30:31.421872+0000","last_became_active":"2024-09-14T05:29:24.006836+0000","last_became_peered":"2024-09-14T05:29:24.006836+0000","last_unstale":"2024-09-14T05:30:31.421872+0000","last_undegraded":"2024-09-14T05:30:31.421872+0000","last_fullsized":"2024-09-14T05:30:31.421872+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:25:47.774852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000518006,"stat_sum":{"num_bytes":37134862,"num_objects":63,"num_object_clones":10,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":35,"num_read":11262,"num_read_kb":29168,"num_write":597,"num_write_kb":13044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.c","version":"77'400","reported_seq":3241,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T05:30:27.369811+0000","last_change":"2024-09-14T05:30:27.370521+0000","last_active":"2024-09-14T05:30:27.369811+0000","last_peered":"2024-09-14T05:30:27.369811+0000","last_clean":"2024-09-14T05:30:27.369811+0000","last_became_active":"2024-09-14T05:29:24.012574+0000","last_became_peered":"2024-09-14T05:29:24.012574+0000","last_unstale":"2024-09-14T05:30:27.369811+0000","last_undegraded":"2024-09-14T05:30:27.369811+0000","last_fullsized":"2024-09-14T05:30:27.369811+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:32:00.583626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001011482,"stat_sum":{"num_bytes":39953671,"num_objects":60,"num_object_clones":20,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":32,"num_read":2807,"num_read_kb":13312,"num_write":353,"num_write_kb":9799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"23","length":"6"}]},{"pgid":"3.d","version":"78'1050","reported_seq":10984,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.393934+0000","last_change":"2024-09-14T05:30:30.390839+0000","last_active":"2024-09-14T05:30:31.393934+0000","last_peered":"2024-09-14T05:30:31.393934+0000","last_clean":"2024-09-14T05:30:31.393934+0000","last_became_active":"2024-09-14T05:29:24.012401+0000","last_became_peered":"2024-09-14T05:29:24.012401+0000","last_unstale":"2024-09-14T05:30:31.393934+0000","last_undegraded":"2024-09-14T05:30:31.393934+0000","last_fullsized":"2024-09-14T05:30:31.393934+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1050,"log_dups_size":0,"ondisk_log_size":1050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:59:18.653978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003548673,"stat_sum":{"num_bytes":43455562,"num_objects":71,"num_object_clones":18,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":40,"num_read":14695,"num_read_kb":22163,"num_write":1296,"num_write_kb":6484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.e","version":"78'323","reported_seq":2571,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.395661+0000","last_change":"2024-09-14T05:30:30.390899+0000","last_active":"2024-09-14T05:30:31.395661+0000","last_peered":"2024-09-14T05:30:31.395661+0000","last_clean":"2024-09-14T05:30:31.395661+0000","last_became_active":"2024-09-14T05:29:24.015315+0000","last_became_peered":"2024-09-14T05:29:24.015315+0000","last_unstale":"2024-09-14T05:30:31.395661+0000","last_undegraded":"2024-09-14T05:30:31.395661+0000","last_fullsized":"2024-09-14T05:30:31.395661+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:12:09.100053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011866310000000001,"stat_sum":{"num_bytes":23957039,"num_objects":63,"num_object_clones":20,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":39,"num_read":2147,"num_read_kb":8870,"num_write":286,"num_write_kb":3939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.f","version":"77'420","reported_seq":11313,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T05:30:27.369872+0000","last_change":"2024-09-14T05:30:27.370113+0000","last_active":"2024-09-14T05:30:27.369872+0000","last_peered":"2024-09-14T05:30:27.369872+0000","last_clean":"2024-09-14T05:30:27.369872+0000","last_became_active":"2024-09-14T05:29:24.018854+0000","last_became_peered":"2024-09-14T05:29:24.018854+0000","last_unstale":"2024-09-14T05:30:27.369872+0000","last_undegraded":"2024-09-14T05:30:27.369872+0000","last_fullsized":"2024-09-14T05:30:27.369872+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:22:25.141976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059265900000000005,"stat_sum":{"num_bytes":33427034,"num_objects":82,"num_object_clones":22,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":45,"num_read":11664,"num_read_kb":31579,"num_write":381,"num_write_kb":11831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"23","length":"6"}]},{"pgid":"3.10","version":"78'354","reported_seq":2757,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.441441+0000","last_change":"2024-09-14T05:30:30.392796+0000","last_active":"2024-09-14T05:30:31.441441+0000","last_peered":"2024-09-14T05:30:31.441441+0000","last_clean":"2024-09-14T05:30:31.441441+0000","last_became_active":"2024-09-14T05:29:24.010897+0000","last_became_peered":"2024-09-14T05:29:24.010897+0000","last_unstale":"2024-09-14T05:30:31.441441+0000","last_undegraded":"2024-09-14T05:30:31.441441+0000","last_fullsized":"2024-09-14T05:30:31.441441+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:51:07.166417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059652499999999998,"stat_sum":{"num_bytes":33936373,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":26,"num_read":2371,"num_read_kb":22455,"num_write":327,"num_write_kb":8939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.11","version":"82'498","reported_seq":3525,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.427680+0000","last_change":"2024-09-14T05:30:30.389605+0000","last_active":"2024-09-14T05:30:31.427680+0000","last_peered":"2024-09-14T05:30:31.427680+0000","last_clean":"2024-09-14T05:30:31.427680+0000","last_became_active":"2024-09-14T05:29:24.013792+0000","last_became_peered":"2024-09-14T05:29:24.013792+0000","last_unstale":"2024-09-14T05:30:31.427680+0000","last_undegraded":"2024-09-14T05:30:31.427680+0000","last_fullsized":"2024-09-14T05:30:31.427680+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:03:47.110460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028610660000000002,"stat_sum":{"num_bytes":28012708,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":28,"num_read":4710,"num_read_kb":24064,"num_write":482,"num_write_kb":3324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.12","version":"78'437","reported_seq":2237,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.393730+0000","last_change":"2024-09-14T05:30:30.388608+0000","last_active":"2024-09-14T05:30:31.393730+0000","last_peered":"2024-09-14T05:30:31.393730+0000","last_clean":"2024-09-14T05:30:31.393730+0000","last_became_active":"2024-09-14T05:29:40.095331+0000","last_became_peered":"2024-09-14T05:29:40.095331+0000","last_unstale":"2024-09-14T05:30:31.393730+0000","last_undegraded":"2024-09-14T05:30:31.393730+0000","last_fullsized":"2024-09-14T05:30:31.393730+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:08:41.743970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002054008,"stat_sum":{"num_bytes":27414058,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":2776,"num_read_kb":16472,"num_write":476,"num_write_kb":10860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"7"}]},{"pgid":"3.13","version":"82'616","reported_seq":2160,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T05:30:31.393867+0000","last_change":"2024-09-14T05:30:30.387593+0000","last_active":"2024-09-14T05:30:31.393867+0000","last_peered":"2024-09-14T05:30:31.393867+0000","last_clean":"2024-09-14T05:30:31.393867+0000","last_became_active":"2024-09-14T05:29:24.014371+0000","last_became_peered":"2024-09-14T05:29:24.014371+0000","last_unstale":"2024-09-14T05:30:31.393867+0000","last_undegraded":"2024-09-14T05:30:31.393867+0000","last_fullsized":"2024-09-14T05:30:31.393867+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:34.669035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001050837,"stat_sum":{"num_bytes":56308778,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":3339,"num_read_kb":19859,"num_write":874,"num_write_kb":5495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"7"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1060860572,"num_objects":1868,"num_object_clones":481,"num_object_copies":3736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1868,"num_whiteouts":1001,"num_read":174514,"num_read_kb":558726,"num_write":19359,"num_write_kb":280425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":648894,"num_keys_recovered":33,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":267632640,"data_stored":276854632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16712,"ondisk_log_size":16712,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476756,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75900,"kb_used_data":48872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94295940,"statfs":{"total":96636764160,"available":96559042560,"internally_reserved":0,"allocated":50044928,"data_stored":50299105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509460,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57840,"kb_used_data":22484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94314000,"statfs":{"total":96636764160,"available":96577536000,"internally_reserved":0,"allocated":23023616,"data_stored":22900537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509461,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70988,"kb_used_data":27312,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94300852,"statfs":{"total":96636764160,"available":96564072448,"internally_reserved":0,"allocated":27967488,"data_stored":29086118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71572,"kb_used_data":44544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94300268,"statfs":{"total":96636764160,"available":96563474432,"internally_reserved":0,"allocated":45613056,"data_stored":46528386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75460,"kb_used_data":31788,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94296380,"statfs":{"total":96636764160,"available":96559493120,"internally_reserved":0,"allocated":32550912,"data_stored":32824814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91324,"kb_used_data":24036,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94280516,"statfs":{"total":96636764160,"available":96543248384,"internally_reserved":0,"allocated":24612864,"data_stored":25514608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52580,"kb_used_data":25544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94319260,"statfs":{"total":96636764160,"available":96582922240,"internally_reserved":0,"allocated":26157056,"data_stored":26887277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95284,"kb_used_data":46040,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94276556,"statfs":{"total":96636764160,"available":96539193344,"internally_reserved":0,"allocated":47144960,"data_stored":48135670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49102848,"data_stored":49850924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45604864,"data_stored":47110626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":25202688,"data_stored":26452549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23654400,"data_stored":25079828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":31600640,"data_stored":32390122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":44072960,"data_stored":45503342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":26316800,"data_stored":28001380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":22077440,"data_stored":22465861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:30:33.133 INFO:tasks.thrashosds.thrasher:Setting 3.12 to [7, 0] 2024-09-14T05:30:33.133 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.12', '7', '0'] 2024-09-14T05:30:33.134 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.12 7 0 2024-09-14T05:30:33.216 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:33.215+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:33.249 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:33.249 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:33.258 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:30:33.303 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:33.302+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:33.361 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:33.361 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:33.370 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:30:33.404 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:33.403+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:33.481 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:33.481 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:33.492 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:30:33.536 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:33.535+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:33.595 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:33.595 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:33.604 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:30:33.645 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:33.644+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:33.706 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:33.705+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:33.708 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:33.708 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:33.717 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:30:33.808 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:33.807+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:33.861 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:33.861 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:33.870 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:30:33.908 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:33.908+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:33.997 INFO:teuthology.orchestra.run.smithi071.stderr:set 3.12 pg_upmap mapping to [7,0] 2024-09-14T05:30:34.009 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:34.008+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:34.025 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:34.025 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:34.035 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:30:34.119 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:34.118+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:34.179 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:34.179 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:34.188 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:30:34.229 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:34.228+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:34.311 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:34.311+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:34.333 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:34.333 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:34.342 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:30:34.413 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:34.412+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:34.419 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T05:30:34.441 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:30:34.489 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:34.490 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:34.500 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:30:34.541 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:34.540+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:34.594 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:34.594 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:34.612 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:34.613 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:34.613 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:34.613+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:34.629 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:34.630 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:34.646 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:34.647 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:34.649 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:34.649 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:34.649 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:34.650 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:34.658 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:30:34.663 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:34.664 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:34.680 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:34.680 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:34.697 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:34.698 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:34.715 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:34.714+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:34.715 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:34.715 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:34.769 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:34.769 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:34.769 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:34.769 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:34.769 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:34.778 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:30:34.816 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:34.815+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:34.881 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:34.882 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:34.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:34.882 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:34.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.882 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73760 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2848768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.882 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:34.882 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.320462962000001, 2024-09-14T05:30:34.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.04469911, 2024-09-14T05:30:34.882 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.882 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.882 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.882 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.883 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.883 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.883 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73760 2024-09-14T05:30:34.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.883 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:34.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:34.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:34.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:34.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559701+0000", 2024-09-14T05:30:34.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1909999999999999e-06 2024-09-14T05:30:34.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559708+0000", 2024-09-14T05:30:34.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7279999999999998e-06 2024-09-14T05:30:34.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559713+0000", 2024-09-14T05:30:34.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8389999999999997e-06 2024-09-14T05:30:34.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604310+0000", 2024-09-14T05:30:34.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044597664000000002 2024-09-14T05:30:34.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604352+0000", 2024-09-14T05:30:34.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1236000000000003e-05 2024-09-14T05:30:34.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604398+0000", 2024-09-14T05:30:34.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6091000000000003e-05 2024-09-14T05:30:34.886 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.886 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.886 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.887 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73773 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2801664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.887 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:34.887 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.320442792999998, 2024-09-14T05:30:34.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941026000000002, 2024-09-14T05:30:34.887 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.887 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.887 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.887 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.887 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.887 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.887 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73773 2024-09-14T05:30:34.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.888 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:34.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:34.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:34.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8599999999999999e-07 2024-09-14T05:30:34.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.889 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559721+0000", 2024-09-14T05:30:34.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0540000000000002e-06 2024-09-14T05:30:34.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559729+0000", 2024-09-14T05:30:34.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1359999999999997e-06 2024-09-14T05:30:34.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559734+0000", 2024-09-14T05:30:34.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2140000000000004e-06 2024-09-14T05:30:34.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604441+0000", 2024-09-14T05:30:34.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044706828999999997 2024-09-14T05:30:34.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604522+0000", 2024-09-14T05:30:34.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0603000000000001e-05 2024-09-14T05:30:34.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604660+0000", 2024-09-14T05:30:34.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000137804 2024-09-14T05:30:34.891 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.891 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.891 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.892 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73777 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2834432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.892 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:34.892 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.320420392000003, 2024-09-14T05:30:34.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045116270999999999, 2024-09-14T05:30:34.892 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.892 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.892 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.892 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.892 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.892 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.892 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73777 2024-09-14T05:30:34.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.893 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:34.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:34.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559742+0000", 2024-09-14T05:30:34.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4099999999999999e-07 2024-09-14T05:30:34.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559744+0000", 2024-09-14T05:30:34.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0040000000000002e-06 2024-09-14T05:30:34.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559749+0000", 2024-09-14T05:30:34.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9739999999999999e-06 2024-09-14T05:30:34.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559754+0000", 2024-09-14T05:30:34.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9540000000000003e-06 2024-09-14T05:30:34.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604695+0000", 2024-09-14T05:30:34.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941003 2024-09-14T05:30:34.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604766+0000", 2024-09-14T05:30:34.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0296999999999995e-05 2024-09-14T05:30:34.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604857+0000", 2024-09-14T05:30:34.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1598e-05 2024-09-14T05:30:34.896 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.896 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.896 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.896 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73790 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.896 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:34.896 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.320400456999998, 2024-09-14T05:30:34.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316437000000001, 2024-09-14T05:30:34.897 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.897 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.897 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.897 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.897 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.897 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.897 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73790 2024-09-14T05:30:34.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.897 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:34.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:34.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:34.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7599999999999998e-07 2024-09-14T05:30:34.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559763+0000", 2024-09-14T05:30:34.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0880000000000002e-06 2024-09-14T05:30:34.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559769+0000", 2024-09-14T05:30:34.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6740000000000002e-06 2024-09-14T05:30:34.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559774+0000", 2024-09-14T05:30:34.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6110000000000004e-06 2024-09-14T05:30:34.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604905+0000", 2024-09-14T05:30:34.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045130827999999998 2024-09-14T05:30:34.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604982+0000", 2024-09-14T05:30:34.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7410999999999995e-05 2024-09-14T05:30:34.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605077+0000", 2024-09-14T05:30:34.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5449e-05 2024-09-14T05:30:34.901 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.901 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.901 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.901 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73793 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2836480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.901 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:34.901 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.320381253000001, 2024-09-14T05:30:34.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045441212000000002, 2024-09-14T05:30:34.901 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.901 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.901 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.901 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.901 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.901 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.902 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73793 2024-09-14T05:30:34.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.902 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:34.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:34.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.903 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559781+0000", 2024-09-14T05:30:34.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2899999999999999e-07 2024-09-14T05:30:34.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.903 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.903 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559783+0000", 2024-09-14T05:30:34.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.075e-06 2024-09-14T05:30:34.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.903 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.903 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559788+0000", 2024-09-14T05:30:34.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5369999999999997e-06 2024-09-14T05:30:34.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559793+0000", 2024-09-14T05:30:34.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:30:34.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605109+0000", 2024-09-14T05:30:34.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316229999999999 2024-09-14T05:30:34.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605168+0000", 2024-09-14T05:30:34.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8949000000000001e-05 2024-09-14T05:30:34.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605221+0000", 2024-09-14T05:30:34.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3260000000000002e-05 2024-09-14T05:30:34.905 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.905 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.905 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.905 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73806 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.905 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:34.905 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.320362627000002, 2024-09-14T05:30:34.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045526189000000002, 2024-09-14T05:30:34.906 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.906 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.906 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.906 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.906 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.906 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.906 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73806 2024-09-14T05:30:34.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.906 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:34.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:34.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:34.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:30:34.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559801+0000", 2024-09-14T05:30:34.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.995e-06 2024-09-14T05:30:34.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559807+0000", 2024-09-14T05:30:34.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5450000000000003e-06 2024-09-14T05:30:34.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559811+0000", 2024-09-14T05:30:34.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5090000000000004e-06 2024-09-14T05:30:34.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605237+0000", 2024-09-14T05:30:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045426133 2024-09-14T05:30:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605278+0000", 2024-09-14T05:30:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1084999999999997e-05 2024-09-14T05:30:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605325+0000", 2024-09-14T05:30:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6582999999999998e-05 2024-09-14T05:30:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73809 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2838528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.320339122, 2024-09-14T05:30:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045596062999999999, 2024-09-14T05:30:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.911 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.911 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.911 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73809 2024-09-14T05:30:34.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.911 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:34.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:34.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559823+0000", 2024-09-14T05:30:34.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.08e-07 2024-09-14T05:30:34.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559825+0000", 2024-09-14T05:30:34.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.26e-06 2024-09-14T05:30:34.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559831+0000", 2024-09-14T05:30:34.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6690000000000003e-06 2024-09-14T05:30:34.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559835+0000", 2024-09-14T05:30:34.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5120000000000002e-06 2024-09-14T05:30:34.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605342+0000", 2024-09-14T05:30:34.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045506957000000001 2024-09-14T05:30:34.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.914 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605375+0000", 2024-09-14T05:30:34.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3250999999999997e-05 2024-09-14T05:30:34.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.914 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605418+0000", 2024-09-14T05:30:34.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3006000000000001e-05 2024-09-14T05:30:34.914 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.914 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.914 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.915 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73822 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2807808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.915 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:34.915 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.320311469000004, 2024-09-14T05:30:34.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045661405000000002, 2024-09-14T05:30:34.915 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.915 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.915 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.915 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.915 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.915 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.915 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73822 2024-09-14T05:30:34.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.915 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:34.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:34.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559851+0000", 2024-09-14T05:30:34.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4000000000000002e-07 2024-09-14T05:30:34.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559853+0000", 2024-09-14T05:30:34.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5380000000000001e-06 2024-09-14T05:30:34.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559859+0000", 2024-09-14T05:30:34.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0619999999999997e-06 2024-09-14T05:30:34.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559864+0000", 2024-09-14T05:30:34.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.977e-06 2024-09-14T05:30:34.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605435+0000", 2024-09-14T05:30:34.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045570672 2024-09-14T05:30:34.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605467+0000", 2024-09-14T05:30:34.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2486e-05 2024-09-14T05:30:34.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605511+0000", 2024-09-14T05:30:34.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4129999999999999e-05 2024-09-14T05:30:34.919 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.919 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.919 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.919 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73825 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2840576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.919 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:34.919 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.320288105000003, 2024-09-14T05:30:34.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045728694, 2024-09-14T05:30:34.919 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.919 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.919 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.920 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.920 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.920 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.920 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73825 2024-09-14T05:30:34.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.920 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.920 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.920 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:34.920 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:34.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559874+0000", 2024-09-14T05:30:34.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5400000000000002e-07 2024-09-14T05:30:34.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559876+0000", 2024-09-14T05:30:34.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.131e-06 2024-09-14T05:30:34.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.922 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.922 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559888+0000", 2024-09-14T05:30:34.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2216e-05 2024-09-14T05:30:34.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.922 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.922 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559893+0000", 2024-09-14T05:30:34.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6539999999999998e-06 2024-09-14T05:30:34.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.922 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.922 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605526+0000", 2024-09-14T05:30:34.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045633119 2024-09-14T05:30:34.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605560+0000", 2024-09-14T05:30:34.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3825999999999998e-05 2024-09-14T05:30:34.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605602+0000", 2024-09-14T05:30:34.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2293999999999999e-05 2024-09-14T05:30:34.923 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.923 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.923 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.924 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74023 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2863104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.924 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:34.924 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.319102129999997, 2024-09-14T05:30:34.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044635385999999999, 2024-09-14T05:30:34.924 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.924 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.924 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.924 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.924 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.924 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.924 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74023 2024-09-14T05:30:34.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.924 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:34.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:34.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561060+0000", 2024-09-14T05:30:34.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0699999999999998e-07 2024-09-14T05:30:34.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561072+0000", 2024-09-14T05:30:34.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2261e-05 2024-09-14T05:30:34.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561079+0000", 2024-09-14T05:30:34.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8399999999999997e-06 2024-09-14T05:30:34.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561084+0000", 2024-09-14T05:30:34.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7380000000000002e-06 2024-09-14T05:30:34.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605620+0000", 2024-09-14T05:30:34.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044536146999999998 2024-09-14T05:30:34.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605653+0000", 2024-09-14T05:30:34.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2969000000000002e-05 2024-09-14T05:30:34.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605695+0000", 2024-09-14T05:30:34.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2024000000000003e-05 2024-09-14T05:30:34.928 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.928 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.928 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.928 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74026 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2895872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.928 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:34.928 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.319070668000002, 2024-09-14T05:30:34.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044692717, 2024-09-14T05:30:34.928 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.928 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.928 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.928 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.928 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.929 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.929 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74026 2024-09-14T05:30:34.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.929 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:34.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.929 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.929 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:34.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:34.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:34.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561093+0000", 2024-09-14T05:30:34.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-14T05:30:34.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561099+0000", 2024-09-14T05:30:34.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0399999999999998e-06 2024-09-14T05:30:34.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561104+0000", 2024-09-14T05:30:34.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8180000000000003e-06 2024-09-14T05:30:34.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605710+0000", 2024-09-14T05:30:34.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044605657 2024-09-14T05:30:34.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605742+0000", 2024-09-14T05:30:34.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1949000000000002e-05 2024-09-14T05:30:34.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605784+0000", 2024-09-14T05:30:34.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1884000000000001e-05 2024-09-14T05:30:34.932 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.932 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.932 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.933 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74039 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.933 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:34.933 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.318944070000001, 2024-09-14T05:30:34.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044654905000000002, 2024-09-14T05:30:34.933 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.933 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.933 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.933 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.933 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.933 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.933 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74039 2024-09-14T05:30:34.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.933 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:34.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:34.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.934 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.934 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561218+0000", 2024-09-14T05:30:34.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4000000000000001e-07 2024-09-14T05:30:34.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561221+0000", 2024-09-14T05:30:34.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9730000000000002e-06 2024-09-14T05:30:34.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561229+0000", 2024-09-14T05:30:34.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1759999999999993e-06 2024-09-14T05:30:34.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561235+0000", 2024-09-14T05:30:34.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0109999999999997e-06 2024-09-14T05:30:34.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.936 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605798+0000", 2024-09-14T05:30:34.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044563285000000001 2024-09-14T05:30:34.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.936 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605831+0000", 2024-09-14T05:30:34.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2054999999999997e-05 2024-09-14T05:30:34.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.936 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605872+0000", 2024-09-14T05:30:34.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1764999999999997e-05 2024-09-14T05:30:34.936 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.937 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.937 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.937 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74042 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.937 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:34.937 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.318895617000003, 2024-09-14T05:30:34.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044773234000000002, 2024-09-14T05:30:34.937 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.937 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.937 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.937 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.937 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.937 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.938 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74042 2024-09-14T05:30:34.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.938 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:34.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:34.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:34.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4099999999999998e-07 2024-09-14T05:30:34.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561269+0000", 2024-09-14T05:30:34.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7930000000000002e-06 2024-09-14T05:30:34.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.939 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.939 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561278+0000", 2024-09-14T05:30:34.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5150000000000006e-06 2024-09-14T05:30:34.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561284+0000", 2024-09-14T05:30:34.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2029999999999997e-06 2024-09-14T05:30:34.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605934+0000", 2024-09-14T05:30:34.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044650449000000002 2024-09-14T05:30:34.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605983+0000", 2024-09-14T05:30:34.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8943999999999998e-05 2024-09-14T05:30:34.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606039+0000", 2024-09-14T05:30:34.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5689000000000002e-05 2024-09-14T05:30:34.941 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.941 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.941 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.941 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74052 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.941 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:34.941 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.318868741999999, 2024-09-14T05:30:34.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044830474000000002, 2024-09-14T05:30:34.942 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.942 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.942 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.942 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.942 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.942 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.942 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74052 2024-09-14T05:30:34.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.942 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.942 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.942 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:34.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:34.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:34.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1099999999999996e-07 2024-09-14T05:30:34.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561295+0000", 2024-09-14T05:30:34.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7770000000000001e-06 2024-09-14T05:30:34.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561303+0000", 2024-09-14T05:30:34.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6860000000000006e-06 2024-09-14T05:30:34.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561308+0000", 2024-09-14T05:30:34.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.023e-06 2024-09-14T05:30:34.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.944 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.944 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605893+0000", 2024-09-14T05:30:34.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044585458000000001 2024-09-14T05:30:34.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605933+0000", 2024-09-14T05:30:34.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9551999999999997e-05 2024-09-14T05:30:34.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606123+0000", 2024-09-14T05:30:34.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019046699999999999 2024-09-14T05:30:34.945 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.945 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.945 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.946 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74054 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.946 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:34.946 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.318846714999999, 2024-09-14T05:30:34.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093610999999999, 2024-09-14T05:30:34.946 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.946 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.946 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.946 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.946 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.946 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.946 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74054 2024-09-14T05:30:34.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.946 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.947 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.947 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:34.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.947 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.947 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:34.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.947 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.947 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:34.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9699999999999996e-07 2024-09-14T05:30:34.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561317+0000", 2024-09-14T05:30:34.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9740000000000001e-06 2024-09-14T05:30:34.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561324+0000", 2024-09-14T05:30:34.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4590000000000003e-06 2024-09-14T05:30:34.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561329+0000", 2024-09-14T05:30:34.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5969999999999999e-06 2024-09-14T05:30:34.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606142+0000", 2024-09-14T05:30:34.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044812347000000002 2024-09-14T05:30:34.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606184+0000", 2024-09-14T05:30:34.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2441999999999998e-05 2024-09-14T05:30:34.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.949 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.949 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606408+0000", 2024-09-14T05:30:34.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022429500000000001 2024-09-14T05:30:34.950 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.950 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.950 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.950 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74056 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.950 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:34.950 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.318652012000001, 2024-09-14T05:30:34.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045191685000000002, 2024-09-14T05:30:34.950 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.950 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.950 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.950 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.950 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.951 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.951 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74056 2024-09-14T05:30:34.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.951 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:34.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:34.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.952 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561510+0000", 2024-09-14T05:30:34.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2399999999999998e-07 2024-09-14T05:30:34.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.952 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561513+0000", 2024-09-14T05:30:34.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7219999999999999e-06 2024-09-14T05:30:34.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561524+0000", 2024-09-14T05:30:34.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1357e-05 2024-09-14T05:30:34.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561530+0000", 2024-09-14T05:30:34.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2890000000000003e-06 2024-09-14T05:30:34.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606438+0000", 2024-09-14T05:30:34.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044907326999999997 2024-09-14T05:30:34.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606520+0000", 2024-09-14T05:30:34.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1924999999999999e-05 2024-09-14T05:30:34.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606701+0000", 2024-09-14T05:30:34.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00018144100000000001 2024-09-14T05:30:34.954 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.955 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.955 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.955 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74058 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.955 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:34.955 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.318173338999998, 2024-09-14T05:30:34.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093368000000002, 2024-09-14T05:30:34.955 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.955 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.955 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.955 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.955 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.956 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.956 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74058 2024-09-14T05:30:34.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.956 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:34.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.956 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.956 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:34.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561989+0000", 2024-09-14T05:30:34.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3300000000000001e-07 2024-09-14T05:30:34.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561993+0000", 2024-09-14T05:30:34.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1790000000000001e-06 2024-09-14T05:30:34.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562007+0000", 2024-09-14T05:30:34.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3434e-05 2024-09-14T05:30:34.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562014+0000", 2024-09-14T05:30:34.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.537e-06 2024-09-14T05:30:34.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606736+0000", 2024-09-14T05:30:34.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044722117999999998 2024-09-14T05:30:34.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606829+0000", 2024-09-14T05:30:34.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2410999999999993e-05 2024-09-14T05:30:34.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607082+0000", 2024-09-14T05:30:34.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025285599999999998 2024-09-14T05:30:34.959 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.959 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.959 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.959 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74059 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.959 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:34.959 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.318115347999999, 2024-09-14T05:30:34.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045479928000000003, 2024-09-14T05:30:34.959 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.960 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.960 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.960 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.960 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.960 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.960 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74059 2024-09-14T05:30:34.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.960 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:34.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:34.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562047+0000", 2024-09-14T05:30:34.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6899999999999997e-07 2024-09-14T05:30:34.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562050+0000", 2024-09-14T05:30:34.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.943e-06 2024-09-14T05:30:34.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562058+0000", 2024-09-14T05:30:34.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0120000000000005e-06 2024-09-14T05:30:34.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562063+0000", 2024-09-14T05:30:34.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:30:34.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607186+0000", 2024-09-14T05:30:34.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045122348999999999 2024-09-14T05:30:34.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607265+0000", 2024-09-14T05:30:34.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9462999999999995e-05 2024-09-14T05:30:34.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607526+0000", 2024-09-14T05:30:34.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026090400000000001 2024-09-14T05:30:34.964 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.964 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.964 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.964 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74060 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2850816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:34.964 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:34.964 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.318089970999999, 2024-09-14T05:30:34.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045200092999999997, 2024-09-14T05:30:34.964 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.964 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.965 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:34.965 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.965 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.965 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.965 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74060 2024-09-14T05:30:34.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.965 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:34.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:34.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:34.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5200000000000002e-07 2024-09-14T05:30:34.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562075+0000", 2024-09-14T05:30:34.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5200000000000002e-06 2024-09-14T05:30:34.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562082+0000", 2024-09-14T05:30:34.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8929999999999999e-06 2024-09-14T05:30:34.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562088+0000", 2024-09-14T05:30:34.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5570000000000003e-06 2024-09-14T05:30:34.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607111+0000", 2024-09-14T05:30:34.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045023464999999999 2024-09-14T05:30:34.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607190+0000", 2024-09-14T05:30:34.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8721999999999996e-05 2024-09-14T05:30:34.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607272+0000", 2024-09-14T05:30:34.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1483999999999994e-05 2024-09-14T05:30:34.969 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.969 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.969 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.969 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:136816 3.7 3:e5cc52c3:::rbd_id.image_client.0-clone31:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-14T05:30:34.969 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:30.438431+0000", 2024-09-14T05:30:34.969 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 4.4417302679999997, 2024-09-14T05:30:34.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.96261680299999997, 2024-09-14T05:30:34.969 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:34.969 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:34.969 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:34.969 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:34.970 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:34.970 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:34.970 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 136816 2024-09-14T05:30:34.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.970 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:34.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:34.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438431+0000", 2024-09-14T05:30:34.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:34.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438431+0000", 2024-09-14T05:30:34.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438432+0000", 2024-09-14T05:30:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0419999999999999e-06 2024-09-14T05:30:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438437+0000", 2024-09-14T05:30:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5000000000000001e-06 2024-09-14T05:30:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:34.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438446+0000", 2024-09-14T05:30:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5740000000000002e-06 2024-09-14T05:30:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438456+0000", 2024-09-14T05:30:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0056e-05 2024-09-14T05:30:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438500+0000", 2024-09-14T05:30:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3738e-05 2024-09-14T05:30:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:30:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438506+0000", 2024-09-14T05:30:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1659999999999999e-06 2024-09-14T05:30:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.399222+0000", 2024-09-14T05:30:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.96071578999999996 2024-09-14T05:30:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.399381+0000", 2024-09-14T05:30:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000158834 2024-09-14T05:30:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-14T05:30:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.399530+0000", 2024-09-14T05:30:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014862400000000001 2024-09-14T05:30:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:30:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.400507+0000", 2024-09-14T05:30:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0009779369999999999 2024-09-14T05:30:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.400969+0000", 2024-09-14T05:30:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046119800000000001 2024-09-14T05:30:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:30:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.401006+0000", 2024-09-14T05:30:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7594e-05 2024-09-14T05:30:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.401048+0000", 2024-09-14T05:30:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1749999999999998e-05 2024-09-14T05:30:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:34.975 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:34.976 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:34.977 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:30:34.978 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:34.977+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:34.999 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-14T05:30:35.078 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:35.077+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:35.121 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:35.121 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:35.122 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:35.122 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:35.131 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:30:35.214 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:35.214+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:35.274 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:35.275 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:35.275 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:35.275 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:35.275 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:35.279 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:35.279+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:35.283 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:30:35.380 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:35.379+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:35.428 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4235.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-14T05:30:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:30:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 81.586902244000001, 2024-09-14T05:30:35.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12887758399999999, 2024-09-14T05:30:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:30:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4235", 2024-09-14T05:30:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/3211559243", 2024-09-14T05:30:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 2 2024-09-14T05:30:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:30:35.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:30:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839833+0000", 2024-09-14T05:30:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:30:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839838+0000", 2024-09-14T05:30:35.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2440000000000001e-06 2024-09-14T05:30:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839848+0000", 2024-09-14T05:30:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0030000000000001e-05 2024-09-14T05:30:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839854+0000", 2024-09-14T05:30:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3750000000000002e-06 2024-09-14T05:30:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839908+0000", 2024-09-14T05:30:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3965999999999998e-05 2024-09-14T05:30:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:30:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839914+0000", 2024-09-14T05:30:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6139999999999997e-06 2024-09-14T05:30:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968558+0000", 2024-09-14T05:30:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12864361199999999 2024-09-14T05:30:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968710+0000", 2024-09-14T05:30:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015177300000000001 2024-09-14T05:30:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27764 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 188416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:30:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.215575862000001, 2024-09-14T05:30:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080826920999999996, 2024-09-14T05:30:35.433 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27764 2024-09-14T05:30:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:30:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:30:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211159+0000", 2024-09-14T05:30:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1800000000000001e-07 2024-09-14T05:30:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211161+0000", 2024-09-14T05:30:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.968e-06 2024-09-14T05:30:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211169+0000", 2024-09-14T05:30:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4800000000000001e-06 2024-09-14T05:30:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.436 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211174+0000", 2024-09-14T05:30:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1880000000000001e-06 2024-09-14T05:30:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291670+0000", 2024-09-14T05:30:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080495199000000003 2024-09-14T05:30:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291775+0000", 2024-09-14T05:30:35.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000105375 2024-09-14T05:30:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291985+0000", 2024-09-14T05:30:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021029299999999999 2024-09-14T05:30:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27765 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:30:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.215553112000002, 2024-09-14T05:30:35.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081115197999999999, 2024-09-14T05:30:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27765 2024-09-14T05:30:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:30:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:30:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211182+0000", 2024-09-14T05:30:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3599999999999999e-07 2024-09-14T05:30:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211184+0000", 2024-09-14T05:30:35.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.03e-06 2024-09-14T05:30:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211192+0000", 2024-09-14T05:30:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2309999999999996e-06 2024-09-14T05:30:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211197+0000", 2024-09-14T05:30:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4199999999999998e-06 2024-09-14T05:30:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.441 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292017+0000", 2024-09-14T05:30:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080819516999999994 2024-09-14T05:30:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292087+0000", 2024-09-14T05:30:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0056999999999997e-05 2024-09-14T05:30:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292296+0000", 2024-09-14T05:30:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000209507 2024-09-14T05:30:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.442 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27766 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:30:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.215529898, 2024-09-14T05:30:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081359077000000002, 2024-09-14T05:30:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.443 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27766 2024-09-14T05:30:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:30:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:30:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211205+0000", 2024-09-14T05:30:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7e-07 2024-09-14T05:30:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211207+0000", 2024-09-14T05:30:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4540000000000001e-06 2024-09-14T05:30:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211216+0000", 2024-09-14T05:30:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3680000000000002e-06 2024-09-14T05:30:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211221+0000", 2024-09-14T05:30:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3499999999999996e-06 2024-09-14T05:30:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292329+0000", 2024-09-14T05:30:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081108240999999998 2024-09-14T05:30:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292370+0000", 2024-09-14T05:30:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.121e-05 2024-09-14T05:30:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292563+0000", 2024-09-14T05:30:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019298399999999999 2024-09-14T05:30:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27767 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.215506384000001, 2024-09-14T05:30:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081623119999999993, 2024-09-14T05:30:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27767 2024-09-14T05:30:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.448 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0499999999999999e-07 2024-09-14T05:30:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211230+0000", 2024-09-14T05:30:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0700000000000001e-06 2024-09-14T05:30:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211236+0000", 2024-09-14T05:30:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8429999999999998e-06 2024-09-14T05:30:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211241+0000", 2024-09-14T05:30:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:30:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292594+0000", 2024-09-14T05:30:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081352868999999994 2024-09-14T05:30:35.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292631+0000", 2024-09-14T05:30:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7083999999999999e-05 2024-09-14T05:30:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292851+0000", 2024-09-14T05:30:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022011700000000001 2024-09-14T05:30:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27768 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 274432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:30:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.215487848000002, 2024-09-14T05:30:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081875032, 2024-09-14T05:30:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27768 2024-09-14T05:30:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.452 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:30:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:30:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211247+0000", 2024-09-14T05:30:35.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.46e-07 2024-09-14T05:30:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211248+0000", 2024-09-14T05:30:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7349999999999999e-06 2024-09-14T05:30:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211253+0000", 2024-09-14T05:30:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.899e-06 2024-09-14T05:30:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211258+0000", 2024-09-14T05:30:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2810000000000002e-06 2024-09-14T05:30:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292888+0000", 2024-09-14T05:30:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081629955000000004 2024-09-14T05:30:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292926+0000", 2024-09-14T05:30:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7947000000000001e-05 2024-09-14T05:30:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293121+0000", 2024-09-14T05:30:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019586899999999999 2024-09-14T05:30:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27769 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 294912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:30:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.215469759000001, 2024-09-14T05:30:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082181899000000003, 2024-09-14T05:30:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.456 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27769 2024-09-14T05:30:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:30:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:30:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211265+0000", 2024-09-14T05:30:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2199999999999999e-07 2024-09-14T05:30:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211267+0000", 2024-09-14T05:30:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2239999999999998e-06 2024-09-14T05:30:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211273+0000", 2024-09-14T05:30:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6540000000000004e-06 2024-09-14T05:30:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211277+0000", 2024-09-14T05:30:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6299999999999997e-06 2024-09-14T05:30:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293150+0000", 2024-09-14T05:30:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081872575000000003 2024-09-14T05:30:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293245+0000", 2024-09-14T05:30:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5327000000000004e-05 2024-09-14T05:30:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293446+0000", 2024-09-14T05:30:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000201067 2024-09-14T05:30:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27770 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.215449597000003, 2024-09-14T05:30:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082663707000000003, 2024-09-14T05:30:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27770 2024-09-14T05:30:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1199999999999998e-07 2024-09-14T05:30:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211292+0000", 2024-09-14T05:30:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1609999999999997e-06 2024-09-14T05:30:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211299+0000", 2024-09-14T05:30:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3300000000000004e-06 2024-09-14T05:30:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211303+0000", 2024-09-14T05:30:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7859999999999996e-06 2024-09-14T05:30:35.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293628+0000", 2024-09-14T05:30:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082324283999999998 2024-09-14T05:30:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293694+0000", 2024-09-14T05:30:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6277000000000003e-05 2024-09-14T05:30:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293948+0000", 2024-09-14T05:30:35.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025445699999999998 2024-09-14T05:30:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27790 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:30:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.215423192999999, 2024-09-14T05:30:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082461006000000003, 2024-09-14T05:30:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.465 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27790 2024-09-14T05:30:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:30:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:30:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211312+0000", 2024-09-14T05:30:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9299999999999998e-07 2024-09-14T05:30:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211314+0000", 2024-09-14T05:30:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000002e-06 2024-09-14T05:30:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211320+0000", 2024-09-14T05:30:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1589999999999998e-06 2024-09-14T05:30:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211325+0000", 2024-09-14T05:30:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9609999999999996e-06 2024-09-14T05:30:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293474+0000", 2024-09-14T05:30:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082149080999999999 2024-09-14T05:30:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293542+0000", 2024-09-14T05:30:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8497000000000005e-05 2024-09-14T05:30:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293772+0000", 2024-09-14T05:30:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022969900000000001 2024-09-14T05:30:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27791 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:30:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.215402879000003, 2024-09-14T05:30:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082700838999999998, 2024-09-14T05:30:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27791 2024-09-14T05:30:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:30:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:30:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211332+0000", 2024-09-14T05:30:35.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7399999999999999e-07 2024-09-14T05:30:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211334+0000", 2024-09-14T05:30:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8500000000000001e-06 2024-09-14T05:30:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211340+0000", 2024-09-14T05:30:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3010000000000003e-06 2024-09-14T05:30:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211345+0000", 2024-09-14T05:30:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.938e-06 2024-09-14T05:30:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293797+0000", 2024-09-14T05:30:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082452152000000001 2024-09-14T05:30:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293869+0000", 2024-09-14T05:30:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2236000000000004e-05 2024-09-14T05:30:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294032+0000", 2024-09-14T05:30:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016298800000000001 2024-09-14T05:30:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27792 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.215383348000003, 2024-09-14T05:30:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082897304000000005, 2024-09-14T05:30:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27792 2024-09-14T05:30:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2000000000000001e-07 2024-09-14T05:30:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211353+0000", 2024-09-14T05:30:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.761e-06 2024-09-14T05:30:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211359+0000", 2024-09-14T05:30:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2430000000000002e-06 2024-09-14T05:30:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211364+0000", 2024-09-14T05:30:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0239999999999997e-06 2024-09-14T05:30:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294059+0000", 2024-09-14T05:30:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082694834999999994 2024-09-14T05:30:35.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294114+0000", 2024-09-14T05:30:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4823e-05 2024-09-14T05:30:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294248+0000", 2024-09-14T05:30:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013429800000000001 2024-09-14T05:30:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27793 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.215363549000003, 2024-09-14T05:30:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083153362999999994, 2024-09-14T05:30:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27793 2024-09-14T05:30:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.479 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5100000000000001e-07 2024-09-14T05:30:35.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211373+0000", 2024-09-14T05:30:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.001e-06 2024-09-14T05:30:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211379+0000", 2024-09-14T05:30:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4559999999999997e-06 2024-09-14T05:30:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211384+0000", 2024-09-14T05:30:35.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9749999999999998e-06 2024-09-14T05:30:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294279+0000", 2024-09-14T05:30:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082894282999999999 2024-09-14T05:30:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294327+0000", 2024-09-14T05:30:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8208000000000003e-05 2024-09-14T05:30:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294524+0000", 2024-09-14T05:30:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019708900000000001 2024-09-14T05:30:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27794 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.215343224999998, 2024-09-14T05:30:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588590000000004, 2024-09-14T05:30:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.483 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27794 2024-09-14T05:30:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:35.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4499999999999997e-07 2024-09-14T05:30:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211393+0000", 2024-09-14T05:30:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.815e-06 2024-09-14T05:30:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211400+0000", 2024-09-14T05:30:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2899999999999999e-06 2024-09-14T05:30:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211404+0000", 2024-09-14T05:30:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:30:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294560+0000", 2024-09-14T05:30:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083155534000000003 2024-09-14T05:30:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294631+0000", 2024-09-14T05:30:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1378999999999995e-05 2024-09-14T05:30:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294980+0000", 2024-09-14T05:30:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00034844900000000002 2024-09-14T05:30:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27795 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:30:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.215323867000002, 2024-09-14T05:30:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083970367000000004, 2024-09-14T05:30:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.489 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27795 2024-09-14T05:30:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:30:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:30:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211411+0000", 2024-09-14T05:30:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5499999999999999e-07 2024-09-14T05:30:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211413+0000", 2024-09-14T05:30:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:30:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211419+0000", 2024-09-14T05:30:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1720000000000004e-06 2024-09-14T05:30:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211424+0000", 2024-09-14T05:30:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0690000000000001e-06 2024-09-14T05:30:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295013+0000", 2024-09-14T05:30:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588898999999994 2024-09-14T05:30:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295102+0000", 2024-09-14T05:30:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9462000000000005e-05 2024-09-14T05:30:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295381+0000", 2024-09-14T05:30:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000278566 2024-09-14T05:30:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27796 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.493 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.215304138, 2024-09-14T05:30:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084379107999999994, 2024-09-14T05:30:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27796 2024-09-14T05:30:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2800000000000002e-07 2024-09-14T05:30:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211432+0000", 2024-09-14T05:30:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:30:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211439+0000", 2024-09-14T05:30:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5239999999999997e-06 2024-09-14T05:30:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211444+0000", 2024-09-14T05:30:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.95e-06 2024-09-14T05:30:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295602+0000", 2024-09-14T05:30:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084158017000000002 2024-09-14T05:30:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295674+0000", 2024-09-14T05:30:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2385999999999994e-05 2024-09-14T05:30:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295809+0000", 2024-09-14T05:30:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013495900000000001 2024-09-14T05:30:35.497 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27797 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 266240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.215284156000003, 2024-09-14T05:30:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084898597000000006, 2024-09-14T05:30:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.498 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27797 2024-09-14T05:30:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:30:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211452+0000", 2024-09-14T05:30:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.708e-06 2024-09-14T05:30:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211458+0000", 2024-09-14T05:30:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3489999999999997e-06 2024-09-14T05:30:35.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211464+0000", 2024-09-14T05:30:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3820000000000003e-06 2024-09-14T05:30:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295839+0000", 2024-09-14T05:30:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084375055000000004 2024-09-14T05:30:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295918+0000", 2024-09-14T05:30:35.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9253999999999997e-05 2024-09-14T05:30:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296349+0000", 2024-09-14T05:30:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043051 2024-09-14T05:30:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27798 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.215264441000002, 2024-09-14T05:30:35.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085300275999999994, 2024-09-14T05:30:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27798 2024-09-14T05:30:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211472+0000", 2024-09-14T05:30:35.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7579999999999999e-06 2024-09-14T05:30:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211478+0000", 2024-09-14T05:30:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0310000000000004e-06 2024-09-14T05:30:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211483+0000", 2024-09-14T05:30:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8110000000000002e-06 2024-09-14T05:30:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296469+0000", 2024-09-14T05:30:35.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084986575999999994 2024-09-14T05:30:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296545+0000", 2024-09-14T05:30:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5749999999999998e-05 2024-09-14T05:30:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296770+0000", 2024-09-14T05:30:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022498900000000001 2024-09-14T05:30:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27799 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 307200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.215245527999997, 2024-09-14T05:30:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085718461999999995, 2024-09-14T05:30:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27799 2024-09-14T05:30:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.507 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:35.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8000000000000001e-07 2024-09-14T05:30:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211491+0000", 2024-09-14T05:30:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-14T05:30:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211497+0000", 2024-09-14T05:30:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.857e-06 2024-09-14T05:30:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211502+0000", 2024-09-14T05:30:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9790000000000001e-06 2024-09-14T05:30:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296800+0000", 2024-09-14T05:30:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085298099000000002 2024-09-14T05:30:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296871+0000", 2024-09-14T05:30:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0879999999999999e-05 2024-09-14T05:30:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297207+0000", 2024-09-14T05:30:35.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033651699999999999 2024-09-14T05:30:35.511 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.511 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.511 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.511 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27826 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.511 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:30:35.511 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.213530407, 2024-09-14T05:30:35.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084306729999999996, 2024-09-14T05:30:35.511 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.511 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.511 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.512 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.512 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.512 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.512 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27826 2024-09-14T05:30:35.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.512 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:30:35.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:30:35.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213205+0000", 2024-09-14T05:30:35.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.277e-06 2024-09-14T05:30:35.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213210+0000", 2024-09-14T05:30:35.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3120000000000001e-06 2024-09-14T05:30:35.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213226+0000", 2024-09-14T05:30:35.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5279000000000001e-05 2024-09-14T05:30:35.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213234+0000", 2024-09-14T05:30:35.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7020000000000007e-06 2024-09-14T05:30:35.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297239+0000", 2024-09-14T05:30:35.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084004876000000006 2024-09-14T05:30:35.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.515 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297311+0000", 2024-09-14T05:30:35.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1336999999999998e-05 2024-09-14T05:30:35.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.515 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.515 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297511+0000", 2024-09-14T05:30:35.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019994699999999999 2024-09-14T05:30:35.515 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.515 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.515 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.515 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27827 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:35.515 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:30:35.516 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 49.213485220000003, 2024-09-14T05:30:35.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084520537000000007, 2024-09-14T05:30:35.516 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:35.516 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:35.516 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:35.516 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:35.516 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:35.516 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.516 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27827 2024-09-14T05:30:35.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.516 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:35.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:35.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:30:35.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:35.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:30:35.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:35.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:35.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213250+0000", 2024-09-14T05:30:35.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1099999999999995e-07 2024-09-14T05:30:35.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:35.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213252+0000", 2024-09-14T05:30:35.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6699999999999998e-06 2024-09-14T05:30:35.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:35.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213259+0000", 2024-09-14T05:30:35.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8469999999999998e-06 2024-09-14T05:30:35.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213265+0000", 2024-09-14T05:30:35.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4940000000000003e-06 2024-09-14T05:30:35.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.519 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:35.519 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297541+0000", 2024-09-14T05:30:35.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084276731999999993 2024-09-14T05:30:35.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.519 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:35.519 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297579+0000", 2024-09-14T05:30:35.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8108999999999997e-05 2024-09-14T05:30:35.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:35.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:35.519 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:35.519 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297770+0000", 2024-09-14T05:30:35.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019007400000000001 2024-09-14T05:30:35.519 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.520 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.520 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.520 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:35.520 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:35.520 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:35.521 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:30:35.522 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:35.520+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:35.622 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:35.622+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:35.636 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:35.636 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:35.636 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:35.637 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:35.646 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:30:35.723 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:35.722+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:35.790 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:35.790 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:35.790 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:35.790 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:35.790 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:35.799 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:30:35.824 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:35.824+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:35.903 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3523 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6062080~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.387402159999994, 2024-09-14T05:30:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096911030999999995, 2024-09-14T05:30:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3523 2024-09-14T05:30:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514359+0000", 2024-09-14T05:30:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9299999999999999e-07 2024-09-14T05:30:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514361+0000", 2024-09-14T05:30:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514366+0000", 2024-09-14T05:30:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1660000000000002e-06 2024-09-14T05:30:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514371+0000", 2024-09-14T05:30:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.648e-06 2024-09-14T05:30:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.610954+0000", 2024-09-14T05:30:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096582170999999994 2024-09-14T05:30:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611002+0000", 2024-09-14T05:30:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7923999999999997e-05 2024-09-14T05:30:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611269+0000", 2024-09-14T05:30:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026789499999999998 2024-09-14T05:30:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3524 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6553600~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.387382967999997, 2024-09-14T05:30:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097263347999999999, 2024-09-14T05:30:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3524 2024-09-14T05:30:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514380+0000", 2024-09-14T05:30:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6500000000000001e-06 2024-09-14T05:30:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514386+0000", 2024-09-14T05:30:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7360000000000004e-06 2024-09-14T05:30:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514391+0000", 2024-09-14T05:30:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2830000000000003e-06 2024-09-14T05:30:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611295+0000", 2024-09-14T05:30:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096904814000000006 2024-09-14T05:30:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611343+0000", 2024-09-14T05:30:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7506999999999998e-05 2024-09-14T05:30:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611641+0000", 2024-09-14T05:30:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029798800000000001 2024-09-14T05:30:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3525 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7045120~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.387361080999995, 2024-09-14T05:30:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097644422999999994, 2024-09-14T05:30:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3525 2024-09-14T05:30:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514400+0000", 2024-09-14T05:30:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.77e-07 2024-09-14T05:30:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514402+0000", 2024-09-14T05:30:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6530000000000001e-06 2024-09-14T05:30:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514411+0000", 2024-09-14T05:30:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7429999999999997e-06 2024-09-14T05:30:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514417+0000", 2024-09-14T05:30:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2249999999999997e-06 2024-09-14T05:30:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611668+0000", 2024-09-14T05:30:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097250681000000005 2024-09-14T05:30:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611716+0000", 2024-09-14T05:30:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7336000000000002e-05 2024-09-14T05:30:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612044+0000", 2024-09-14T05:30:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032840799999999998 2024-09-14T05:30:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3526 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7536640~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.387336597000001, 2024-09-14T05:30:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097943884999999994, 2024-09-14T05:30:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3526 2024-09-14T05:30:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0400000000000002e-07 2024-09-14T05:30:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514426+0000", 2024-09-14T05:30:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514432+0000", 2024-09-14T05:30:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.105e-06 2024-09-14T05:30:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514438+0000", 2024-09-14T05:30:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5149999999999997e-06 2024-09-14T05:30:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612072+0000", 2024-09-14T05:30:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097633947999999998 2024-09-14T05:30:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612120+0000", 2024-09-14T05:30:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8458999999999998e-05 2024-09-14T05:30:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612368+0000", 2024-09-14T05:30:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024761999999999999 2024-09-14T05:30:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3527 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 8028160~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.387315884000003, 2024-09-14T05:30:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098244955999999994, 2024-09-14T05:30:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3527 2024-09-14T05:30:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519842+0000", 2024-09-14T05:30:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053975230000000004 2024-09-14T05:30:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519851+0000", 2024-09-14T05:30:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5599999999999994e-06 2024-09-14T05:30:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519858+0000", 2024-09-14T05:30:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2409999999999998e-06 2024-09-14T05:30:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519863+0000", 2024-09-14T05:30:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:30:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612395+0000", 2024-09-14T05:30:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092532552000000004 2024-09-14T05:30:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612442+0000", 2024-09-14T05:30:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6486000000000002e-05 2024-09-14T05:30:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612690+0000", 2024-09-14T05:30:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024791599999999999 2024-09-14T05:30:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3603 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5177344~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.381763438999997, 2024-09-14T05:30:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096796436, 2024-09-14T05:30:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3603 2024-09-14T05:30:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1899999999999999e-07 2024-09-14T05:30:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519998+0000", 2024-09-14T05:30:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5000000000000001e-07 2024-09-14T05:30:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520002+0000", 2024-09-14T05:30:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7299999999999999e-06 2024-09-14T05:30:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520004+0000", 2024-09-14T05:30:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.429e-06 2024-09-14T05:30:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616493+0000", 2024-09-14T05:30:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096488345000000003 2024-09-14T05:30:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616544+0000", 2024-09-14T05:30:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1724000000000001e-05 2024-09-14T05:30:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616794+0000", 2024-09-14T05:30:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000249139 2024-09-14T05:30:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3604 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5668864~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.381748801000001, 2024-09-14T05:30:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108222999999994, 2024-09-14T05:30:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3604 2024-09-14T05:30:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7000000000000001e-07 2024-09-14T05:30:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520013+0000", 2024-09-14T05:30:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.389e-06 2024-09-14T05:30:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520018+0000", 2024-09-14T05:30:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3019999999999996e-06 2024-09-14T05:30:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520026+0000", 2024-09-14T05:30:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6170000000000007e-06 2024-09-14T05:30:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616816+0000", 2024-09-14T05:30:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096789766999999999 2024-09-14T05:30:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616871+0000", 2024-09-14T05:30:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4811000000000001e-05 2024-09-14T05:30:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617120+0000", 2024-09-14T05:30:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024906700000000001 2024-09-14T05:30:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3605 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6160384~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.381730340999994, 2024-09-14T05:30:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097423945999999997, 2024-09-14T05:30:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3605 2024-09-14T05:30:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3999999999999998e-07 2024-09-14T05:30:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520032+0000", 2024-09-14T05:30:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.083e-06 2024-09-14T05:30:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520036+0000", 2024-09-14T05:30:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.019e-06 2024-09-14T05:30:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520038+0000", 2024-09-14T05:30:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4849999999999999e-06 2024-09-14T05:30:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617147+0000", 2024-09-14T05:30:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108665999999996 2024-09-14T05:30:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617199+0000", 2024-09-14T05:30:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2558000000000001e-05 2024-09-14T05:30:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617454+0000", 2024-09-14T05:30:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025489499999999999 2024-09-14T05:30:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3606 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6651904~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.381719618999995, 2024-09-14T05:30:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097745705000000002, 2024-09-14T05:30:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3606 2024-09-14T05:30:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-14T05:30:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520042+0000", 2024-09-14T05:30:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4799999999999997e-07 2024-09-14T05:30:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520046+0000", 2024-09-14T05:30:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8149999999999999e-06 2024-09-14T05:30:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520049+0000", 2024-09-14T05:30:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.576e-06 2024-09-14T05:30:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617476+0000", 2024-09-14T05:30:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097427558999999997 2024-09-14T05:30:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617522+0000", 2024-09-14T05:30:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5927999999999998e-05 2024-09-14T05:30:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617787+0000", 2024-09-14T05:30:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000264663 2024-09-14T05:30:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3607 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7143424~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.381709091999994, 2024-09-14T05:30:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097871125000000003, 2024-09-14T05:30:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3607 2024-09-14T05:30:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520052+0000", 2024-09-14T05:30:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.03e-07 2024-09-14T05:30:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520053+0000", 2024-09-14T05:30:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.85e-07 2024-09-14T05:30:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520056+0000", 2024-09-14T05:30:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9140000000000001e-06 2024-09-14T05:30:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520059+0000", 2024-09-14T05:30:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4600000000000002e-06 2024-09-14T05:30:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617809+0000", 2024-09-14T05:30:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097750216000000001 2024-09-14T05:30:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617864+0000", 2024-09-14T05:30:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5211999999999999e-05 2024-09-14T05:30:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617923+0000", 2024-09-14T05:30:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8235000000000001e-05 2024-09-14T05:30:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3608 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7634944~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.381698783999994, 2024-09-14T05:30:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098199212999999994, 2024-09-14T05:30:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3608 2024-09-14T05:30:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525953+0000", 2024-09-14T05:30:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058911850000000002 2024-09-14T05:30:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525957+0000", 2024-09-14T05:30:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2379999999999999e-06 2024-09-14T05:30:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525967+0000", 2024-09-14T05:30:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5389999999999996e-06 2024-09-14T05:30:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525972+0000", 2024-09-14T05:30:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.541e-06 2024-09-14T05:30:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617945+0000", 2024-09-14T05:30:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091972597000000003 2024-09-14T05:30:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617992+0000", 2024-09-14T05:30:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7397e-05 2024-09-14T05:30:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.618261+0000", 2024-09-14T05:30:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026871600000000001 2024-09-14T05:30:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3640 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6586368~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.375590942000002, 2024-09-14T05:30:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097068580000000002, 2024-09-14T05:30:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3640 2024-09-14T05:30:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.23e-07 2024-09-14T05:30:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526171+0000", 2024-09-14T05:30:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:30:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526175+0000", 2024-09-14T05:30:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8369999999999999e-06 2024-09-14T05:30:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526177+0000", 2024-09-14T05:30:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4959999999999999e-06 2024-09-14T05:30:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622847+0000", 2024-09-14T05:30:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096669790000000005 2024-09-14T05:30:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622902+0000", 2024-09-14T05:30:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5157000000000001e-05 2024-09-14T05:30:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623238+0000", 2024-09-14T05:30:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033603399999999997 2024-09-14T05:30:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3641 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7077888~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.375580432000007, 2024-09-14T05:30:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097371141999999994, 2024-09-14T05:30:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3641 2024-09-14T05:30:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531851+0000", 2024-09-14T05:30:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005671265 2024-09-14T05:30:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531855+0000", 2024-09-14T05:30:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0849999999999999e-06 2024-09-14T05:30:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531867+0000", 2024-09-14T05:30:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1028e-05 2024-09-14T05:30:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531872+0000", 2024-09-14T05:30:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000003e-06 2024-09-14T05:30:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623251+0000", 2024-09-14T05:30:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091378980999999998 2024-09-14T05:30:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623295+0000", 2024-09-14T05:30:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3881999999999998e-05 2024-09-14T05:30:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623551+0000", 2024-09-14T05:30:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025636099999999998 2024-09-14T05:30:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3721 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.369722156999998, 2024-09-14T05:30:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096806472000000005, 2024-09-14T05:30:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3721 2024-09-14T05:30:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532039+0000", 2024-09-14T05:30:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.05e-07 2024-09-14T05:30:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532040+0000", 2024-09-14T05:30:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9099999999999991e-07 2024-09-14T05:30:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532043+0000", 2024-09-14T05:30:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6610000000000002e-06 2024-09-14T05:30:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532046+0000", 2024-09-14T05:30:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4080000000000001e-06 2024-09-14T05:30:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628545+0000", 2024-09-14T05:30:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096499615999999996 2024-09-14T05:30:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628592+0000", 2024-09-14T05:30:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6907000000000003e-05 2024-09-14T05:30:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628845+0000", 2024-09-14T05:30:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025268399999999998 2024-09-14T05:30:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3722 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6193152~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.369712077000003, 2024-09-14T05:30:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097146587000000006, 2024-09-14T05:30:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3722 2024-09-14T05:30:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538191+0000", 2024-09-14T05:30:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061429969999999999 2024-09-14T05:30:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538200+0000", 2024-09-14T05:30:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7069999999999998e-06 2024-09-14T05:30:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538212+0000", 2024-09-14T05:30:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1504e-05 2024-09-14T05:30:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538219+0000", 2024-09-14T05:30:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7500000000000003e-06 2024-09-14T05:30:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628869+0000", 2024-09-14T05:30:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090649224 2024-09-14T05:30:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628919+0000", 2024-09-14T05:30:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0144e-05 2024-09-14T05:30:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.629195+0000", 2024-09-14T05:30:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027626099999999998 2024-09-14T05:30:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3758 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7110656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.363117861000006, 2024-09-14T05:30:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096977544999999998, 2024-09-14T05:30:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3758 2024-09-14T05:30:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538645+0000", 2024-09-14T05:30:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.843e-06 2024-09-14T05:30:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538651+0000", 2024-09-14T05:30:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6169999999999998e-06 2024-09-14T05:30:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538655+0000", 2024-09-14T05:30:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5399999999999997e-06 2024-09-14T05:30:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635225+0000", 2024-09-14T05:30:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096570149999999993 2024-09-14T05:30:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635272+0000", 2024-09-14T05:30:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7199e-05 2024-09-14T05:30:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635620+0000", 2024-09-14T05:30:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034782600000000003 2024-09-14T05:30:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3759 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.363099950000006, 2024-09-14T05:30:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097335331999999997, 2024-09-14T05:30:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3759 2024-09-14T05:30:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544013+0000", 2024-09-14T05:30:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053520829999999997 2024-09-14T05:30:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544016+0000", 2024-09-14T05:30:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6660000000000001e-06 2024-09-14T05:30:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544026+0000", 2024-09-14T05:30:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7089999999999997e-06 2024-09-14T05:30:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544043+0000", 2024-09-14T05:30:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833999999999998e-05 2024-09-14T05:30:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635646+0000", 2024-09-14T05:30:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091603544999999995 2024-09-14T05:30:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635696+0000", 2024-09-14T05:30:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9307000000000001e-05 2024-09-14T05:30:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635996+0000", 2024-09-14T05:30:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030018800000000001 2024-09-14T05:30:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3838 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6225920~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.357438270000003, 2024-09-14T05:30:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096894235999999995, 2024-09-14T05:30:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3838 2024-09-14T05:30:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544323+0000", 2024-09-14T05:30:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-14T05:30:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544325+0000", 2024-09-14T05:30:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.75e-06 2024-09-14T05:30:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544331+0000", 2024-09-14T05:30:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5019999999999998e-06 2024-09-14T05:30:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544335+0000", 2024-09-14T05:30:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2989999999999998e-06 2024-09-14T05:30:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640869+0000", 2024-09-14T05:30:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096533351000000003 2024-09-14T05:30:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640919+0000", 2024-09-14T05:30:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999999e-05 2024-09-14T05:30:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641217+0000", 2024-09-14T05:30:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029717600000000002 2024-09-14T05:30:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.984 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3839 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.984 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:35.984 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.357419332000006, 2024-09-14T05:30:35.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097240339999999995, 2024-09-14T05:30:35.984 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.984 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.984 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.984 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3839 2024-09-14T05:30:35.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.985 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:35.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:35.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549834+0000", 2024-09-14T05:30:35.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054927819999999999 2024-09-14T05:30:35.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549838+0000", 2024-09-14T05:30:35.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8689999999999997e-06 2024-09-14T05:30:35.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549848+0000", 2024-09-14T05:30:35.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0154e-05 2024-09-14T05:30:35.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549853+0000", 2024-09-14T05:30:35.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9740000000000001e-06 2024-09-14T05:30:35.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641245+0000", 2024-09-14T05:30:35.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091392245999999996 2024-09-14T05:30:35.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641295+0000", 2024-09-14T05:30:35.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0018999999999997e-05 2024-09-14T05:30:35.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641582+0000", 2024-09-14T05:30:35.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028629600000000001 2024-09-14T05:30:35.988 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.988 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.988 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.988 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3868 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:35.988 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:35.988 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.351756753999993, 2024-09-14T05:30:35.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096859555999999999, 2024-09-14T05:30:35.988 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:35.988 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:35.989 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:35.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:35.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:35.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:35.989 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3868 2024-09-14T05:30:35.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.989 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:35.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:35.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:35.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:35.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:35.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:35.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:35.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556173+0000", 2024-09-14T05:30:35.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006169038 2024-09-14T05:30:35.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:35.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556177+0000", 2024-09-14T05:30:35.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0010000000000003e-06 2024-09-14T05:30:35.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556188+0000", 2024-09-14T05:30:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0675e-05 2024-09-14T05:30:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556199+0000", 2024-09-14T05:30:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.155e-05 2024-09-14T05:30:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646511+0000", 2024-09-14T05:30:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090312187000000002 2024-09-14T05:30:35.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646563+0000", 2024-09-14T05:30:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2219000000000003e-05 2024-09-14T05:30:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646863+0000", 2024-09-14T05:30:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029988600000000002 2024-09-14T05:30:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:35.993 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:35.993 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:35.994 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:30:35.994 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:35.993+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:36.094 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:36.094+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:36.150 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:36.150 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:36.151 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:36.151 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:36.159 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:30:36.195 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:36.195+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:36.296 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:36.296+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:36.303 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:36.303 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:36.303 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:36.303 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:36.303 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:36.312 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:30:36.397 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:36.396+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:36.456 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:36.456 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:36.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:36.456 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:36.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.457 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1989 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 16326080~54496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:36.457 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:36.457 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.893471773000002, 2024-09-14T05:30:36.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096664322999999996, 2024-09-14T05:30:36.457 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.457 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.457 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:36.457 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.457 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.457 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.457 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1989 2024-09-14T05:30:36.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.458 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:36.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:36.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561455+0000", 2024-09-14T05:30:36.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1699999999999997e-07 2024-09-14T05:30:36.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590703+0000", 2024-09-14T05:30:36.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029248225999999999 2024-09-14T05:30:36.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590732+0000", 2024-09-14T05:30:36.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8679000000000001e-05 2024-09-14T05:30:36.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590759+0000", 2024-09-14T05:30:36.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7319999999999999e-05 2024-09-14T05:30:36.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590835+0000", 2024-09-14T05:30:36.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5543999999999994e-05 2024-09-14T05:30:36.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590956+0000", 2024-09-14T05:30:36.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012168 2024-09-14T05:30:36.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 1", 2024-09-14T05:30:36.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.591261+0000", 2024-09-14T05:30:36.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030484799999999998 2024-09-14T05:30:36.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:36.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.637682+0000", 2024-09-14T05:30:36.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046420550999999997 2024-09-14T05:30:36.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:36.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.657912+0000", 2024-09-14T05:30:36.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020230011999999999 2024-09-14T05:30:36.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:36.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658007+0000", 2024-09-14T05:30:36.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5814000000000001e-05 2024-09-14T05:30:36.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658118+0000", 2024-09-14T05:30:36.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110832 2024-09-14T05:30:36.463 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.463 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.463 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.463 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2033 3.1e 3:7e47f6eb:::rbd_data.112d1f36d097.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30853661~63531] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:36.463 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:36.463 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.782197107000002, 2024-09-14T05:30:36.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065167630000000004, 2024-09-14T05:30:36.463 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.463 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.464 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:36.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.464 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2033 2024-09-14T05:30:36.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.464 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:36.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:36.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672730+0000", 2024-09-14T05:30:36.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.393e-06 2024-09-14T05:30:36.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691289+0000", 2024-09-14T05:30:36.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018558991 2024-09-14T05:30:36.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691316+0000", 2024-09-14T05:30:36.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7498999999999999e-05 2024-09-14T05:30:36.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691331+0000", 2024-09-14T05:30:36.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4652e-05 2024-09-14T05:30:36.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691397+0000", 2024-09-14T05:30:36.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6112000000000007e-05 2024-09-14T05:30:36.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691509+0000", 2024-09-14T05:30:36.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111599 2024-09-14T05:30:36.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:36.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.692253+0000", 2024-09-14T05:30:36.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074456200000000004 2024-09-14T05:30:36.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:36.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.719140+0000", 2024-09-14T05:30:36.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026886542999999999 2024-09-14T05:30:36.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:36.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737789+0000", 2024-09-14T05:30:36.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018649203999999999 2024-09-14T05:30:36.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:36.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737846+0000", 2024-09-14T05:30:36.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6501e-05 2024-09-14T05:30:36.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737896+0000", 2024-09-14T05:30:36.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0574000000000001e-05 2024-09-14T05:30:36.469 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.469 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.469 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.469 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3008 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13729792~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:36.469 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:36.469 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.151273236999998, 2024-09-14T05:30:36.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061007651000000003, 2024-09-14T05:30:36.470 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.470 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.470 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:36.470 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.470 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.470 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.470 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3008 2024-09-14T05:30:36.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.470 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:36.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:36.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303653+0000", 2024-09-14T05:30:36.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0699999999999999e-07 2024-09-14T05:30:36.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303654+0000", 2024-09-14T05:30:36.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-07 2024-09-14T05:30:36.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303657+0000", 2024-09-14T05:30:36.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2890000000000002e-06 2024-09-14T05:30:36.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303661+0000", 2024-09-14T05:30:36.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3290000000000004e-06 2024-09-14T05:30:36.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364233+0000", 2024-09-14T05:30:36.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060572134 2024-09-14T05:30:36.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364315+0000", 2024-09-14T05:30:36.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1549000000000001e-05 2024-09-14T05:30:36.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364660+0000", 2024-09-14T05:30:36.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034525300000000001 2024-09-14T05:30:36.474 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.474 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.474 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.474 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3009 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14221312~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:36.474 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:36.474 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.151259147000005, 2024-09-14T05:30:36.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061445063000000001, 2024-09-14T05:30:36.474 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.474 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.475 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:36.475 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.475 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.475 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.475 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3009 2024-09-14T05:30:36.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.475 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:36.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:36.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:36.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:36.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303669+0000", 2024-09-14T05:30:36.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7150000000000001e-06 2024-09-14T05:30:36.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303675+0000", 2024-09-14T05:30:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5339999999999996e-06 2024-09-14T05:30:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303681+0000", 2024-09-14T05:30:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3639999999999998e-06 2024-09-14T05:30:36.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364705+0000", 2024-09-14T05:30:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061024123 2024-09-14T05:30:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364786+0000", 2024-09-14T05:30:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1378999999999995e-05 2024-09-14T05:30:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365112+0000", 2024-09-14T05:30:36.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032558700000000001 2024-09-14T05:30:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3010 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14712832~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.151238282999998, 2024-09-14T05:30:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061834630000000002, 2024-09-14T05:30:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.482 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3010 2024-09-14T05:30:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303688+0000", 2024-09-14T05:30:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:30:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303690+0000", 2024-09-14T05:30:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9460000000000001e-06 2024-09-14T05:30:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303696+0000", 2024-09-14T05:30:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2230000000000004e-06 2024-09-14T05:30:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303700+0000", 2024-09-14T05:30:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7290000000000002e-06 2024-09-14T05:30:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365163+0000", 2024-09-14T05:30:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061463093000000003 2024-09-14T05:30:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365214+0000", 2024-09-14T05:30:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0977999999999999e-05 2024-09-14T05:30:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365522+0000", 2024-09-14T05:30:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000308227 2024-09-14T05:30:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3241 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:36.486 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.125563861000003, 2024-09-14T05:30:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061041892, 2024-09-14T05:30:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3241 2024-09-14T05:30:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:36.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329496+0000", 2024-09-14T05:30:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013421799999999999 2024-09-14T05:30:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329499+0000", 2024-09-14T05:30:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8459999999999999e-06 2024-09-14T05:30:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329505+0000", 2024-09-14T05:30:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4320000000000004e-06 2024-09-14T05:30:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329509+0000", 2024-09-14T05:30:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0339999999999998e-06 2024-09-14T05:30:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.389947+0000", 2024-09-14T05:30:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060438093999999998 2024-09-14T05:30:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390038+0000", 2024-09-14T05:30:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0253999999999993e-05 2024-09-14T05:30:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390404+0000", 2024-09-14T05:30:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036601400000000001 2024-09-14T05:30:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.490 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3242 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.125408953000004, 2024-09-14T05:30:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061282426000000001, 2024-09-14T05:30:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.491 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3242 2024-09-14T05:30:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9900000000000001e-07 2024-09-14T05:30:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329518+0000", 2024-09-14T05:30:36.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.328e-06 2024-09-14T05:30:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329522+0000", 2024-09-14T05:30:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.563e-06 2024-09-14T05:30:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329525+0000", 2024-09-14T05:30:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7379999999999999e-06 2024-09-14T05:30:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390437+0000", 2024-09-14T05:30:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060912463 2024-09-14T05:30:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390498+0000", 2024-09-14T05:30:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0615999999999997e-05 2024-09-14T05:30:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390799+0000", 2024-09-14T05:30:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030131899999999998 2024-09-14T05:30:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.495 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3243 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.125397167000003, 2024-09-14T05:30:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061728980000000003, 2024-09-14T05:30:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.496 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3243 2024-09-14T05:30:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329692+0000", 2024-09-14T05:30:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000163948 2024-09-14T05:30:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329696+0000", 2024-09-14T05:30:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5540000000000002e-06 2024-09-14T05:30:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329706+0000", 2024-09-14T05:30:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7960000000000007e-06 2024-09-14T05:30:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329712+0000", 2024-09-14T05:30:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6109999999999999e-06 2024-09-14T05:30:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390861+0000", 2024-09-14T05:30:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061148767 2024-09-14T05:30:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390942+0000", 2024-09-14T05:30:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1266000000000004e-05 2024-09-14T05:30:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391258+0000", 2024-09-14T05:30:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031503800000000002 2024-09-14T05:30:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3244 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14778368~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.125202118999994, 2024-09-14T05:30:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062050594000000001, 2024-09-14T05:30:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.500 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3244 2024-09-14T05:30:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329916+0000", 2024-09-14T05:30:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019290799999999999 2024-09-14T05:30:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329919+0000", 2024-09-14T05:30:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9019999999999999e-06 2024-09-14T05:30:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329926+0000", 2024-09-14T05:30:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9e-06 2024-09-14T05:30:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329932+0000", 2024-09-14T05:30:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.088e-06 2024-09-14T05:30:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391299+0000", 2024-09-14T05:30:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061366136000000002 2024-09-14T05:30:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391391+0000", 2024-09-14T05:30:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2204e-05 2024-09-14T05:30:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391774+0000", 2024-09-14T05:30:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038345600000000001 2024-09-14T05:30:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3921 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 5308416~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.053658308999999, 2024-09-14T05:30:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060842694000000003, 2024-09-14T05:30:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3921 2024-09-14T05:30:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401705+0000", 2024-09-14T05:30:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768499999999999 2024-09-14T05:30:36.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401709+0000", 2024-09-14T05:30:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0319999999999997e-06 2024-09-14T05:30:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401720+0000", 2024-09-14T05:30:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1005000000000001e-05 2024-09-14T05:30:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401727+0000", 2024-09-14T05:30:36.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1879999999999996e-06 2024-09-14T05:30:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461748+0000", 2024-09-14T05:30:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060021199999999997 2024-09-14T05:30:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461793+0000", 2024-09-14T05:30:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4530999999999998e-05 2024-09-14T05:30:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462110+0000", 2024-09-14T05:30:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031705299999999997 2024-09-14T05:30:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3923 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 6291456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.053012280999994, 2024-09-14T05:30:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060891273000000003, 2024-09-14T05:30:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.509 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3923 2024-09-14T05:30:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407054+0000", 2024-09-14T05:30:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005140787 2024-09-14T05:30:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407059+0000", 2024-09-14T05:30:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.04e-06 2024-09-14T05:30:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407069+0000", 2024-09-14T05:30:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4429999999999992e-06 2024-09-14T05:30:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407076+0000", 2024-09-14T05:30:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1679999999999998e-06 2024-09-14T05:30:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462514+0000", 2024-09-14T05:30:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055438551000000003 2024-09-14T05:30:36.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462562+0000", 2024-09-14T05:30:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7649999999999999e-05 2024-09-14T05:30:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462805+0000", 2024-09-14T05:30:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024263400000000001 2024-09-14T05:30:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4052 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 12550144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.037440271999998, 2024-09-14T05:30:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060872904999999998, 2024-09-14T05:30:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4052 2024-09-14T05:30:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417486+0000", 2024-09-14T05:30:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-14T05:30:36.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417487+0000", 2024-09-14T05:30:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4499999999999995e-07 2024-09-14T05:30:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417489+0000", 2024-09-14T05:30:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8899999999999999e-06 2024-09-14T05:30:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417492+0000", 2024-09-14T05:30:36.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4389999999999999e-06 2024-09-14T05:30:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478027+0000", 2024-09-14T05:30:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060534959999999999 2024-09-14T05:30:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478079+0000", 2024-09-14T05:30:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2104999999999997e-05 2024-09-14T05:30:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478358+0000", 2024-09-14T05:30:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000279351 2024-09-14T05:30:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4053 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13041664~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.037430888000003, 2024-09-14T05:30:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061258866000000002, 2024-09-14T05:30:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.518 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4053 2024-09-14T05:30:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:36.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.17e-07 2024-09-14T05:30:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417496+0000", 2024-09-14T05:30:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0890000000000001e-06 2024-09-14T05:30:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417499+0000", 2024-09-14T05:30:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8629999999999999e-06 2024-09-14T05:30:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417502+0000", 2024-09-14T05:30:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6290000000000002e-06 2024-09-14T05:30:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478387+0000", 2024-09-14T05:30:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060885433000000003 2024-09-14T05:30:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478435+0000", 2024-09-14T05:30:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8167999999999997e-05 2024-09-14T05:30:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478754+0000", 2024-09-14T05:30:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031846700000000001 2024-09-14T05:30:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.522 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4054 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13533184~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.037421003999995, 2024-09-14T05:30:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061604694000000002, 2024-09-14T05:30:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4054 2024-09-14T05:30:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.04e-07 2024-09-14T05:30:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417506+0000", 2024-09-14T05:30:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8299999999999995e-07 2024-09-14T05:30:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417509+0000", 2024-09-14T05:30:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.88e-06 2024-09-14T05:30:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417511+0000", 2024-09-14T05:30:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5950000000000001e-06 2024-09-14T05:30:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478784+0000", 2024-09-14T05:30:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061272971000000002 2024-09-14T05:30:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478841+0000", 2024-09-14T05:30:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-14T05:30:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479109+0000", 2024-09-14T05:30:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026794600000000003 2024-09-14T05:30:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4055 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14024704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.037411301999995, 2024-09-14T05:30:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061932299000000003, 2024-09-14T05:30:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4055 2024-09-14T05:30:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417515+0000", 2024-09-14T05:30:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-14T05:30:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417516+0000", 2024-09-14T05:30:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7199999999999997e-07 2024-09-14T05:30:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417519+0000", 2024-09-14T05:30:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.937e-06 2024-09-14T05:30:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417522+0000", 2024-09-14T05:30:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1420000000000002e-06 2024-09-14T05:30:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479140+0000", 2024-09-14T05:30:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061617933999999999 2024-09-14T05:30:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479188+0000", 2024-09-14T05:30:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8717000000000001e-05 2024-09-14T05:30:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479447+0000", 2024-09-14T05:30:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025838800000000002 2024-09-14T05:30:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4056 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14516224~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.037401129000003, 2024-09-14T05:30:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062263185999999998, 2024-09-14T05:30:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4056 2024-09-14T05:30:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8700000000000002e-07 2024-09-14T05:30:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417528+0000", 2024-09-14T05:30:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.579e-06 2024-09-14T05:30:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417533+0000", 2024-09-14T05:30:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2400000000000001e-06 2024-09-14T05:30:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417536+0000", 2024-09-14T05:30:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0309999999999999e-06 2024-09-14T05:30:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479477+0000", 2024-09-14T05:30:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061940967 2024-09-14T05:30:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479523+0000", 2024-09-14T05:30:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6335000000000003e-05 2024-09-14T05:30:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479788+0000", 2024-09-14T05:30:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026474700000000002 2024-09-14T05:30:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4057 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15007744~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.037386667999996, 2024-09-14T05:30:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062589928000000003, 2024-09-14T05:30:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.536 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4057 2024-09-14T05:30:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422838+0000", 2024-09-14T05:30:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052990349999999997 2024-09-14T05:30:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422854+0000", 2024-09-14T05:30:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5523999999999999e-05 2024-09-14T05:30:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422864+0000", 2024-09-14T05:30:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0084e-05 2024-09-14T05:30:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422872+0000", 2024-09-14T05:30:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-14T05:30:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479813+0000", 2024-09-14T05:30:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056941389000000002 2024-09-14T05:30:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479868+0000", 2024-09-14T05:30:36.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4973999999999999e-05 2024-09-14T05:30:36.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.480129+0000", 2024-09-14T05:30:36.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026084200000000001 2024-09-14T05:30:36.540 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.540 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.540 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.540 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4171 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14057472~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:36.540 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:36.540 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.025389107999999, 2024-09-14T05:30:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061184929999999998, 2024-09-14T05:30:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4171 2024-09-14T05:30:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.39e-07 2024-09-14T05:30:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429541+0000", 2024-09-14T05:30:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:30:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429547+0000", 2024-09-14T05:30:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3280000000000002e-06 2024-09-14T05:30:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429552+0000", 2024-09-14T05:30:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4859999999999997e-06 2024-09-14T05:30:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490379+0000", 2024-09-14T05:30:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060826681 2024-09-14T05:30:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490434+0000", 2024-09-14T05:30:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4577999999999998e-05 2024-09-14T05:30:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490722+0000", 2024-09-14T05:30:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028781400000000001 2024-09-14T05:30:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.544 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4172 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14548992~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.025367099999997, 2024-09-14T05:30:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061498256000000001, 2024-09-14T05:30:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.545 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4172 2024-09-14T05:30:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.72e-07 2024-09-14T05:30:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429561+0000", 2024-09-14T05:30:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.717e-06 2024-09-14T05:30:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429566+0000", 2024-09-14T05:30:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6529999999999999e-06 2024-09-14T05:30:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429571+0000", 2024-09-14T05:30:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5979999999999999e-06 2024-09-14T05:30:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490751+0000", 2024-09-14T05:30:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061180336000000002 2024-09-14T05:30:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490798+0000", 2024-09-14T05:30:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6875000000000001e-05 2024-09-14T05:30:36.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491057+0000", 2024-09-14T05:30:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025870500000000002 2024-09-14T05:30:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4173 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15040512~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.025349473999995, 2024-09-14T05:30:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061894820000000003, 2024-09-14T05:30:36.549 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4173 2024-09-14T05:30:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429577+0000", 2024-09-14T05:30:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-14T05:30:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429578+0000", 2024-09-14T05:30:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6980000000000001e-06 2024-09-14T05:30:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429585+0000", 2024-09-14T05:30:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.725e-06 2024-09-14T05:30:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429590+0000", 2024-09-14T05:30:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.938e-06 2024-09-14T05:30:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491091+0000", 2024-09-14T05:30:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061500773000000002 2024-09-14T05:30:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491136+0000", 2024-09-14T05:30:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5284000000000002e-05 2024-09-14T05:30:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491471+0000", 2024-09-14T05:30:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033485699999999998 2024-09-14T05:30:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:36.553 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:36.553 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:36.555 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:30:36.555 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:36.555+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:36.655 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:36.654+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:36.699 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:36.699 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:36.699 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:36.699 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:36.709 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:30:36.727 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:30:36.792 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:36.792+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:36.853 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:36.853 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:36.853 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:36.853 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:36.853 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:36.857 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:36.857+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:36.859 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:36.859 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:36.861 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:30:36.876 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:36.876 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:36.891 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:36.892 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:36.907 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:36.907 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:36.923 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:36.924 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:36.941 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:36.941 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:36.958 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:36.958 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:36.958 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:36.957+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:36.974 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:36.975 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:37.008 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:37.008 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:37.019 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:30:37.024 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T05:30:37.062 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:37.062+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:37.128 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:37.128 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:37.137 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:30:37.160 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:37.159+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:37.261 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:37.260+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:37.282 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:37.282 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:37.291 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:30:37.362 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:37.362+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:37.395 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:37.395 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:37.404 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:30:37.463 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:37.462+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:37.507 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:37.508 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:37.515 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:30:37.564 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:37.563+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:37.618 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:37.618 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:37.626 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:30:37.671 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:37.670+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:37.729 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:37.729 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:37.737 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:30:37.766 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:37.765+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:37.867 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:37.867+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:37.880 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:37.880 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:37.888 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:30:37.968 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:37.968+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:37.990 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:37.991 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:38.000 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:30:38.008 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-14T05:30:38.069 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:38.069+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:38.106 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:38.106 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:38.114 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:30:38.199 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:38.198+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:38.259 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:38.259 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:38.267 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:30:38.309 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:38.308+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:38.372 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:38.372+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:38.411 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:38.411 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:38.419 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:30:38.473 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:38.472+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:38.523 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:38.523 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:38.532 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:30:38.576 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:38.575+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:38.635 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:38.635 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:38.643 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:30:38.685 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:38.684+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:38.776 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:38.775+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:38.787 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:38.787 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:38.796 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:30:38.877 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:38.876+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:38.899 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:38.899 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:38.908 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:30:38.978 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:38.978+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:38.991 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:30:39.023 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T05:30:39.023 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T05:30:39.029 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:39.029 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:39.039 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:30:39.082 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:39.082+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:39.132 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:39.132 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:39.147 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:39.148 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:39.149 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:39.150 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:39.157 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:30:39.170 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:39.171 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:39.180 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:39.179+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:39.187 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:39.187 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:39.203 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:39.203 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:39.219 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:39.220 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:39.236 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:39.236 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:39.252 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:39.253 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:39.280 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:39.280+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:39.302 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:39.303 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:39.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:39.303 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:39.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.303 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3523 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6062080~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.303 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:39.303 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.787253290999999, 2024-09-14T05:30:39.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096911030999999995, 2024-09-14T05:30:39.303 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.303 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.303 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.303 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.303 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.303 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.304 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3523 2024-09-14T05:30:39.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.304 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:39.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:39.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514359+0000", 2024-09-14T05:30:39.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9299999999999999e-07 2024-09-14T05:30:39.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514361+0000", 2024-09-14T05:30:39.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:39.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514366+0000", 2024-09-14T05:30:39.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1660000000000002e-06 2024-09-14T05:30:39.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514371+0000", 2024-09-14T05:30:39.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.648e-06 2024-09-14T05:30:39.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.610954+0000", 2024-09-14T05:30:39.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096582170999999994 2024-09-14T05:30:39.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611002+0000", 2024-09-14T05:30:39.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7923999999999997e-05 2024-09-14T05:30:39.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611269+0000", 2024-09-14T05:30:39.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026789499999999998 2024-09-14T05:30:39.307 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.307 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.307 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.307 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3524 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6553600~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.307 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:39.308 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.787234099000003, 2024-09-14T05:30:39.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097263347999999999, 2024-09-14T05:30:39.308 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.308 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.308 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.308 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.308 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.308 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.308 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3524 2024-09-14T05:30:39.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.308 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:39.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:39.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:39.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:39.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514380+0000", 2024-09-14T05:30:39.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6500000000000001e-06 2024-09-14T05:30:39.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514386+0000", 2024-09-14T05:30:39.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7360000000000004e-06 2024-09-14T05:30:39.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514391+0000", 2024-09-14T05:30:39.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2830000000000003e-06 2024-09-14T05:30:39.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611295+0000", 2024-09-14T05:30:39.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096904814000000006 2024-09-14T05:30:39.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611343+0000", 2024-09-14T05:30:39.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7506999999999998e-05 2024-09-14T05:30:39.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611641+0000", 2024-09-14T05:30:39.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029798800000000001 2024-09-14T05:30:39.312 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.312 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.312 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.312 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3525 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7045120~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.312 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:39.312 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.787212212, 2024-09-14T05:30:39.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097644422999999994, 2024-09-14T05:30:39.312 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.312 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.312 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.312 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.313 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.313 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.313 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3525 2024-09-14T05:30:39.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.313 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:39.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:39.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514400+0000", 2024-09-14T05:30:39.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.77e-07 2024-09-14T05:30:39.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514402+0000", 2024-09-14T05:30:39.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6530000000000001e-06 2024-09-14T05:30:39.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514411+0000", 2024-09-14T05:30:39.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7429999999999997e-06 2024-09-14T05:30:39.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514417+0000", 2024-09-14T05:30:39.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2249999999999997e-06 2024-09-14T05:30:39.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611668+0000", 2024-09-14T05:30:39.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097250681000000005 2024-09-14T05:30:39.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611716+0000", 2024-09-14T05:30:39.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7336000000000002e-05 2024-09-14T05:30:39.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612044+0000", 2024-09-14T05:30:39.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032840799999999998 2024-09-14T05:30:39.316 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.316 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.316 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.316 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3526 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7536640~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.316 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:39.317 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.787187728000006, 2024-09-14T05:30:39.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097943884999999994, 2024-09-14T05:30:39.317 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.317 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.317 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.317 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.317 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.317 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.317 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3526 2024-09-14T05:30:39.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.317 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:39.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:39.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:39.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0400000000000002e-07 2024-09-14T05:30:39.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514426+0000", 2024-09-14T05:30:39.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:39.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514432+0000", 2024-09-14T05:30:39.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.105e-06 2024-09-14T05:30:39.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514438+0000", 2024-09-14T05:30:39.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5149999999999997e-06 2024-09-14T05:30:39.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612072+0000", 2024-09-14T05:30:39.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097633947999999998 2024-09-14T05:30:39.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612120+0000", 2024-09-14T05:30:39.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8458999999999998e-05 2024-09-14T05:30:39.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612368+0000", 2024-09-14T05:30:39.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024761999999999999 2024-09-14T05:30:39.320 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.321 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.321 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.321 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3527 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 8028160~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.321 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:39.321 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.787167014999994, 2024-09-14T05:30:39.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098244955999999994, 2024-09-14T05:30:39.321 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.321 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.321 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.321 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.321 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.321 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.322 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3527 2024-09-14T05:30:39.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.322 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:39.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:39.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519842+0000", 2024-09-14T05:30:39.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053975230000000004 2024-09-14T05:30:39.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519851+0000", 2024-09-14T05:30:39.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5599999999999994e-06 2024-09-14T05:30:39.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519858+0000", 2024-09-14T05:30:39.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2409999999999998e-06 2024-09-14T05:30:39.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519863+0000", 2024-09-14T05:30:39.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:30:39.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612395+0000", 2024-09-14T05:30:39.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092532552000000004 2024-09-14T05:30:39.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612442+0000", 2024-09-14T05:30:39.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6486000000000002e-05 2024-09-14T05:30:39.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612690+0000", 2024-09-14T05:30:39.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024791599999999999 2024-09-14T05:30:39.325 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.325 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.325 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.325 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3603 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5177344~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.325 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:39.325 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.781614570000002, 2024-09-14T05:30:39.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096796436, 2024-09-14T05:30:39.325 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.326 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.326 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.326 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.326 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.326 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.326 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3603 2024-09-14T05:30:39.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.326 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:39.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:39.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:39.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1899999999999999e-07 2024-09-14T05:30:39.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519998+0000", 2024-09-14T05:30:39.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5000000000000001e-07 2024-09-14T05:30:39.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520002+0000", 2024-09-14T05:30:39.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7299999999999999e-06 2024-09-14T05:30:39.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520004+0000", 2024-09-14T05:30:39.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.429e-06 2024-09-14T05:30:39.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616493+0000", 2024-09-14T05:30:39.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096488345000000003 2024-09-14T05:30:39.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616544+0000", 2024-09-14T05:30:39.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1724000000000001e-05 2024-09-14T05:30:39.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616794+0000", 2024-09-14T05:30:39.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000249139 2024-09-14T05:30:39.329 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.329 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.330 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.330 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3604 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5668864~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.330 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:39.330 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.781599932000006, 2024-09-14T05:30:39.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108222999999994, 2024-09-14T05:30:39.330 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.330 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.330 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.330 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.330 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.330 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.330 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3604 2024-09-14T05:30:39.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.331 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:39.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:39.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:39.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7000000000000001e-07 2024-09-14T05:30:39.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520013+0000", 2024-09-14T05:30:39.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.389e-06 2024-09-14T05:30:39.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520018+0000", 2024-09-14T05:30:39.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3019999999999996e-06 2024-09-14T05:30:39.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520026+0000", 2024-09-14T05:30:39.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6170000000000007e-06 2024-09-14T05:30:39.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616816+0000", 2024-09-14T05:30:39.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096789766999999999 2024-09-14T05:30:39.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616871+0000", 2024-09-14T05:30:39.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4811000000000001e-05 2024-09-14T05:30:39.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617120+0000", 2024-09-14T05:30:39.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024906700000000001 2024-09-14T05:30:39.334 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.334 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.334 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.334 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3605 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6160384~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.334 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:39.334 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.781581471999999, 2024-09-14T05:30:39.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097423945999999997, 2024-09-14T05:30:39.335 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.335 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.335 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.335 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.335 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.335 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.335 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3605 2024-09-14T05:30:39.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.335 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:39.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:39.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:39.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3999999999999998e-07 2024-09-14T05:30:39.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520032+0000", 2024-09-14T05:30:39.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.083e-06 2024-09-14T05:30:39.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520036+0000", 2024-09-14T05:30:39.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.019e-06 2024-09-14T05:30:39.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520038+0000", 2024-09-14T05:30:39.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4849999999999999e-06 2024-09-14T05:30:39.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617147+0000", 2024-09-14T05:30:39.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108665999999996 2024-09-14T05:30:39.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617199+0000", 2024-09-14T05:30:39.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2558000000000001e-05 2024-09-14T05:30:39.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617454+0000", 2024-09-14T05:30:39.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025489499999999999 2024-09-14T05:30:39.338 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.338 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.339 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.339 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3606 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6651904~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.339 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:39.339 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.78157075, 2024-09-14T05:30:39.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097745705000000002, 2024-09-14T05:30:39.339 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.339 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.339 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.339 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.339 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.339 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.340 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3606 2024-09-14T05:30:39.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.340 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:39.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:39.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:39.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-14T05:30:39.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520042+0000", 2024-09-14T05:30:39.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4799999999999997e-07 2024-09-14T05:30:39.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520046+0000", 2024-09-14T05:30:39.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8149999999999999e-06 2024-09-14T05:30:39.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520049+0000", 2024-09-14T05:30:39.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.576e-06 2024-09-14T05:30:39.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617476+0000", 2024-09-14T05:30:39.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097427558999999997 2024-09-14T05:30:39.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617522+0000", 2024-09-14T05:30:39.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5927999999999998e-05 2024-09-14T05:30:39.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617787+0000", 2024-09-14T05:30:39.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000264663 2024-09-14T05:30:39.343 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.343 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.343 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.344 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3607 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7143424~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.344 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:39.344 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.781560223, 2024-09-14T05:30:39.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097871125000000003, 2024-09-14T05:30:39.344 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.344 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.344 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.344 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.344 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.344 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.344 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3607 2024-09-14T05:30:39.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.344 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:39.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:39.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520052+0000", 2024-09-14T05:30:39.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.03e-07 2024-09-14T05:30:39.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520053+0000", 2024-09-14T05:30:39.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.85e-07 2024-09-14T05:30:39.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520056+0000", 2024-09-14T05:30:39.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9140000000000001e-06 2024-09-14T05:30:39.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520059+0000", 2024-09-14T05:30:39.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4600000000000002e-06 2024-09-14T05:30:39.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617809+0000", 2024-09-14T05:30:39.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097750216000000001 2024-09-14T05:30:39.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617864+0000", 2024-09-14T05:30:39.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5211999999999999e-05 2024-09-14T05:30:39.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617923+0000", 2024-09-14T05:30:39.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8235000000000001e-05 2024-09-14T05:30:39.348 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.348 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.348 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.348 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3608 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7634944~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.348 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:39.348 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.781549914999999, 2024-09-14T05:30:39.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098199212999999994, 2024-09-14T05:30:39.348 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.348 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.348 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.348 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.348 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.349 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.349 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3608 2024-09-14T05:30:39.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.349 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:39.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:39.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525953+0000", 2024-09-14T05:30:39.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058911850000000002 2024-09-14T05:30:39.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525957+0000", 2024-09-14T05:30:39.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2379999999999999e-06 2024-09-14T05:30:39.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525967+0000", 2024-09-14T05:30:39.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5389999999999996e-06 2024-09-14T05:30:39.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525972+0000", 2024-09-14T05:30:39.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.541e-06 2024-09-14T05:30:39.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617945+0000", 2024-09-14T05:30:39.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091972597000000003 2024-09-14T05:30:39.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617992+0000", 2024-09-14T05:30:39.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7397e-05 2024-09-14T05:30:39.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.618261+0000", 2024-09-14T05:30:39.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026871600000000001 2024-09-14T05:30:39.352 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.352 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.352 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.352 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3640 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6586368~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.352 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:39.352 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.775442072999994, 2024-09-14T05:30:39.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097068580000000002, 2024-09-14T05:30:39.353 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.353 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.353 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.353 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.353 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.353 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.353 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3640 2024-09-14T05:30:39.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.353 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:39.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:39.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:39.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.23e-07 2024-09-14T05:30:39.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526171+0000", 2024-09-14T05:30:39.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:30:39.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526175+0000", 2024-09-14T05:30:39.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8369999999999999e-06 2024-09-14T05:30:39.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526177+0000", 2024-09-14T05:30:39.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4959999999999999e-06 2024-09-14T05:30:39.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622847+0000", 2024-09-14T05:30:39.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096669790000000005 2024-09-14T05:30:39.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622902+0000", 2024-09-14T05:30:39.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5157000000000001e-05 2024-09-14T05:30:39.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623238+0000", 2024-09-14T05:30:39.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033603399999999997 2024-09-14T05:30:39.356 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.356 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.356 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.357 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3641 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7077888~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.357 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:39.357 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.775431562999998, 2024-09-14T05:30:39.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097371141999999994, 2024-09-14T05:30:39.357 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.357 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.357 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.357 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.357 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.357 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.357 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3641 2024-09-14T05:30:39.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.357 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:39.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:39.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531851+0000", 2024-09-14T05:30:39.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005671265 2024-09-14T05:30:39.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531855+0000", 2024-09-14T05:30:39.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0849999999999999e-06 2024-09-14T05:30:39.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531867+0000", 2024-09-14T05:30:39.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1028e-05 2024-09-14T05:30:39.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531872+0000", 2024-09-14T05:30:39.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000003e-06 2024-09-14T05:30:39.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623251+0000", 2024-09-14T05:30:39.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091378980999999998 2024-09-14T05:30:39.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623295+0000", 2024-09-14T05:30:39.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3881999999999998e-05 2024-09-14T05:30:39.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623551+0000", 2024-09-14T05:30:39.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025636099999999998 2024-09-14T05:30:39.361 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.361 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.361 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.361 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3721 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.361 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:39.361 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.769573288000004, 2024-09-14T05:30:39.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096806472000000005, 2024-09-14T05:30:39.361 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.361 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.361 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.362 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.362 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.362 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.362 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3721 2024-09-14T05:30:39.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.362 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:39.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:39.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532039+0000", 2024-09-14T05:30:39.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.05e-07 2024-09-14T05:30:39.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532040+0000", 2024-09-14T05:30:39.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9099999999999991e-07 2024-09-14T05:30:39.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532043+0000", 2024-09-14T05:30:39.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6610000000000002e-06 2024-09-14T05:30:39.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532046+0000", 2024-09-14T05:30:39.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4080000000000001e-06 2024-09-14T05:30:39.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628545+0000", 2024-09-14T05:30:39.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096499615999999996 2024-09-14T05:30:39.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628592+0000", 2024-09-14T05:30:39.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6907000000000003e-05 2024-09-14T05:30:39.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628845+0000", 2024-09-14T05:30:39.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025268399999999998 2024-09-14T05:30:39.368 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.368 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.369 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.369 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3722 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6193152~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.369 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:39.369 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.769563207999994, 2024-09-14T05:30:39.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097146587000000006, 2024-09-14T05:30:39.369 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.369 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.369 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.369 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.369 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.369 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.370 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3722 2024-09-14T05:30:39.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.370 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:39.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:39.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538191+0000", 2024-09-14T05:30:39.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061429969999999999 2024-09-14T05:30:39.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538200+0000", 2024-09-14T05:30:39.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7069999999999998e-06 2024-09-14T05:30:39.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538212+0000", 2024-09-14T05:30:39.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1504e-05 2024-09-14T05:30:39.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538219+0000", 2024-09-14T05:30:39.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7500000000000003e-06 2024-09-14T05:30:39.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628869+0000", 2024-09-14T05:30:39.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090649224 2024-09-14T05:30:39.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628919+0000", 2024-09-14T05:30:39.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0144e-05 2024-09-14T05:30:39.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.629195+0000", 2024-09-14T05:30:39.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027626099999999998 2024-09-14T05:30:39.373 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.373 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.373 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.373 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3758 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7110656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.373 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:39.373 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.762968991999998, 2024-09-14T05:30:39.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096977544999999998, 2024-09-14T05:30:39.374 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.374 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.374 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.374 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.374 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.374 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.374 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3758 2024-09-14T05:30:39.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.374 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:39.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:39.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:39.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:39.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538645+0000", 2024-09-14T05:30:39.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.843e-06 2024-09-14T05:30:39.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538651+0000", 2024-09-14T05:30:39.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6169999999999998e-06 2024-09-14T05:30:39.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538655+0000", 2024-09-14T05:30:39.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5399999999999997e-06 2024-09-14T05:30:39.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635225+0000", 2024-09-14T05:30:39.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096570149999999993 2024-09-14T05:30:39.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635272+0000", 2024-09-14T05:30:39.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7199e-05 2024-09-14T05:30:39.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635620+0000", 2024-09-14T05:30:39.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034782600000000003 2024-09-14T05:30:39.377 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.377 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.377 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.378 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3759 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.378 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:39.378 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.762951080999997, 2024-09-14T05:30:39.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097335331999999997, 2024-09-14T05:30:39.378 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.378 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.378 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.378 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.378 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.378 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.378 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3759 2024-09-14T05:30:39.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.378 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:39.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:39.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544013+0000", 2024-09-14T05:30:39.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053520829999999997 2024-09-14T05:30:39.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544016+0000", 2024-09-14T05:30:39.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6660000000000001e-06 2024-09-14T05:30:39.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544026+0000", 2024-09-14T05:30:39.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7089999999999997e-06 2024-09-14T05:30:39.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544043+0000", 2024-09-14T05:30:39.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833999999999998e-05 2024-09-14T05:30:39.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635646+0000", 2024-09-14T05:30:39.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091603544999999995 2024-09-14T05:30:39.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635696+0000", 2024-09-14T05:30:39.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9307000000000001e-05 2024-09-14T05:30:39.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635996+0000", 2024-09-14T05:30:39.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030018800000000001 2024-09-14T05:30:39.382 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.382 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.382 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.382 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3838 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6225920~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.382 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:39.382 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.757289400999994, 2024-09-14T05:30:39.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096894235999999995, 2024-09-14T05:30:39.382 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.382 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.382 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.382 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.382 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.382 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.383 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3838 2024-09-14T05:30:39.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.383 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:39.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:39.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544323+0000", 2024-09-14T05:30:39.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-14T05:30:39.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544325+0000", 2024-09-14T05:30:39.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.75e-06 2024-09-14T05:30:39.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544331+0000", 2024-09-14T05:30:39.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5019999999999998e-06 2024-09-14T05:30:39.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544335+0000", 2024-09-14T05:30:39.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2989999999999998e-06 2024-09-14T05:30:39.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640869+0000", 2024-09-14T05:30:39.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096533351000000003 2024-09-14T05:30:39.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640919+0000", 2024-09-14T05:30:39.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999999e-05 2024-09-14T05:30:39.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641217+0000", 2024-09-14T05:30:39.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029717600000000002 2024-09-14T05:30:39.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.386 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.386 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3839 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.386 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:39.386 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.757270462999998, 2024-09-14T05:30:39.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097240339999999995, 2024-09-14T05:30:39.386 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.386 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.387 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.387 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3839 2024-09-14T05:30:39.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.387 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:39.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:39.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549834+0000", 2024-09-14T05:30:39.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054927819999999999 2024-09-14T05:30:39.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549838+0000", 2024-09-14T05:30:39.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8689999999999997e-06 2024-09-14T05:30:39.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549848+0000", 2024-09-14T05:30:39.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0154e-05 2024-09-14T05:30:39.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549853+0000", 2024-09-14T05:30:39.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9740000000000001e-06 2024-09-14T05:30:39.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641245+0000", 2024-09-14T05:30:39.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091392245999999996 2024-09-14T05:30:39.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641295+0000", 2024-09-14T05:30:39.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0018999999999997e-05 2024-09-14T05:30:39.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641582+0000", 2024-09-14T05:30:39.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028629600000000001 2024-09-14T05:30:39.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.390 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.390 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3868 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.391 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:39.391 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.751607884999999, 2024-09-14T05:30:39.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096859555999999999, 2024-09-14T05:30:39.391 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.391 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.391 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.391 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.391 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.391 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.391 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3868 2024-09-14T05:30:39.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.391 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:39.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:39.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556173+0000", 2024-09-14T05:30:39.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006169038 2024-09-14T05:30:39.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556177+0000", 2024-09-14T05:30:39.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0010000000000003e-06 2024-09-14T05:30:39.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556188+0000", 2024-09-14T05:30:39.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0675e-05 2024-09-14T05:30:39.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556199+0000", 2024-09-14T05:30:39.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.155e-05 2024-09-14T05:30:39.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646511+0000", 2024-09-14T05:30:39.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090312187000000002 2024-09-14T05:30:39.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646563+0000", 2024-09-14T05:30:39.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2219000000000003e-05 2024-09-14T05:30:39.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646863+0000", 2024-09-14T05:30:39.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029988600000000002 2024-09-14T05:30:39.394 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.395 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.395 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.395 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.395 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.395 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:39.396 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:30:39.397 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:39.396+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:39.497 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:39.496+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:39.540 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:39.541 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:39.541 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:39.541 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:39.549 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:30:39.634 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:39.633+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:39.693 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:39.693 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:39.693 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:39.693 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:39.693 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:39.698 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:39.698+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:39.701 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:30:39.799 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:39.799+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:39.847 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:39.848 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:39.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:39.848 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:39.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.848 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1989 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 16326080~54496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:39.848 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:39.848 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.284869580000006, 2024-09-14T05:30:39.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096664322999999996, 2024-09-14T05:30:39.848 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.848 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.848 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:39.849 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.849 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.849 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.849 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1989 2024-09-14T05:30:39.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.849 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:39.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:39.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561455+0000", 2024-09-14T05:30:39.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1699999999999997e-07 2024-09-14T05:30:39.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590703+0000", 2024-09-14T05:30:39.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029248225999999999 2024-09-14T05:30:39.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590732+0000", 2024-09-14T05:30:39.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8679000000000001e-05 2024-09-14T05:30:39.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590759+0000", 2024-09-14T05:30:39.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7319999999999999e-05 2024-09-14T05:30:39.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590835+0000", 2024-09-14T05:30:39.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5543999999999994e-05 2024-09-14T05:30:39.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590956+0000", 2024-09-14T05:30:39.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012168 2024-09-14T05:30:39.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 1", 2024-09-14T05:30:39.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.591261+0000", 2024-09-14T05:30:39.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030484799999999998 2024-09-14T05:30:39.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:39.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.637682+0000", 2024-09-14T05:30:39.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046420550999999997 2024-09-14T05:30:39.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:39.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.657912+0000", 2024-09-14T05:30:39.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020230011999999999 2024-09-14T05:30:39.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:39.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658007+0000", 2024-09-14T05:30:39.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5814000000000001e-05 2024-09-14T05:30:39.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658118+0000", 2024-09-14T05:30:39.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110832 2024-09-14T05:30:39.854 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.854 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.854 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.854 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2033 3.1e 3:7e47f6eb:::rbd_data.112d1f36d097.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30853661~63531] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:39.854 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:39.854 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.173594914000006, 2024-09-14T05:30:39.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065167630000000004, 2024-09-14T05:30:39.854 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.854 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.855 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:39.855 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.855 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.855 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.855 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2033 2024-09-14T05:30:39.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.855 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:39.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:39.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672730+0000", 2024-09-14T05:30:39.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.393e-06 2024-09-14T05:30:39.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691289+0000", 2024-09-14T05:30:39.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018558991 2024-09-14T05:30:39.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691316+0000", 2024-09-14T05:30:39.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7498999999999999e-05 2024-09-14T05:30:39.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691331+0000", 2024-09-14T05:30:39.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4652e-05 2024-09-14T05:30:39.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691397+0000", 2024-09-14T05:30:39.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6112000000000007e-05 2024-09-14T05:30:39.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691509+0000", 2024-09-14T05:30:39.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111599 2024-09-14T05:30:39.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:39.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.692253+0000", 2024-09-14T05:30:39.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074456200000000004 2024-09-14T05:30:39.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:39.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.719140+0000", 2024-09-14T05:30:39.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026886542999999999 2024-09-14T05:30:39.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:39.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737789+0000", 2024-09-14T05:30:39.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018649203999999999 2024-09-14T05:30:39.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:39.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737846+0000", 2024-09-14T05:30:39.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6501e-05 2024-09-14T05:30:39.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737896+0000", 2024-09-14T05:30:39.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0574000000000001e-05 2024-09-14T05:30:39.860 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.860 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.860 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.860 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3008 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13729792~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.860 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:39.860 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.542671044000002, 2024-09-14T05:30:39.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061007651000000003, 2024-09-14T05:30:39.860 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.861 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.861 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.861 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.861 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.861 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.861 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3008 2024-09-14T05:30:39.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.861 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:39.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:39.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303653+0000", 2024-09-14T05:30:39.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0699999999999999e-07 2024-09-14T05:30:39.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303654+0000", 2024-09-14T05:30:39.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-07 2024-09-14T05:30:39.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303657+0000", 2024-09-14T05:30:39.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2890000000000002e-06 2024-09-14T05:30:39.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303661+0000", 2024-09-14T05:30:39.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3290000000000004e-06 2024-09-14T05:30:39.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364233+0000", 2024-09-14T05:30:39.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060572134 2024-09-14T05:30:39.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364315+0000", 2024-09-14T05:30:39.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1549000000000001e-05 2024-09-14T05:30:39.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364660+0000", 2024-09-14T05:30:39.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034525300000000001 2024-09-14T05:30:39.864 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.864 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.865 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.865 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3009 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14221312~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.865 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:39.865 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.542656953999995, 2024-09-14T05:30:39.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061445063000000001, 2024-09-14T05:30:39.865 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.865 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.865 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.865 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.865 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.865 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.865 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3009 2024-09-14T05:30:39.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.866 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:39.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:39.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:39.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:39.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303669+0000", 2024-09-14T05:30:39.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7150000000000001e-06 2024-09-14T05:30:39.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303675+0000", 2024-09-14T05:30:39.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5339999999999996e-06 2024-09-14T05:30:39.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303681+0000", 2024-09-14T05:30:39.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3639999999999998e-06 2024-09-14T05:30:39.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364705+0000", 2024-09-14T05:30:39.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061024123 2024-09-14T05:30:39.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364786+0000", 2024-09-14T05:30:39.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1378999999999995e-05 2024-09-14T05:30:39.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365112+0000", 2024-09-14T05:30:39.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032558700000000001 2024-09-14T05:30:39.869 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.869 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.869 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.869 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3010 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14712832~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.869 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:39.869 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.542636090000002, 2024-09-14T05:30:39.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061834630000000002, 2024-09-14T05:30:39.870 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.870 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.870 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.870 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.870 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.870 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.870 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3010 2024-09-14T05:30:39.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.870 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:39.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:39.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303688+0000", 2024-09-14T05:30:39.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:30:39.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303690+0000", 2024-09-14T05:30:39.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9460000000000001e-06 2024-09-14T05:30:39.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303696+0000", 2024-09-14T05:30:39.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2230000000000004e-06 2024-09-14T05:30:39.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303700+0000", 2024-09-14T05:30:39.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7290000000000002e-06 2024-09-14T05:30:39.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365163+0000", 2024-09-14T05:30:39.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061463093000000003 2024-09-14T05:30:39.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365214+0000", 2024-09-14T05:30:39.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0977999999999999e-05 2024-09-14T05:30:39.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365522+0000", 2024-09-14T05:30:39.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000308227 2024-09-14T05:30:39.874 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.874 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.874 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.874 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3241 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.874 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:39.874 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.516961667999993, 2024-09-14T05:30:39.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061041892, 2024-09-14T05:30:39.874 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.874 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.874 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.875 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.875 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.875 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.875 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3241 2024-09-14T05:30:39.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.875 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:39.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:39.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329496+0000", 2024-09-14T05:30:39.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013421799999999999 2024-09-14T05:30:39.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329499+0000", 2024-09-14T05:30:39.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8459999999999999e-06 2024-09-14T05:30:39.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329505+0000", 2024-09-14T05:30:39.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4320000000000004e-06 2024-09-14T05:30:39.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329509+0000", 2024-09-14T05:30:39.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0339999999999998e-06 2024-09-14T05:30:39.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.389947+0000", 2024-09-14T05:30:39.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060438093999999998 2024-09-14T05:30:39.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390038+0000", 2024-09-14T05:30:39.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0253999999999993e-05 2024-09-14T05:30:39.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390404+0000", 2024-09-14T05:30:39.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036601400000000001 2024-09-14T05:30:39.878 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.878 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.879 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.879 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3242 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.879 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:39.879 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.516806759999994, 2024-09-14T05:30:39.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061282426000000001, 2024-09-14T05:30:39.879 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.879 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.879 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.879 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3242 2024-09-14T05:30:39.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.880 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:39.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:39.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:39.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9900000000000001e-07 2024-09-14T05:30:39.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329518+0000", 2024-09-14T05:30:39.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.328e-06 2024-09-14T05:30:39.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329522+0000", 2024-09-14T05:30:39.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.563e-06 2024-09-14T05:30:39.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329525+0000", 2024-09-14T05:30:39.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7379999999999999e-06 2024-09-14T05:30:39.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390437+0000", 2024-09-14T05:30:39.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060912463 2024-09-14T05:30:39.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390498+0000", 2024-09-14T05:30:39.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0615999999999997e-05 2024-09-14T05:30:39.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390799+0000", 2024-09-14T05:30:39.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030131899999999998 2024-09-14T05:30:39.883 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.883 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.883 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.883 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3243 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.883 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:39.883 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.516794974000007, 2024-09-14T05:30:39.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061728980000000003, 2024-09-14T05:30:39.884 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.884 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.884 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.884 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.884 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.884 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.884 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3243 2024-09-14T05:30:39.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.884 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:39.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:39.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329692+0000", 2024-09-14T05:30:39.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000163948 2024-09-14T05:30:39.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329696+0000", 2024-09-14T05:30:39.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5540000000000002e-06 2024-09-14T05:30:39.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329706+0000", 2024-09-14T05:30:39.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7960000000000007e-06 2024-09-14T05:30:39.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329712+0000", 2024-09-14T05:30:39.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6109999999999999e-06 2024-09-14T05:30:39.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390861+0000", 2024-09-14T05:30:39.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061148767 2024-09-14T05:30:39.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390942+0000", 2024-09-14T05:30:39.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1266000000000004e-05 2024-09-14T05:30:39.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391258+0000", 2024-09-14T05:30:39.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031503800000000002 2024-09-14T05:30:39.888 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.888 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.888 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.888 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3244 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14778368~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.888 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:39.888 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.516599925999998, 2024-09-14T05:30:39.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062050594000000001, 2024-09-14T05:30:39.888 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.888 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.888 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.888 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.889 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.889 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.889 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3244 2024-09-14T05:30:39.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.889 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:39.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:39.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329916+0000", 2024-09-14T05:30:39.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019290799999999999 2024-09-14T05:30:39.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329919+0000", 2024-09-14T05:30:39.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9019999999999999e-06 2024-09-14T05:30:39.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329926+0000", 2024-09-14T05:30:39.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9e-06 2024-09-14T05:30:39.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329932+0000", 2024-09-14T05:30:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.088e-06 2024-09-14T05:30:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391299+0000", 2024-09-14T05:30:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061366136000000002 2024-09-14T05:30:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391391+0000", 2024-09-14T05:30:39.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2204e-05 2024-09-14T05:30:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391774+0000", 2024-09-14T05:30:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038345600000000001 2024-09-14T05:30:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3921 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 5308416~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:39.892 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.445056116000003, 2024-09-14T05:30:39.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060842694000000003, 2024-09-14T05:30:39.893 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.893 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.893 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.893 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.893 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.893 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.893 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3921 2024-09-14T05:30:39.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.893 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:39.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:39.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401705+0000", 2024-09-14T05:30:39.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768499999999999 2024-09-14T05:30:39.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401709+0000", 2024-09-14T05:30:39.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0319999999999997e-06 2024-09-14T05:30:39.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401720+0000", 2024-09-14T05:30:39.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1005000000000001e-05 2024-09-14T05:30:39.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401727+0000", 2024-09-14T05:30:39.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1879999999999996e-06 2024-09-14T05:30:39.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461748+0000", 2024-09-14T05:30:39.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060021199999999997 2024-09-14T05:30:39.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461793+0000", 2024-09-14T05:30:39.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4530999999999998e-05 2024-09-14T05:30:39.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462110+0000", 2024-09-14T05:30:39.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031705299999999997 2024-09-14T05:30:39.896 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.896 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.896 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.896 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3923 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 6291456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.897 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:39.897 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.444410087999998, 2024-09-14T05:30:39.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060891273000000003, 2024-09-14T05:30:39.897 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.897 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.897 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.897 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.897 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.897 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.897 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3923 2024-09-14T05:30:39.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.897 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:39.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:39.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407054+0000", 2024-09-14T05:30:39.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005140787 2024-09-14T05:30:39.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407059+0000", 2024-09-14T05:30:39.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.04e-06 2024-09-14T05:30:39.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407069+0000", 2024-09-14T05:30:39.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4429999999999992e-06 2024-09-14T05:30:39.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407076+0000", 2024-09-14T05:30:39.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1679999999999998e-06 2024-09-14T05:30:39.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462514+0000", 2024-09-14T05:30:39.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055438551000000003 2024-09-14T05:30:39.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462562+0000", 2024-09-14T05:30:39.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7649999999999999e-05 2024-09-14T05:30:39.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462805+0000", 2024-09-14T05:30:39.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024263400000000001 2024-09-14T05:30:39.901 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.901 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.901 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.901 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4052 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 12550144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.901 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:39.901 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.428838079000002, 2024-09-14T05:30:39.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060872904999999998, 2024-09-14T05:30:39.901 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.901 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.901 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.901 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.901 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.902 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.902 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4052 2024-09-14T05:30:39.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.902 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:39.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:39.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417486+0000", 2024-09-14T05:30:39.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-14T05:30:39.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417487+0000", 2024-09-14T05:30:39.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4499999999999995e-07 2024-09-14T05:30:39.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417489+0000", 2024-09-14T05:30:39.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8899999999999999e-06 2024-09-14T05:30:39.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417492+0000", 2024-09-14T05:30:39.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4389999999999999e-06 2024-09-14T05:30:39.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478027+0000", 2024-09-14T05:30:39.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060534959999999999 2024-09-14T05:30:39.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478079+0000", 2024-09-14T05:30:39.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2104999999999997e-05 2024-09-14T05:30:39.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478358+0000", 2024-09-14T05:30:39.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000279351 2024-09-14T05:30:39.905 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.905 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.905 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.905 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4053 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13041664~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.905 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:39.905 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.428828694999993, 2024-09-14T05:30:39.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061258866000000002, 2024-09-14T05:30:39.905 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.906 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.906 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.906 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.906 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.906 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.906 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4053 2024-09-14T05:30:39.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.906 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:39.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:39.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:39.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.17e-07 2024-09-14T05:30:39.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417496+0000", 2024-09-14T05:30:39.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0890000000000001e-06 2024-09-14T05:30:39.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417499+0000", 2024-09-14T05:30:39.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8629999999999999e-06 2024-09-14T05:30:39.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417502+0000", 2024-09-14T05:30:39.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6290000000000002e-06 2024-09-14T05:30:39.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478387+0000", 2024-09-14T05:30:39.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060885433000000003 2024-09-14T05:30:39.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478435+0000", 2024-09-14T05:30:39.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8167999999999997e-05 2024-09-14T05:30:39.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478754+0000", 2024-09-14T05:30:39.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031846700000000001 2024-09-14T05:30:39.909 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.910 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.910 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.910 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4054 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13533184~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.910 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:39.910 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.428818810999999, 2024-09-14T05:30:39.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061604694000000002, 2024-09-14T05:30:39.910 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.910 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.910 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.910 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.910 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.910 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.910 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4054 2024-09-14T05:30:39.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.911 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:39.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:39.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:39.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.04e-07 2024-09-14T05:30:39.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417506+0000", 2024-09-14T05:30:39.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8299999999999995e-07 2024-09-14T05:30:39.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417509+0000", 2024-09-14T05:30:39.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.88e-06 2024-09-14T05:30:39.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417511+0000", 2024-09-14T05:30:39.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5950000000000001e-06 2024-09-14T05:30:39.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478784+0000", 2024-09-14T05:30:39.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061272971000000002 2024-09-14T05:30:39.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478841+0000", 2024-09-14T05:30:39.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-14T05:30:39.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479109+0000", 2024-09-14T05:30:39.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026794600000000003 2024-09-14T05:30:39.914 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.914 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.914 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.914 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4055 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14024704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.914 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:39.914 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.428809108999999, 2024-09-14T05:30:39.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061932299000000003, 2024-09-14T05:30:39.914 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.914 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.914 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.915 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.915 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.915 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4055 2024-09-14T05:30:39.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.915 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:39.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:39.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417515+0000", 2024-09-14T05:30:39.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-14T05:30:39.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417516+0000", 2024-09-14T05:30:39.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7199999999999997e-07 2024-09-14T05:30:39.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417519+0000", 2024-09-14T05:30:39.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.937e-06 2024-09-14T05:30:39.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417522+0000", 2024-09-14T05:30:39.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1420000000000002e-06 2024-09-14T05:30:39.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479140+0000", 2024-09-14T05:30:39.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061617933999999999 2024-09-14T05:30:39.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479188+0000", 2024-09-14T05:30:39.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8717000000000001e-05 2024-09-14T05:30:39.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479447+0000", 2024-09-14T05:30:39.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025838800000000002 2024-09-14T05:30:39.918 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.918 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.918 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.918 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4056 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14516224~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.918 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:39.918 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.428798936000007, 2024-09-14T05:30:39.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062263185999999998, 2024-09-14T05:30:39.918 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.919 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.919 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.919 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.919 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.919 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.919 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4056 2024-09-14T05:30:39.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.919 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:39.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:39.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:39.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8700000000000002e-07 2024-09-14T05:30:39.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417528+0000", 2024-09-14T05:30:39.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.579e-06 2024-09-14T05:30:39.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417533+0000", 2024-09-14T05:30:39.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2400000000000001e-06 2024-09-14T05:30:39.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417536+0000", 2024-09-14T05:30:39.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0309999999999999e-06 2024-09-14T05:30:39.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479477+0000", 2024-09-14T05:30:39.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061940967 2024-09-14T05:30:39.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479523+0000", 2024-09-14T05:30:39.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6335000000000003e-05 2024-09-14T05:30:39.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479788+0000", 2024-09-14T05:30:39.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026474700000000002 2024-09-14T05:30:39.922 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.922 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.922 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.922 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4057 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15007744~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.428784475, 2024-09-14T05:30:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062589928000000003, 2024-09-14T05:30:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4057 2024-09-14T05:30:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422838+0000", 2024-09-14T05:30:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052990349999999997 2024-09-14T05:30:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422854+0000", 2024-09-14T05:30:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5523999999999999e-05 2024-09-14T05:30:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422864+0000", 2024-09-14T05:30:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0084e-05 2024-09-14T05:30:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422872+0000", 2024-09-14T05:30:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-14T05:30:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479813+0000", 2024-09-14T05:30:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056941389000000002 2024-09-14T05:30:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479868+0000", 2024-09-14T05:30:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4973999999999999e-05 2024-09-14T05:30:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.480129+0000", 2024-09-14T05:30:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026084200000000001 2024-09-14T05:30:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4171 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14057472~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.416786915000003, 2024-09-14T05:30:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061184929999999998, 2024-09-14T05:30:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4171 2024-09-14T05:30:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.39e-07 2024-09-14T05:30:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429541+0000", 2024-09-14T05:30:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:30:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429547+0000", 2024-09-14T05:30:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3280000000000002e-06 2024-09-14T05:30:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429552+0000", 2024-09-14T05:30:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4859999999999997e-06 2024-09-14T05:30:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490379+0000", 2024-09-14T05:30:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060826681 2024-09-14T05:30:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490434+0000", 2024-09-14T05:30:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4577999999999998e-05 2024-09-14T05:30:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490722+0000", 2024-09-14T05:30:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028781400000000001 2024-09-14T05:30:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4172 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14548992~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.416764907000001, 2024-09-14T05:30:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061498256000000001, 2024-09-14T05:30:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4172 2024-09-14T05:30:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.72e-07 2024-09-14T05:30:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429561+0000", 2024-09-14T05:30:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.717e-06 2024-09-14T05:30:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429566+0000", 2024-09-14T05:30:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6529999999999999e-06 2024-09-14T05:30:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429571+0000", 2024-09-14T05:30:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5979999999999999e-06 2024-09-14T05:30:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490751+0000", 2024-09-14T05:30:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061180336000000002 2024-09-14T05:30:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490798+0000", 2024-09-14T05:30:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6875000000000001e-05 2024-09-14T05:30:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491057+0000", 2024-09-14T05:30:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025870500000000002 2024-09-14T05:30:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4173 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15040512~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.416747280999999, 2024-09-14T05:30:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061894820000000003, 2024-09-14T05:30:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4173 2024-09-14T05:30:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429577+0000", 2024-09-14T05:30:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-14T05:30:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429578+0000", 2024-09-14T05:30:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6980000000000001e-06 2024-09-14T05:30:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429585+0000", 2024-09-14T05:30:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.725e-06 2024-09-14T05:30:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429590+0000", 2024-09-14T05:30:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.938e-06 2024-09-14T05:30:39.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:39.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491091+0000", 2024-09-14T05:30:39.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061500773000000002 2024-09-14T05:30:39.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:39.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491136+0000", 2024-09-14T05:30:39.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5284000000000002e-05 2024-09-14T05:30:39.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:39.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:39.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:39.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491471+0000", 2024-09-14T05:30:39.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033485699999999998 2024-09-14T05:30:39.940 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.940 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.940 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.940 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:39.940 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:39.940 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:39.941 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:30:39.942 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:39.941+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:40.019 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T05:30:40.026 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T05:30:40.041 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T05:30:40.042 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:40.041+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:40.096 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:40.096 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:40.096 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:40.096 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:40.105 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:30:40.147 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:40.146+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:40.243 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:40.243+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:40.249 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:40.250 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:40.250 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:40.250 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:40.250 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:40.258 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:30:40.344 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:40.344+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:40.403 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:40.403 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:40.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:40.403 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:40.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.403 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1205 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:40.404 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:40.404 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 70.203845013000006, 2024-09-14T05:30:40.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0086364279999999998, 2024-09-14T05:30:40.404 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.404 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.404 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:40.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.404 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1205 2024-09-14T05:30:40.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.404 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:40.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:40.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197891+0000", 2024-09-14T05:30:40.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.364e-06 2024-09-14T05:30:40.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197896+0000", 2024-09-14T05:30:40.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3000000000000001e-06 2024-09-14T05:30:40.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197910+0000", 2024-09-14T05:30:40.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3339000000000001e-05 2024-09-14T05:30:40.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197919+0000", 2024-09-14T05:30:40.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8869999999999998e-06 2024-09-14T05:30:40.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197974+0000", 2024-09-14T05:30:40.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5720000000000002e-05 2024-09-14T05:30:40.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.198094+0000", 2024-09-14T05:30:40.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119739 2024-09-14T05:30:40.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206526+0000", 2024-09-14T05:30:40.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084320790000000003 2024-09-14T05:30:40.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.408 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.408 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1220 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:40.408 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:40.408 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 70.195088648999999, 2024-09-14T05:30:40.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012822337, 2024-09-14T05:30:40.408 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.408 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.408 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.409 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1220 2024-09-14T05:30:40.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.409 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:40.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:40.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206647+0000", 2024-09-14T05:30:40.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4600000000000003e-07 2024-09-14T05:30:40.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206652+0000", 2024-09-14T05:30:40.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6619999999999996e-06 2024-09-14T05:30:40.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206662+0000", 2024-09-14T05:30:40.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.995e-06 2024-09-14T05:30:40.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206667+0000", 2024-09-14T05:30:40.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-14T05:30:40.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206706+0000", 2024-09-14T05:30:40.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9523000000000001e-05 2024-09-14T05:30:40.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206755+0000", 2024-09-14T05:30:40.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000002e-05 2024-09-14T05:30:40.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:40.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206877+0000", 2024-09-14T05:30:40.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121938 2024-09-14T05:30:40.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:40.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.207310+0000", 2024-09-14T05:30:40.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000432452 2024-09-14T05:30:40.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:40.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219364+0000", 2024-09-14T05:30:40.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012054481000000001 2024-09-14T05:30:40.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219425+0000", 2024-09-14T05:30:40.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0618000000000001e-05 2024-09-14T05:30:40.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219468+0000", 2024-09-14T05:30:40.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3739000000000003e-05 2024-09-14T05:30:40.414 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.414 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.414 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.414 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1243 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:40.414 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:40.414 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 70.182123422000004, 2024-09-14T05:30:40.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078660580000000004, 2024-09-14T05:30:40.415 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.415 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.415 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:40.415 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.415 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.415 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.415 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1243 2024-09-14T05:30:40.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.415 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:40.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:40.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219612+0000", 2024-09-14T05:30:40.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.09e-07 2024-09-14T05:30:40.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219616+0000", 2024-09-14T05:30:40.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.216e-06 2024-09-14T05:30:40.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219630+0000", 2024-09-14T05:30:40.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3064999999999999e-05 2024-09-14T05:30:40.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219637+0000", 2024-09-14T05:30:40.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7409999999999992e-06 2024-09-14T05:30:40.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219697+0000", 2024-09-14T05:30:40.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.02e-05 2024-09-14T05:30:40.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219833+0000", 2024-09-14T05:30:40.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013595600000000001 2024-09-14T05:30:40.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227477+0000", 2024-09-14T05:30:40.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076439710000000003 2024-09-14T05:30:40.418 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.418 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.418 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.419 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1264 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:40.419 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:40.419 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 70.174112629000007, 2024-09-14T05:30:40.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080867990000000004, 2024-09-14T05:30:40.419 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.419 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.419 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.419 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1264 2024-09-14T05:30:40.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.419 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:40.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:40.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227623+0000", 2024-09-14T05:30:40.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:30:40.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227627+0000", 2024-09-14T05:30:40.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.194e-06 2024-09-14T05:30:40.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227639+0000", 2024-09-14T05:30:40.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1296e-05 2024-09-14T05:30:40.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227648+0000", 2024-09-14T05:30:40.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2280000000000004e-06 2024-09-14T05:30:40.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227689+0000", 2024-09-14T05:30:40.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1329999999999999e-05 2024-09-14T05:30:40.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227741+0000", 2024-09-14T05:30:40.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1641000000000002e-05 2024-09-14T05:30:40.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:30:40.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227914+0000", 2024-09-14T05:30:40.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000173215 2024-09-14T05:30:40.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:40.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228474+0000", 2024-09-14T05:30:40.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055980199999999998 2024-09-14T05:30:40.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:40.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235608+0000", 2024-09-14T05:30:40.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0071343830000000002 2024-09-14T05:30:40.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235669+0000", 2024-09-14T05:30:40.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1068999999999994e-05 2024-09-14T05:30:40.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235709+0000", 2024-09-14T05:30:40.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9671e-05 2024-09-14T05:30:40.424 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.424 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.424 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.424 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1526 3.4 3:2695cbe4:::rbd_data.112dd25fb4c.0000000000000119:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:40.425 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:40.425 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 70.075572066000007, 2024-09-14T05:30:40.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0079537359999999994, 2024-09-14T05:30:40.425 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.425 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.425 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.425 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.425 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.425 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.425 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1526 2024-09-14T05:30:40.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.425 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:40.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:40.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:40.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3600000000000003e-07 2024-09-14T05:30:40.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326168+0000", 2024-09-14T05:30:40.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3460000000000004e-06 2024-09-14T05:30:40.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326178+0000", 2024-09-14T05:30:40.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0291e-05 2024-09-14T05:30:40.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326183+0000", 2024-09-14T05:30:40.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-14T05:30:40.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326230+0000", 2024-09-14T05:30:40.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6650999999999998e-05 2024-09-14T05:30:40.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326295+0000", 2024-09-14T05:30:40.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5158000000000002e-05 2024-09-14T05:30:40.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:40.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326408+0000", 2024-09-14T05:30:40.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113039 2024-09-14T05:30:40.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:40.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.331648+0000", 2024-09-14T05:30:40.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052404929999999997 2024-09-14T05:30:40.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:40.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334011+0000", 2024-09-14T05:30:40.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002362669 2024-09-14T05:30:40.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334064+0000", 2024-09-14T05:30:40.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3198999999999997e-05 2024-09-14T05:30:40.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334116+0000", 2024-09-14T05:30:40.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2133999999999999e-05 2024-09-14T05:30:40.430 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.430 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.430 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.430 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1529 3.1 3:809746f7:::rbd_data.112dd25fb4c.0000000000000118:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:40.430 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:30:40.431 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 70.073807516000002, 2024-09-14T05:30:40.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061981060000000001, 2024-09-14T05:30:40.431 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.431 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.431 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.431 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.431 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.431 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.431 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1529 2024-09-14T05:30:40.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.431 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:30:40.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327927+0000", 2024-09-14T05:30:40.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327928+0000", 2024-09-14T05:30:40.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0100000000000004e-07 2024-09-14T05:30:40.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327933+0000", 2024-09-14T05:30:40.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4689999999999999e-06 2024-09-14T05:30:40.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327944+0000", 2024-09-14T05:30:40.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1623999999999999e-05 2024-09-14T05:30:40.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.327951+0000", 2024-09-14T05:30:40.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9759999999999998e-06 2024-09-14T05:30:40.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328003+0000", 2024-09-14T05:30:40.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1521999999999999e-05 2024-09-14T05:30:40.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328087+0000", 2024-09-14T05:30:40.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3988999999999998e-05 2024-09-14T05:30:40.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:40.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328304+0000", 2024-09-14T05:30:40.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000217254 2024-09-14T05:30:40.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:40.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.333363+0000", 2024-09-14T05:30:40.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050586499999999996 2024-09-14T05:30:40.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:40.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334026+0000", 2024-09-14T05:30:40.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066303499999999997 2024-09-14T05:30:40.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334080+0000", 2024-09-14T05:30:40.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4261000000000001e-05 2024-09-14T05:30:40.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334125+0000", 2024-09-14T05:30:40.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5525000000000002e-05 2024-09-14T05:30:40.436 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.436 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.436 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.436 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1531 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:40.436 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:40.437 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 70.072753899999995, 2024-09-14T05:30:40.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082114309999999999, 2024-09-14T05:30:40.437 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.437 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.437 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.437 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1531 2024-09-14T05:30:40.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.437 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:40.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:40.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328982+0000", 2024-09-14T05:30:40.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:30:40.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328985+0000", 2024-09-14T05:30:40.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7900000000000001e-06 2024-09-14T05:30:40.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328996+0000", 2024-09-14T05:30:40.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.028e-05 2024-09-14T05:30:40.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329001+0000", 2024-09-14T05:30:40.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0610000000000003e-06 2024-09-14T05:30:40.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329035+0000", 2024-09-14T05:30:40.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4366999999999998e-05 2024-09-14T05:30:40.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329082+0000", 2024-09-14T05:30:40.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6646e-05 2024-09-14T05:30:40.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:40.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329195+0000", 2024-09-14T05:30:40.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113279 2024-09-14T05:30:40.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:40.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334293+0000", 2024-09-14T05:30:40.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050980399999999999 2024-09-14T05:30:40.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:40.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337101+0000", 2024-09-14T05:30:40.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0028076170000000001 2024-09-14T05:30:40.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337154+0000", 2024-09-14T05:30:40.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3641999999999999e-05 2024-09-14T05:30:40.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337192+0000", 2024-09-14T05:30:40.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7932000000000002e-05 2024-09-14T05:30:40.442 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.442 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.442 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.442 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:2131 3.b e30/24 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head v 30'36, mlcod=30'34)", 2024-09-14T05:30:40.443 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:40.443 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.609406544999999, 2024-09-14T05:30:40.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031871709999999998, 2024-09-14T05:30:40.443 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.443 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.443 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.443 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:40.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:40.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792331+0000", 2024-09-14T05:30:40.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6900000000000001e-06 2024-09-14T05:30:40.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797319+0000", 2024-09-14T05:30:40.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049876349999999998 2024-09-14T05:30:40.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797349+0000", 2024-09-14T05:30:40.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0046e-05 2024-09-14T05:30:40.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797359+0000", 2024-09-14T05:30:40.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0037999999999999e-05 2024-09-14T05:30:40.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797445+0000", 2024-09-14T05:30:40.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6132999999999997e-05 2024-09-14T05:30:40.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797472+0000", 2024-09-14T05:30:40.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7478999999999999e-05 2024-09-14T05:30:40.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824161+0000", 2024-09-14T05:30:40.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026688714999999998 2024-09-14T05:30:40.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824200+0000", 2024-09-14T05:30:40.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8974000000000003e-05 2024-09-14T05:30:40.447 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.447 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.447 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.447 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4259 3.4 3:25f21dc9:::rbd_data.112d3d611e54.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 294912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.447 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:40.447 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.010233931000002, 2024-09-14T05:30:40.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010043553, 2024-09-14T05:30:40.447 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.447 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.447 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.448 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4259 2024-09-14T05:30:40.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.448 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:40.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:40.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391503+0000", 2024-09-14T05:30:40.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9099999999999999e-06 2024-09-14T05:30:40.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391821+0000", 2024-09-14T05:30:40.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031871800000000001 2024-09-14T05:30:40.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391844+0000", 2024-09-14T05:30:40.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2045e-05 2024-09-14T05:30:40.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391853+0000", 2024-09-14T05:30:40.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7249999999999992e-06 2024-09-14T05:30:40.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391917+0000", 2024-09-14T05:30:40.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3458000000000001e-05 2024-09-14T05:30:40.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392037+0000", 2024-09-14T05:30:40.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120449 2024-09-14T05:30:40.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:40.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392305+0000", 2024-09-14T05:30:40.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026739299999999999 2024-09-14T05:30:40.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:40.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.393609+0000", 2024-09-14T05:30:40.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013042450000000001 2024-09-14T05:30:40.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:40.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401430+0000", 2024-09-14T05:30:40.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078211670000000004 2024-09-14T05:30:40.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401499+0000", 2024-09-14T05:30:40.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9374000000000004e-05 2024-09-14T05:30:40.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401544+0000", 2024-09-14T05:30:40.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5068999999999998e-05 2024-09-14T05:30:40.452 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.452 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.452 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.453 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4452 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.453 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:30:40.453 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.519421323000003, 2024-09-14T05:30:40.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066061380000000001, 2024-09-14T05:30:40.453 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.453 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.453 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.453 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.453 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.453 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.453 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4452 2024-09-14T05:30:40.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.453 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:30:40.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882313+0000", 2024-09-14T05:30:40.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882314+0000", 2024-09-14T05:30:40.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5899999999999998e-07 2024-09-14T05:30:40.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882315+0000", 2024-09-14T05:30:40.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6169999999999999e-06 2024-09-14T05:30:40.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882319+0000", 2024-09-14T05:30:40.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3629999999999998e-06 2024-09-14T05:30:40.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.882324+0000", 2024-09-14T05:30:40.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1370000000000001e-06 2024-09-14T05:30:40.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883369+0000", 2024-09-14T05:30:40.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010448320000000001 2024-09-14T05:30:40.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883421+0000", 2024-09-14T05:30:40.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2401000000000002e-05 2024-09-14T05:30:40.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:40.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.883638+0000", 2024-09-14T05:30:40.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021725600000000001 2024-09-14T05:30:40.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:40.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.884464+0000", 2024-09-14T05:30:40.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00082569800000000004 2024-09-14T05:30:40.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:40.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888860+0000", 2024-09-14T05:30:40.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0043963370000000002 2024-09-14T05:30:40.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888896+0000", 2024-09-14T05:30:40.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6022999999999997e-05 2024-09-14T05:30:40.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.888920+0000", 2024-09-14T05:30:40.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3215000000000001e-05 2024-09-14T05:30:40.458 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.458 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.458 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.458 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4467 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.458 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:40.458 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.501052873000006, 2024-09-14T05:30:40.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0065700189999999999, 2024-09-14T05:30:40.459 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.459 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.459 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.459 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.459 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.459 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.459 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4467 2024-09-14T05:30:40.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.459 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:40.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:40.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900682+0000", 2024-09-14T05:30:40.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-14T05:30:40.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900684+0000", 2024-09-14T05:30:40.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.629e-06 2024-09-14T05:30:40.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900688+0000", 2024-09-14T05:30:40.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8469999999999998e-06 2024-09-14T05:30:40.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.900691+0000", 2024-09-14T05:30:40.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-14T05:30:40.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.901775+0000", 2024-09-14T05:30:40.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001084177 2024-09-14T05:30:40.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.901862+0000", 2024-09-14T05:30:40.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7167000000000001e-05 2024-09-14T05:30:40.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:40.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.902223+0000", 2024-09-14T05:30:40.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036068800000000001 2024-09-14T05:30:40.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:40.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.902843+0000", 2024-09-14T05:30:40.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000620489 2024-09-14T05:30:40.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:40.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907167+0000", 2024-09-14T05:30:40.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0043242649999999999 2024-09-14T05:30:40.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907217+0000", 2024-09-14T05:30:40.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9648000000000003e-05 2024-09-14T05:30:40.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.907252+0000", 2024-09-14T05:30:40.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4780000000000002e-05 2024-09-14T05:30:40.464 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.464 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.464 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.464 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4554 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.465 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:40.465 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.417333119999995, 2024-09-14T05:30:40.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069998999999999999, 2024-09-14T05:30:40.465 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.465 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.465 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.465 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.465 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.465 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.465 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4554 2024-09-14T05:30:40.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.465 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:40.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:40.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:40.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9299999999999999e-07 2024-09-14T05:30:40.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984404+0000", 2024-09-14T05:30:40.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.855e-06 2024-09-14T05:30:40.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984407+0000", 2024-09-14T05:30:40.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-14T05:30:40.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984410+0000", 2024-09-14T05:30:40.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8159999999999998e-06 2024-09-14T05:30:40.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985511+0000", 2024-09-14T05:30:40.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00110053 2024-09-14T05:30:40.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985561+0000", 2024-09-14T05:30:40.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0389000000000002e-05 2024-09-14T05:30:40.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:40.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985772+0000", 2024-09-14T05:30:40.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000210571 2024-09-14T05:30:40.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:40.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.987075+0000", 2024-09-14T05:30:40.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001303128 2024-09-14T05:30:40.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:40.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991303+0000", 2024-09-14T05:30:40.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042283310000000001 2024-09-14T05:30:40.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991359+0000", 2024-09-14T05:30:40.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6335999999999997e-05 2024-09-14T05:30:40.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991402+0000", 2024-09-14T05:30:40.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2074999999999999e-05 2024-09-14T05:30:40.470 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.470 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.470 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.470 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4711 3.4 3:226df663:::rbd_data.112d3d611e54.000000000000002d:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:40.470 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:40.470 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.234661963999997, 2024-09-14T05:30:40.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011304193000000001, 2024-09-14T05:30:40.471 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.471 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.471 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.471 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.471 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.471 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.471 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4711 2024-09-14T05:30:40.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.471 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:40.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:40.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167074+0000", 2024-09-14T05:30:40.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0999999999999997e-07 2024-09-14T05:30:40.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167078+0000", 2024-09-14T05:30:40.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.878e-06 2024-09-14T05:30:40.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167087+0000", 2024-09-14T05:30:40.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6940000000000007e-06 2024-09-14T05:30:40.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167093+0000", 2024-09-14T05:30:40.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7740000000000001e-06 2024-09-14T05:30:40.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167128+0000", 2024-09-14T05:30:40.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5231999999999997e-05 2024-09-14T05:30:40.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167207+0000", 2024-09-14T05:30:40.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9036999999999996e-05 2024-09-14T05:30:40.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:40.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.169140+0000", 2024-09-14T05:30:40.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001932897 2024-09-14T05:30:40.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:40.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178258+0000", 2024-09-14T05:30:40.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091180819999999996 2024-09-14T05:30:40.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:40.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178314+0000", 2024-09-14T05:30:40.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5571e-05 2024-09-14T05:30:40.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178348+0000", 2024-09-14T05:30:40.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3747000000000001e-05 2024-09-14T05:30:40.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178377+0000", 2024-09-14T05:30:40.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9471e-05 2024-09-14T05:30:40.476 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.476 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.476 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.476 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4721 3.1 3:86302a6e:::rbd_data.112d3d611e54.0000000000000011:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:40.476 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:30:40.476 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.202676478000001, 2024-09-14T05:30:40.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066831750000000004, 2024-09-14T05:30:40.476 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.477 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.477 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.477 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4721 2024-09-14T05:30:40.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.477 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:30:40.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199058+0000", 2024-09-14T05:30:40.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199059+0000", 2024-09-14T05:30:40.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.06e-07 2024-09-14T05:30:40.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199062+0000", 2024-09-14T05:30:40.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2279999999999998e-06 2024-09-14T05:30:40.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199072+0000", 2024-09-14T05:30:40.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5999999999999996e-06 2024-09-14T05:30:40.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199078+0000", 2024-09-14T05:30:40.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9109999999999998e-06 2024-09-14T05:30:40.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199123+0000", 2024-09-14T05:30:40.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5278000000000003e-05 2024-09-14T05:30:40.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199202+0000", 2024-09-14T05:30:40.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8521000000000003e-05 2024-09-14T05:30:40.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:40.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199271+0000", 2024-09-14T05:30:40.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9857999999999994e-05 2024-09-14T05:30:40.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:40.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.199941+0000", 2024-09-14T05:30:40.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066984599999999998 2024-09-14T05:30:40.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:40.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205653+0000", 2024-09-14T05:30:40.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057117319999999997 2024-09-14T05:30:40.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205701+0000", 2024-09-14T05:30:40.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8254000000000003e-05 2024-09-14T05:30:40.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.205741+0000", 2024-09-14T05:30:40.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0241000000000003e-05 2024-09-14T05:30:40.482 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.482 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.482 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.482 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4819 3.12 3:4e8efced:::rbd_data.112d3d611e54.000000000000000e:head [stat,truncate 1114112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:40.482 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:30:40.482 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.117362318999994, 2024-09-14T05:30:40.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062422770000000001, 2024-09-14T05:30:40.482 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.482 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.483 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.483 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4819 2024-09-14T05:30:40.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.483 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:30:40.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284372+0000", 2024-09-14T05:30:40.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284373+0000", 2024-09-14T05:30:40.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3399999999999998e-07 2024-09-14T05:30:40.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284377+0000", 2024-09-14T05:30:40.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8539999999999999e-06 2024-09-14T05:30:40.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284388+0000", 2024-09-14T05:30:40.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1328999999999999e-05 2024-09-14T05:30:40.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284395+0000", 2024-09-14T05:30:40.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9759999999999998e-06 2024-09-14T05:30:40.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284723+0000", 2024-09-14T05:30:40.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032775799999999999 2024-09-14T05:30:40.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284801+0000", 2024-09-14T05:30:40.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7562e-05 2024-09-14T05:30:40.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:30:40.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.284933+0000", 2024-09-14T05:30:40.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000131989 2024-09-14T05:30:40.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:40.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.287450+0000", 2024-09-14T05:30:40.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025177699999999999 2024-09-14T05:30:40.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:40.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290531+0000", 2024-09-14T05:30:40.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0030809959999999999 2024-09-14T05:30:40.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290588+0000", 2024-09-14T05:30:40.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6177000000000001e-05 2024-09-14T05:30:40.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.290615+0000", 2024-09-14T05:30:40.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7132e-05 2024-09-14T05:30:40.488 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.488 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.488 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.488 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:7067 3.5 e35/24 3:a65d0cff:::rbd_data.112df7f2fe0a.0000000000000028:head v 35'89, mlcod=34'86)", 2024-09-14T05:30:40.488 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:40.489 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.258969047999997, 2024-09-14T05:30:40.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0064513080000000002, 2024-09-14T05:30:40.489 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.489 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.489 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.489 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:40.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:40.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142766+0000", 2024-09-14T05:30:40.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:40.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142771+0000", 2024-09-14T05:30:40.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8949999999999997e-06 2024-09-14T05:30:40.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142779+0000", 2024-09-14T05:30:40.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8699999999999992e-06 2024-09-14T05:30:40.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142785+0000", 2024-09-14T05:30:40.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4899999999999997e-06 2024-09-14T05:30:40.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142843+0000", 2024-09-14T05:30:40.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7377000000000003e-05 2024-09-14T05:30:40.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.142871+0000", 2024-09-14T05:30:40.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8305000000000001e-05 2024-09-14T05:30:40.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149177+0000", 2024-09-14T05:30:40.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063058639999999996 2024-09-14T05:30:40.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:35.149217+0000", 2024-09-14T05:30:40.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0080999999999998e-05 2024-09-14T05:30:40.492 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.493 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.493 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.493 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:72582 3.b e47/24 3:d5072567:::rbd_data.112d4a90444c.0000000000000119:head v 47'386, mlcod=46'383)", 2024-09-14T05:30:40.493 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:40.493 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.876708110999999, 2024-09-14T05:30:40.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0067720940000000002, 2024-09-14T05:30:40.493 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.493 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.493 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.493 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:40.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:40.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:40.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1100000000000002e-07 2024-09-14T05:30:40.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525031+0000", 2024-09-14T05:30:40.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0740000000000003e-06 2024-09-14T05:30:40.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525040+0000", 2024-09-14T05:30:40.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5329999999999994e-06 2024-09-14T05:30:40.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525045+0000", 2024-09-14T05:30:40.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:30:40.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525090+0000", 2024-09-14T05:30:40.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4743000000000002e-05 2024-09-14T05:30:40.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525102+0000", 2024-09-14T05:30:40.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2067e-05 2024-09-14T05:30:40.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531757+0000", 2024-09-14T05:30:40.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066550380000000003 2024-09-14T05:30:40.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531799+0000", 2024-09-14T05:30:40.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1640000000000001e-05 2024-09-14T05:30:40.497 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.497 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.497 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.497 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:79671 3.15 e47/24 3:ab572765:::rbd_data.112d956b542f.0000000000000013:head v 47'120, mlcod=47'118)", 2024-09-14T05:30:40.498 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:30:40.498 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.250727892999997, 2024-09-14T05:30:40.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066700609999999997, 2024-09-14T05:30:40.498 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.498 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.498 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.498 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:30:40.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151007+0000", 2024-09-14T05:30:40.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151008+0000", 2024-09-14T05:30:40.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:30:40.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151264+0000", 2024-09-14T05:30:40.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025656099999999999 2024-09-14T05:30:40.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151278+0000", 2024-09-14T05:30:40.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3288999999999999e-05 2024-09-14T05:30:40.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151287+0000", 2024-09-14T05:30:40.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7909999999999993e-06 2024-09-14T05:30:40.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151361+0000", 2024-09-14T05:30:40.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3979000000000004e-05 2024-09-14T05:30:40.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.151383+0000", 2024-09-14T05:30:40.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2039999999999999e-05 2024-09-14T05:30:40.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.157542+0000", 2024-09-14T05:30:40.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006158915 2024-09-14T05:30:40.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:50.157677+0000", 2024-09-14T05:30:40.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013544300000000001 2024-09-14T05:30:40.502 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.502 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.502 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.502 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:123378 3.4 e71/39 3:24969376:::rbd_data.112d471e22e6.0000000000000009:head v 71'440, mlcod=69'438)", 2024-09-14T05:30:40.502 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:17.791627+0000", 2024-09-14T05:30:40.502 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.610108181000001, 2024-09-14T05:30:40.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066686189999999998, 2024-09-14T05:30:40.502 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.502 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.502 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.503 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.791627+0000", 2024-09-14T05:30:40.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.791627+0000", 2024-09-14T05:30:40.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.791627+0000", 2024-09-14T05:30:40.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.65e-07 2024-09-14T05:30:40.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792806+0000", 2024-09-14T05:30:40.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001179193 2024-09-14T05:30:40.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792832+0000", 2024-09-14T05:30:40.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5514999999999999e-05 2024-09-14T05:30:40.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792842+0000", 2024-09-14T05:30:40.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0481e-05 2024-09-14T05:30:40.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792919+0000", 2024-09-14T05:30:40.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6720999999999999e-05 2024-09-14T05:30:40.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.792945+0000", 2024-09-14T05:30:40.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6367999999999999e-05 2024-09-14T05:30:40.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.798239+0000", 2024-09-14T05:30:40.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052942509999999998 2024-09-14T05:30:40.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:17.798295+0000", 2024-09-14T05:30:40.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5825000000000002e-05 2024-09-14T05:30:40.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.506 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.506 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.2.0:1261 3.1 e77/39 3:861b8c15:::rbd_data.112d7803edfe.0000000000000006:24 v 77'629, mlcod=75'626)", 2024-09-14T05:30:40.506 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:26.354463+0000", 2024-09-14T05:30:40.507 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.0472719, 2024-09-14T05:30:40.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0065166069999999998, 2024-09-14T05:30:40.507 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.507 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.507 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.507 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:26.354463+0000", 2024-09-14T05:30:40.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:26.354463+0000", 2024-09-14T05:30:40.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:26.354463+0000", 2024-09-14T05:30:40.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7300000000000002e-07 2024-09-14T05:30:40.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:26.354468+0000", 2024-09-14T05:30:40.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9799999999999998e-06 2024-09-14T05:30:40.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:26.354476+0000", 2024-09-14T05:30:40.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6029999999999999e-06 2024-09-14T05:30:40.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:26.354482+0000", 2024-09-14T05:30:40.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0360000000000003e-06 2024-09-14T05:30:40.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:26.354555+0000", 2024-09-14T05:30:40.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2675999999999993e-05 2024-09-14T05:30:40.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:26.354577+0000", 2024-09-14T05:30:40.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2118999999999999e-05 2024-09-14T05:30:40.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:26.360922+0000", 2024-09-14T05:30:40.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063450019999999998 2024-09-14T05:30:40.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:26.360979+0000", 2024-09-14T05:30:40.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7818000000000001e-05 2024-09-14T05:30:40.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.511 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.511 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.511 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:40.514 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:30:40.514 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:40.514+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:40.615 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:40.614+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:40.617 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:40.617 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:40.617 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:40.617 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:40.626 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:30:40.715 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:40.715+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:40.769 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:40.769 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:40.770 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:40.770 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:40.770 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:40.778 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:30:40.819 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:40.818+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:40.917 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:40.916+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:40.922 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2131 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 70.146018643000005, 2024-09-14T05:30:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049526593000000001, 2024-09-14T05:30:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2131 2024-09-14T05:30:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774867+0000", 2024-09-14T05:30:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4430000000000002e-06 2024-09-14T05:30:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790357+0000", 2024-09-14T05:30:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015489466 2024-09-14T05:30:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790383+0000", 2024-09-14T05:30:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6247000000000001e-05 2024-09-14T05:30:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790394+0000", 2024-09-14T05:30:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0728e-05 2024-09-14T05:30:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790471+0000", 2024-09-14T05:30:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7539000000000004e-05 2024-09-14T05:30:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790639+0000", 2024-09-14T05:30:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016787700000000001 2024-09-14T05:30:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-14T05:30:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790998+0000", 2024-09-14T05:30:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035931299999999999 2024-09-14T05:30:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.823529+0000", 2024-09-14T05:30:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032530798 2024-09-14T05:30:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824324+0000", 2024-09-14T05:30:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00079442399999999995 2024-09-14T05:30:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824362+0000", 2024-09-14T05:30:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8398000000000001e-05 2024-09-14T05:30:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824391+0000", 2024-09-14T05:30:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.936e-05 2024-09-14T05:30:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3892 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7602176~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.564440645000005, 2024-09-14T05:30:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047082951999999997, 2024-09-14T05:30:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3892 2024-09-14T05:30:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0199999999999998e-07 2024-09-14T05:30:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356454+0000", 2024-09-14T05:30:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0414999999999999e-05 2024-09-14T05:30:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356463+0000", 2024-09-14T05:30:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6739999999999992e-06 2024-09-14T05:30:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356468+0000", 2024-09-14T05:30:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5680000000000001e-06 2024-09-14T05:30:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403143+0000", 2024-09-14T05:30:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046675051000000002 2024-09-14T05:30:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403206+0000", 2024-09-14T05:30:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3127999999999996e-05 2024-09-14T05:30:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403526+0000", 2024-09-14T05:30:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031981400000000002 2024-09-14T05:30:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3893 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8093696~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.564395808, 2024-09-14T05:30:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047516305000000002, 2024-09-14T05:30:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3893 2024-09-14T05:30:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6899999999999997e-07 2024-09-14T05:30:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356491+0000", 2024-09-14T05:30:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8109999999999999e-06 2024-09-14T05:30:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356498+0000", 2024-09-14T05:30:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5320000000000003e-06 2024-09-14T05:30:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356502+0000", 2024-09-14T05:30:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000002e-06 2024-09-14T05:30:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403579+0000", 2024-09-14T05:30:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047076706000000003 2024-09-14T05:30:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403664+0000", 2024-09-14T05:30:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5240000000000004e-05 2024-09-14T05:30:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404004+0000", 2024-09-14T05:30:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033974500000000002 2024-09-14T05:30:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3993 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 262144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.564373936999999, 2024-09-14T05:30:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048088476999999998, 2024-09-14T05:30:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3993 2024-09-14T05:30:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.46e-07 2024-09-14T05:30:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356512+0000", 2024-09-14T05:30:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.827e-06 2024-09-14T05:30:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356518+0000", 2024-09-14T05:30:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7780000000000004e-06 2024-09-14T05:30:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356522+0000", 2024-09-14T05:30:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.865e-06 2024-09-14T05:30:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404036+0000", 2024-09-14T05:30:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047513725999999999 2024-09-14T05:30:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404135+0000", 2024-09-14T05:30:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9352999999999996e-05 2024-09-14T05:30:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404598+0000", 2024-09-14T05:30:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046258200000000001 2024-09-14T05:30:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3994 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.564331925000005, 2024-09-14T05:30:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048274625000000002, 2024-09-14T05:30:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3994 2024-09-14T05:30:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3799999999999997e-07 2024-09-14T05:30:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356554+0000", 2024-09-14T05:30:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2259999999999999e-06 2024-09-14T05:30:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356560+0000", 2024-09-14T05:30:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-14T05:30:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356566+0000", 2024-09-14T05:30:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3909999999999997e-06 2024-09-14T05:30:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404632+0000", 2024-09-14T05:30:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048066154 2024-09-14T05:30:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404719+0000", 2024-09-14T05:30:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7440000000000003e-05 2024-09-14T05:30:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404826+0000", 2024-09-14T05:30:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106813 2024-09-14T05:30:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3995 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.564306786000003, 2024-09-14T05:30:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048828209999999997, 2024-09-14T05:30:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3995 2024-09-14T05:30:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6599999999999996e-07 2024-09-14T05:30:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356579+0000", 2024-09-14T05:30:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0719999999999998e-06 2024-09-14T05:30:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356585+0000", 2024-09-14T05:30:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.959e-06 2024-09-14T05:30:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356590+0000", 2024-09-14T05:30:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.464e-06 2024-09-14T05:30:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404875+0000", 2024-09-14T05:30:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048285649999999999 2024-09-14T05:30:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404985+0000", 2024-09-14T05:30:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109417 2024-09-14T05:30:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405405+0000", 2024-09-14T05:30:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042008200000000001 2024-09-14T05:30:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3996 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.563997903000001, 2024-09-14T05:30:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049061595, 2024-09-14T05:30:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3996 2024-09-14T05:30:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.61e-07 2024-09-14T05:30:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356892+0000", 2024-09-14T05:30:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9050000000000002e-06 2024-09-14T05:30:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356901+0000", 2024-09-14T05:30:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4149999999999999e-06 2024-09-14T05:30:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356907+0000", 2024-09-14T05:30:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0140000000000004e-06 2024-09-14T05:30:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405448+0000", 2024-09-14T05:30:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048541647 2024-09-14T05:30:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405542+0000", 2024-09-14T05:30:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3639000000000003e-05 2024-09-14T05:30:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405947+0000", 2024-09-14T05:30:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040521399999999999 2024-09-14T05:30:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3997 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2228224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.563783920999995, 2024-09-14T05:30:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049308774999999999, 2024-09-14T05:30:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3997 2024-09-14T05:30:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6499999999999999e-07 2024-09-14T05:30:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357104+0000", 2024-09-14T05:30:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4989999999999999e-06 2024-09-14T05:30:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357112+0000", 2024-09-14T05:30:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4279999999999996e-06 2024-09-14T05:30:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357117+0000", 2024-09-14T05:30:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6190000000000002e-06 2024-09-14T05:30:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405989+0000", 2024-09-14T05:30:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048872080999999998 2024-09-14T05:30:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406074+0000", 2024-09-14T05:30:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4676e-05 2024-09-14T05:30:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406408+0000", 2024-09-14T05:30:40.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033480700000000001 2024-09-14T05:30:40.961 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.961 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.961 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.961 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3998 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.961 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:40.961 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.563462491999999, 2024-09-14T05:30:40.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049462314, 2024-09-14T05:30:40.961 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.961 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.962 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:40.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.962 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3998 2024-09-14T05:30:40.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.962 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:40.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:40.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357422+0000", 2024-09-14T05:30:40.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3099999999999996e-07 2024-09-14T05:30:40.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357429+0000", 2024-09-14T05:30:40.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2880000000000004e-06 2024-09-14T05:30:40.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357439+0000", 2024-09-14T05:30:40.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.302e-06 2024-09-14T05:30:40.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357445+0000", 2024-09-14T05:30:40.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4269999999999996e-06 2024-09-14T05:30:40.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406448+0000", 2024-09-14T05:30:40.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049003096000000003 2024-09-14T05:30:40.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406537+0000", 2024-09-14T05:30:40.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9135999999999995e-05 2024-09-14T05:30:40.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406883+0000", 2024-09-14T05:30:40.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000346134 2024-09-14T05:30:40.965 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.965 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.965 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.966 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3999 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.966 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:40.966 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.563392135000001, 2024-09-14T05:30:40.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049797556, 2024-09-14T05:30:40.966 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.966 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.966 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:40.966 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.966 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.966 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.966 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3999 2024-09-14T05:30:40.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.967 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:40.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:40.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357492+0000", 2024-09-14T05:30:40.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8299999999999996e-07 2024-09-14T05:30:40.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357495+0000", 2024-09-14T05:30:40.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0450000000000001e-06 2024-09-14T05:30:40.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357503+0000", 2024-09-14T05:30:40.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9689999999999994e-06 2024-09-14T05:30:40.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357510+0000", 2024-09-14T05:30:40.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-14T05:30:40.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406914+0000", 2024-09-14T05:30:40.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049404293000000002 2024-09-14T05:30:40.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406965+0000", 2024-09-14T05:30:40.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0470999999999998e-05 2024-09-14T05:30:40.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407289+0000", 2024-09-14T05:30:40.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032434899999999998 2024-09-14T05:30:40.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.970 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.970 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4000 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3702784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.970 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:40.970 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.563045790000004, 2024-09-14T05:30:40.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049936728999999999, 2024-09-14T05:30:40.970 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.971 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.971 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:40.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.971 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4000 2024-09-14T05:30:40.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.971 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:40.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:40.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357839+0000", 2024-09-14T05:30:40.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-14T05:30:40.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357842+0000", 2024-09-14T05:30:40.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4850000000000001e-06 2024-09-14T05:30:40.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357851+0000", 2024-09-14T05:30:40.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1819999999999995e-06 2024-09-14T05:30:40.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357859+0000", 2024-09-14T05:30:40.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8129999999999996e-06 2024-09-14T05:30:40.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407337+0000", 2024-09-14T05:30:40.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049478084999999998 2024-09-14T05:30:40.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407442+0000", 2024-09-14T05:30:40.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000105302 2024-09-14T05:30:40.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407774+0000", 2024-09-14T05:30:40.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033200100000000001 2024-09-14T05:30:40.974 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.974 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.974 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.975 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4001 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.975 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:40.975 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.562950435000005, 2024-09-14T05:30:40.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050304205999999997, 2024-09-14T05:30:40.975 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.975 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.975 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:40.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.975 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4001 2024-09-14T05:30:40.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.976 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:40.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:40.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357934+0000", 2024-09-14T05:30:40.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000004e-07 2024-09-14T05:30:40.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357937+0000", 2024-09-14T05:30:40.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0869999999999998e-06 2024-09-14T05:30:40.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357946+0000", 2024-09-14T05:30:40.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1439999999999992e-06 2024-09-14T05:30:40.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357952+0000", 2024-09-14T05:30:40.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.609e-06 2024-09-14T05:30:40.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407820+0000", 2024-09-14T05:30:40.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049868580000000003 2024-09-14T05:30:40.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407909+0000", 2024-09-14T05:30:40.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8887000000000005e-05 2024-09-14T05:30:40.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408237+0000", 2024-09-14T05:30:40.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032824599999999997 2024-09-14T05:30:40.979 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.979 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.979 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.979 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4002 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.979 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:40.979 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.562733840000007, 2024-09-14T05:30:40.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050539529999999999, 2024-09-14T05:30:40.979 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.979 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.980 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:40.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.980 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4002 2024-09-14T05:30:40.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.980 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:40.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:40.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:40.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5099999999999999e-07 2024-09-14T05:30:40.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358154+0000", 2024-09-14T05:30:40.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6799999999999999e-06 2024-09-14T05:30:40.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358165+0000", 2024-09-14T05:30:40.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0899e-05 2024-09-14T05:30:40.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358172+0000", 2024-09-14T05:30:40.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8689999999999998e-06 2024-09-14T05:30:40.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408281+0000", 2024-09-14T05:30:40.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050108699 2024-09-14T05:30:40.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408369+0000", 2024-09-14T05:30:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8542999999999996e-05 2024-09-14T05:30:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408689+0000", 2024-09-14T05:30:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032008900000000002 2024-09-14T05:30:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.983 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4003 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5177344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.984 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:40.984 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.562694004999997, 2024-09-14T05:30:40.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050868759, 2024-09-14T05:30:40.984 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.984 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.984 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:40.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.984 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4003 2024-09-14T05:30:40.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.984 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:40.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:40.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:40.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-07 2024-09-14T05:30:40.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358193+0000", 2024-09-14T05:30:40.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.588e-06 2024-09-14T05:30:40.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358204+0000", 2024-09-14T05:30:40.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1525999999999999e-05 2024-09-14T05:30:40.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358213+0000", 2024-09-14T05:30:40.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5010000000000004e-06 2024-09-14T05:30:40.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408719+0000", 2024-09-14T05:30:40.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050506708999999997 2024-09-14T05:30:40.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408766+0000", 2024-09-14T05:30:40.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6579999999999998e-05 2024-09-14T05:30:40.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409058+0000", 2024-09-14T05:30:40.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000292365 2024-09-14T05:30:40.988 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.988 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.988 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.988 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4004 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5668864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.988 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:40.988 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.562271162000002, 2024-09-14T05:30:40.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050839848, 2024-09-14T05:30:40.988 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.989 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.989 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:40.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.989 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4004 2024-09-14T05:30:40.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.989 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:40.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:40.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358613+0000", 2024-09-14T05:30:40.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:30:40.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358618+0000", 2024-09-14T05:30:40.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6009999999999997e-06 2024-09-14T05:30:40.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358630+0000", 2024-09-14T05:30:40.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2122e-05 2024-09-14T05:30:40.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358638+0000", 2024-09-14T05:30:40.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3969999999999994e-06 2024-09-14T05:30:40.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409088+0000", 2024-09-14T05:30:40.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050449565000000002 2024-09-14T05:30:40.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409153+0000", 2024-09-14T05:30:40.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4728999999999996e-05 2024-09-14T05:30:40.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409452+0000", 2024-09-14T05:30:40.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029958900000000001 2024-09-14T05:30:40.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.992 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4005 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6160384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.562054227999994, 2024-09-14T05:30:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051054752000000002, 2024-09-14T05:30:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4005 2024-09-14T05:30:40.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.994 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:40.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:40.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358830+0000", 2024-09-14T05:30:40.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-14T05:30:40.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358833+0000", 2024-09-14T05:30:40.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9529999999999999e-06 2024-09-14T05:30:40.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:40.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358843+0000", 2024-09-14T05:30:40.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8069999999999994e-06 2024-09-14T05:30:40.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:40.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358849+0000", 2024-09-14T05:30:40.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1040000000000003e-06 2024-09-14T05:30:40.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:40.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409487+0000", 2024-09-14T05:30:40.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050638626999999999 2024-09-14T05:30:40.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:40.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409535+0000", 2024-09-14T05:30:40.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7525000000000003e-05 2024-09-14T05:30:40.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:40.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409884+0000", 2024-09-14T05:30:40.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034914800000000003 2024-09-14T05:30:40.997 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.997 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:40.997 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:40.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.997 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4006 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:40.997 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:40.997 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.562017116000007, 2024-09-14T05:30:40.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051425323000000002, 2024-09-14T05:30:40.997 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:40.998 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:40.998 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:40.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:40.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:40.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:40.998 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4006 2024-09-14T05:30:40.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.998 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:40.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:40.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:40.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:40.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:40.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:40.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:40.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358867+0000", 2024-09-14T05:30:40.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2900000000000004e-07 2024-09-14T05:30:40.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:40.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:40.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:40.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358869+0000", 2024-09-14T05:30:40.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4210000000000002e-06 2024-09-14T05:30:41.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:41.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358876+0000", 2024-09-14T05:30:41.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4980000000000003e-06 2024-09-14T05:30:41.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:41.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358881+0000", 2024-09-14T05:30:41.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3090000000000003e-06 2024-09-14T05:30:41.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:41.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409915+0000", 2024-09-14T05:30:41.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051033348999999999 2024-09-14T05:30:41.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:41.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409964+0000", 2024-09-14T05:30:41.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9858000000000003e-05 2024-09-14T05:30:41.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:41.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410292+0000", 2024-09-14T05:30:41.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032735899999999999 2024-09-14T05:30:41.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:41.001 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:41.002 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:41.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.002 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4007 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:41.002 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:41.002 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.561994522999996, 2024-09-14T05:30:41.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05184076, 2024-09-14T05:30:41.002 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:41.002 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:41.002 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:41.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:41.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:41.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:41.003 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4007 2024-09-14T05:30:41.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.003 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:41.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:41.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:41.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:41.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:41.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:41.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:41.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:41.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:41.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:30:41.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:41.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358891+0000", 2024-09-14T05:30:41.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9819999999999998e-06 2024-09-14T05:30:41.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:41.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358897+0000", 2024-09-14T05:30:41.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9390000000000002e-06 2024-09-14T05:30:41.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:41.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358902+0000", 2024-09-14T05:30:41.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6770000000000003e-06 2024-09-14T05:30:41.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:41.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410335+0000", 2024-09-14T05:30:41.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051432991999999997 2024-09-14T05:30:41.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:41.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410416+0000", 2024-09-14T05:30:41.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1316999999999994e-05 2024-09-14T05:30:41.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:41.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410730+0000", 2024-09-14T05:30:41.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000313419 2024-09-14T05:30:41.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:41.006 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:41.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:41.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.006 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4008 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7634944~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:41.006 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:41.006 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.561970161999994, 2024-09-14T05:30:41.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052208082000000003, 2024-09-14T05:30:41.007 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:41.007 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:41.007 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:41.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:41.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:41.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:41.007 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4008 2024-09-14T05:30:41.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.007 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:41.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:41.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:41.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:41.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:41.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:41.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:41.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:41.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358914+0000", 2024-09-14T05:30:41.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6100000000000001e-07 2024-09-14T05:30:41.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:41.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358916+0000", 2024-09-14T05:30:41.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.902e-06 2024-09-14T05:30:41.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:41.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358923+0000", 2024-09-14T05:30:41.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4780000000000002e-06 2024-09-14T05:30:41.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:41.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358929+0000", 2024-09-14T05:30:41.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0970000000000002e-06 2024-09-14T05:30:41.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:41.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410761+0000", 2024-09-14T05:30:41.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051831436000000002 2024-09-14T05:30:41.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:41.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410829+0000", 2024-09-14T05:30:41.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7874999999999996e-05 2024-09-14T05:30:41.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:41.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411121+0000", 2024-09-14T05:30:41.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029283299999999999 2024-09-14T05:30:41.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:41.011 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:41.011 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:41.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.011 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4009 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8126464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:41.011 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:41.011 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.561822937000002, 2024-09-14T05:30:41.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052518169000000003, 2024-09-14T05:30:41.011 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:41.011 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:41.011 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:41.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:41.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:41.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:41.012 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4009 2024-09-14T05:30:41.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.012 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:41.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:41.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:41.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:41.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:41.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:41.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:41.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:41.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:41.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0200000000000002e-07 2024-09-14T05:30:41.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:41.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359066+0000", 2024-09-14T05:30:41.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7899999999999999e-06 2024-09-14T05:30:41.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:41.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359074+0000", 2024-09-14T05:30:41.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2579999999999995e-06 2024-09-14T05:30:41.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:41.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359082+0000", 2024-09-14T05:30:41.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.464e-06 2024-09-14T05:30:41.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:41.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411160+0000", 2024-09-14T05:30:41.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052078078 2024-09-14T05:30:41.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:41.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411244+0000", 2024-09-14T05:30:41.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4041000000000004e-05 2024-09-14T05:30:41.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:41.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:41.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:41.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411579+0000", 2024-09-14T05:30:41.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033493599999999998 2024-09-14T05:30:41.015 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:41.015 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:41.016 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:41.016 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:41.016 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:41.016 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:41.017 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:30:41.018 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:41.018+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:41.033 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-14T05:30:41.033 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T05:30:41.056 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T05:30:41.119 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:41.118+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:41.146 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:41.146 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:41.156 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:30:41.220 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:41.220+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:41.264 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:30:41.277 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:41.277 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:41.277 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:41.277 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:41.278 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:41.287 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:30:41.328 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:41.327+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:41.386 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:41.387 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:41.403 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:41.404 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:41.421 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:41.421 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:41.422 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:41.421+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:41.434 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:41.434 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:41.438 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:41.438 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:41.443 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:30:41.455 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:41.455 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:41.473 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:41.473 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:41.489 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:41.490 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:41.506 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:41.506 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:41.549 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:41.549 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:41.549 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:41.548+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:41.557 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:30:41.641 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:41.641+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:41.701 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:41.701 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:41.709 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:30:41.751 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:41.750+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:41.825 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:41.824+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:41.852 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:41.852 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:41.861 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:30:41.925 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:41.924+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:41.964 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:41.965 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:41.973 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:30:42.040 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T05:30:42.041 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:42.040+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:42.057 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T05:30:42.128 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:42.128 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:42.129 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:42.128+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:42.138 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:30:42.229 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:42.229+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:42.285 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:42.285 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:42.294 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:30:42.337 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:42.337+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:42.398 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:42.398 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:42.407 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:30:42.431 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:42.430+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:42.531 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:42.530+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:42.551 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:42.551 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:42.560 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:30:42.632 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:42.631+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:42.703 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:42.703 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:42.712 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:30:42.756 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:42.755+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:42.827 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:42.827 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:42.834 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:42.833+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:42.836 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:30:42.934 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:42.934+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:42.979 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:42.980 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:42.988 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:30:43.035 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:43.035+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:43.051 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T05:30:43.057 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T05:30:43.067 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T05:30:43.095 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:43.096 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:43.105 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:30:43.151 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:43.151+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:43.238 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:43.237+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:43.252 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:43.252 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:43.260 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:30:43.338 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:43.338+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:43.404 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:43.404 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:43.413 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:30:43.456 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:43.456+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:43.516 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:43.516 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:43.518 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:30:43.525 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:30:43.567 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:43.566+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:43.638 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:43.641 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:43.640+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:43.655 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:43.655 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:43.670 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:43.670 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:43.671 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:43.671 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:43.679 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:30:43.687 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:43.687 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:43.703 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:43.704 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:43.742 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:43.741+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:43.821 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:43.821 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:43.830 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:30:43.874 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:43.874+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:43.921 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:43.921 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:43.933 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:43.933 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:43.937 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:43.938 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:43.943 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:30:43.944 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:43.942+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:43.954 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:43.954 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:43.999 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-14T05:30:44.008 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T05:30:44.025 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T05:30:44.046 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:44.045+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:44.099 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:44.099 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:44.099 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:44.099 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:44.108 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:30:44.147 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:44.145+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:44.248 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:44.247+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:44.253 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:44.253 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:44.253 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:44.253 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:44.253 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:44.262 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:30:44.348 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:44.347+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:44.407 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:44.407 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:44.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:44.408 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:44.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.408 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2131 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:44.408 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:44.408 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.631230345000006, 2024-09-14T05:30:44.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049526593000000001, 2024-09-14T05:30:44.408 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.408 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.408 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:44.408 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.408 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.408 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.409 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2131 2024-09-14T05:30:44.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.409 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:44.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:44.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774867+0000", 2024-09-14T05:30:44.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4430000000000002e-06 2024-09-14T05:30:44.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790357+0000", 2024-09-14T05:30:44.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015489466 2024-09-14T05:30:44.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790383+0000", 2024-09-14T05:30:44.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6247000000000001e-05 2024-09-14T05:30:44.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790394+0000", 2024-09-14T05:30:44.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0728e-05 2024-09-14T05:30:44.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790471+0000", 2024-09-14T05:30:44.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7539000000000004e-05 2024-09-14T05:30:44.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790639+0000", 2024-09-14T05:30:44.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016787700000000001 2024-09-14T05:30:44.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-14T05:30:44.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790998+0000", 2024-09-14T05:30:44.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035931299999999999 2024-09-14T05:30:44.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:44.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.823529+0000", 2024-09-14T05:30:44.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032530798 2024-09-14T05:30:44.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:44.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824324+0000", 2024-09-14T05:30:44.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00079442399999999995 2024-09-14T05:30:44.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:44.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824362+0000", 2024-09-14T05:30:44.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8398000000000001e-05 2024-09-14T05:30:44.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824391+0000", 2024-09-14T05:30:44.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.936e-05 2024-09-14T05:30:44.414 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.414 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.414 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.414 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3892 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7602176~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.414 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:44.414 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.049652347000006, 2024-09-14T05:30:44.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047082951999999997, 2024-09-14T05:30:44.414 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.414 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.414 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.415 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.415 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.415 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.415 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3892 2024-09-14T05:30:44.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.415 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:44.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:44.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:44.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0199999999999998e-07 2024-09-14T05:30:44.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356454+0000", 2024-09-14T05:30:44.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0414999999999999e-05 2024-09-14T05:30:44.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356463+0000", 2024-09-14T05:30:44.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6739999999999992e-06 2024-09-14T05:30:44.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356468+0000", 2024-09-14T05:30:44.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5680000000000001e-06 2024-09-14T05:30:44.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403143+0000", 2024-09-14T05:30:44.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046675051000000002 2024-09-14T05:30:44.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403206+0000", 2024-09-14T05:30:44.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3127999999999996e-05 2024-09-14T05:30:44.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403526+0000", 2024-09-14T05:30:44.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031981400000000002 2024-09-14T05:30:44.418 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.418 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.418 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.419 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3893 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8093696~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.419 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:44.419 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.049607510000001, 2024-09-14T05:30:44.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047516305000000002, 2024-09-14T05:30:44.419 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.419 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.419 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.419 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3893 2024-09-14T05:30:44.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.419 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:44.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:44.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:44.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6899999999999997e-07 2024-09-14T05:30:44.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356491+0000", 2024-09-14T05:30:44.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8109999999999999e-06 2024-09-14T05:30:44.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356498+0000", 2024-09-14T05:30:44.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5320000000000003e-06 2024-09-14T05:30:44.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356502+0000", 2024-09-14T05:30:44.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000002e-06 2024-09-14T05:30:44.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403579+0000", 2024-09-14T05:30:44.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047076706000000003 2024-09-14T05:30:44.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403664+0000", 2024-09-14T05:30:44.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5240000000000004e-05 2024-09-14T05:30:44.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404004+0000", 2024-09-14T05:30:44.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033974500000000002 2024-09-14T05:30:44.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.423 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.423 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3993 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 262144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.424 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:44.424 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.049585639, 2024-09-14T05:30:44.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048088476999999998, 2024-09-14T05:30:44.424 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.424 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.424 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.424 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3993 2024-09-14T05:30:44.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.424 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:44.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:44.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:44.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.46e-07 2024-09-14T05:30:44.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356512+0000", 2024-09-14T05:30:44.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.827e-06 2024-09-14T05:30:44.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356518+0000", 2024-09-14T05:30:44.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7780000000000004e-06 2024-09-14T05:30:44.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356522+0000", 2024-09-14T05:30:44.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.865e-06 2024-09-14T05:30:44.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404036+0000", 2024-09-14T05:30:44.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047513725999999999 2024-09-14T05:30:44.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404135+0000", 2024-09-14T05:30:44.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9352999999999996e-05 2024-09-14T05:30:44.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404598+0000", 2024-09-14T05:30:44.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046258200000000001 2024-09-14T05:30:44.428 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.428 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.428 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.428 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3994 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.428 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:44.428 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.049543627000006, 2024-09-14T05:30:44.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048274625000000002, 2024-09-14T05:30:44.429 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.429 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.429 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.429 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3994 2024-09-14T05:30:44.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.429 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:44.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:44.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:44.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3799999999999997e-07 2024-09-14T05:30:44.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356554+0000", 2024-09-14T05:30:44.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2259999999999999e-06 2024-09-14T05:30:44.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356560+0000", 2024-09-14T05:30:44.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-14T05:30:44.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356566+0000", 2024-09-14T05:30:44.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3909999999999997e-06 2024-09-14T05:30:44.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404632+0000", 2024-09-14T05:30:44.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048066154 2024-09-14T05:30:44.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404719+0000", 2024-09-14T05:30:44.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7440000000000003e-05 2024-09-14T05:30:44.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404826+0000", 2024-09-14T05:30:44.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106813 2024-09-14T05:30:44.433 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.433 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.433 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.433 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3995 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.433 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:44.433 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.049518488000004, 2024-09-14T05:30:44.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048828209999999997, 2024-09-14T05:30:44.433 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.433 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.434 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.434 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3995 2024-09-14T05:30:44.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.434 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:44.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:44.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:44.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6599999999999996e-07 2024-09-14T05:30:44.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356579+0000", 2024-09-14T05:30:44.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0719999999999998e-06 2024-09-14T05:30:44.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356585+0000", 2024-09-14T05:30:44.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.959e-06 2024-09-14T05:30:44.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356590+0000", 2024-09-14T05:30:44.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.464e-06 2024-09-14T05:30:44.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404875+0000", 2024-09-14T05:30:44.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048285649999999999 2024-09-14T05:30:44.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404985+0000", 2024-09-14T05:30:44.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109417 2024-09-14T05:30:44.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405405+0000", 2024-09-14T05:30:44.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042008200000000001 2024-09-14T05:30:44.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.437 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.438 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3996 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.438 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:44.438 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.049209605000001, 2024-09-14T05:30:44.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049061595, 2024-09-14T05:30:44.438 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.438 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.438 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.438 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3996 2024-09-14T05:30:44.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.439 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:44.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:44.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:44.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.61e-07 2024-09-14T05:30:44.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356892+0000", 2024-09-14T05:30:44.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9050000000000002e-06 2024-09-14T05:30:44.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356901+0000", 2024-09-14T05:30:44.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4149999999999999e-06 2024-09-14T05:30:44.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356907+0000", 2024-09-14T05:30:44.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0140000000000004e-06 2024-09-14T05:30:44.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405448+0000", 2024-09-14T05:30:44.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048541647 2024-09-14T05:30:44.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405542+0000", 2024-09-14T05:30:44.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3639000000000003e-05 2024-09-14T05:30:44.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405947+0000", 2024-09-14T05:30:44.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040521399999999999 2024-09-14T05:30:44.443 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.443 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.443 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.443 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3997 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2228224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.443 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:44.443 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.048995622999996, 2024-09-14T05:30:44.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049308774999999999, 2024-09-14T05:30:44.443 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.443 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.443 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.443 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.444 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.444 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.444 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3997 2024-09-14T05:30:44.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.444 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:44.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:44.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:44.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6499999999999999e-07 2024-09-14T05:30:44.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357104+0000", 2024-09-14T05:30:44.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4989999999999999e-06 2024-09-14T05:30:44.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357112+0000", 2024-09-14T05:30:44.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4279999999999996e-06 2024-09-14T05:30:44.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357117+0000", 2024-09-14T05:30:44.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6190000000000002e-06 2024-09-14T05:30:44.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405989+0000", 2024-09-14T05:30:44.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048872080999999998 2024-09-14T05:30:44.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406074+0000", 2024-09-14T05:30:44.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4676e-05 2024-09-14T05:30:44.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406408+0000", 2024-09-14T05:30:44.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033480700000000001 2024-09-14T05:30:44.447 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.447 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.447 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.448 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3998 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.448 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:44.448 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.048674194, 2024-09-14T05:30:44.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049462314, 2024-09-14T05:30:44.448 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.448 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.448 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.448 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.448 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.448 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.448 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3998 2024-09-14T05:30:44.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.448 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:44.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:44.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357422+0000", 2024-09-14T05:30:44.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3099999999999996e-07 2024-09-14T05:30:44.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357429+0000", 2024-09-14T05:30:44.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2880000000000004e-06 2024-09-14T05:30:44.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357439+0000", 2024-09-14T05:30:44.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.302e-06 2024-09-14T05:30:44.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357445+0000", 2024-09-14T05:30:44.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4269999999999996e-06 2024-09-14T05:30:44.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406448+0000", 2024-09-14T05:30:44.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049003096000000003 2024-09-14T05:30:44.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406537+0000", 2024-09-14T05:30:44.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9135999999999995e-05 2024-09-14T05:30:44.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406883+0000", 2024-09-14T05:30:44.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000346134 2024-09-14T05:30:44.452 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.452 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.452 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.452 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3999 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.452 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:44.452 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.048603837000002, 2024-09-14T05:30:44.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049797556, 2024-09-14T05:30:44.452 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.453 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.453 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.453 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.453 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.453 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.453 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3999 2024-09-14T05:30:44.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.453 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:44.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:44.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357492+0000", 2024-09-14T05:30:44.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8299999999999996e-07 2024-09-14T05:30:44.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357495+0000", 2024-09-14T05:30:44.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0450000000000001e-06 2024-09-14T05:30:44.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357503+0000", 2024-09-14T05:30:44.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9689999999999994e-06 2024-09-14T05:30:44.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357510+0000", 2024-09-14T05:30:44.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-14T05:30:44.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406914+0000", 2024-09-14T05:30:44.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049404293000000002 2024-09-14T05:30:44.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406965+0000", 2024-09-14T05:30:44.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0470999999999998e-05 2024-09-14T05:30:44.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407289+0000", 2024-09-14T05:30:44.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032434899999999998 2024-09-14T05:30:44.456 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.456 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.456 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.457 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4000 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3702784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.457 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:44.457 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.048257492000005, 2024-09-14T05:30:44.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049936728999999999, 2024-09-14T05:30:44.457 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.457 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.457 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.457 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.457 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.457 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.457 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4000 2024-09-14T05:30:44.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.458 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:44.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:44.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357839+0000", 2024-09-14T05:30:44.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-14T05:30:44.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357842+0000", 2024-09-14T05:30:44.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4850000000000001e-06 2024-09-14T05:30:44.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357851+0000", 2024-09-14T05:30:44.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1819999999999995e-06 2024-09-14T05:30:44.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357859+0000", 2024-09-14T05:30:44.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8129999999999996e-06 2024-09-14T05:30:44.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407337+0000", 2024-09-14T05:30:44.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049478084999999998 2024-09-14T05:30:44.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407442+0000", 2024-09-14T05:30:44.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000105302 2024-09-14T05:30:44.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407774+0000", 2024-09-14T05:30:44.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033200100000000001 2024-09-14T05:30:44.461 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.461 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.461 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.461 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4001 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.048162137000006, 2024-09-14T05:30:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050304205999999997, 2024-09-14T05:30:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4001 2024-09-14T05:30:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357934+0000", 2024-09-14T05:30:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000004e-07 2024-09-14T05:30:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357937+0000", 2024-09-14T05:30:44.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0869999999999998e-06 2024-09-14T05:30:44.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357946+0000", 2024-09-14T05:30:44.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1439999999999992e-06 2024-09-14T05:30:44.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357952+0000", 2024-09-14T05:30:44.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.609e-06 2024-09-14T05:30:44.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407820+0000", 2024-09-14T05:30:44.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049868580000000003 2024-09-14T05:30:44.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407909+0000", 2024-09-14T05:30:44.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8887000000000005e-05 2024-09-14T05:30:44.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408237+0000", 2024-09-14T05:30:44.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032824599999999997 2024-09-14T05:30:44.466 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.466 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.466 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.466 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4002 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.466 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:44.466 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.047945541999994, 2024-09-14T05:30:44.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050539529999999999, 2024-09-14T05:30:44.466 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.466 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.467 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.467 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.467 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.467 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.467 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4002 2024-09-14T05:30:44.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.467 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:44.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:44.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:44.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5099999999999999e-07 2024-09-14T05:30:44.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358154+0000", 2024-09-14T05:30:44.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6799999999999999e-06 2024-09-14T05:30:44.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358165+0000", 2024-09-14T05:30:44.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0899e-05 2024-09-14T05:30:44.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358172+0000", 2024-09-14T05:30:44.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8689999999999998e-06 2024-09-14T05:30:44.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408281+0000", 2024-09-14T05:30:44.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050108699 2024-09-14T05:30:44.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408369+0000", 2024-09-14T05:30:44.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8542999999999996e-05 2024-09-14T05:30:44.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408689+0000", 2024-09-14T05:30:44.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032008900000000002 2024-09-14T05:30:44.470 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.471 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.471 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.471 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4003 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5177344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.471 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:44.471 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.047905706999998, 2024-09-14T05:30:44.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050868759, 2024-09-14T05:30:44.471 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.471 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.471 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.471 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.471 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.472 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.472 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4003 2024-09-14T05:30:44.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.472 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:44.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:44.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:44.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-07 2024-09-14T05:30:44.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358193+0000", 2024-09-14T05:30:44.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.588e-06 2024-09-14T05:30:44.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358204+0000", 2024-09-14T05:30:44.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1525999999999999e-05 2024-09-14T05:30:44.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358213+0000", 2024-09-14T05:30:44.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5010000000000004e-06 2024-09-14T05:30:44.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408719+0000", 2024-09-14T05:30:44.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050506708999999997 2024-09-14T05:30:44.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408766+0000", 2024-09-14T05:30:44.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6579999999999998e-05 2024-09-14T05:30:44.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409058+0000", 2024-09-14T05:30:44.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000292365 2024-09-14T05:30:44.475 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.475 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.475 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.476 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4004 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5668864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.476 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:44.476 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.047482864000003, 2024-09-14T05:30:44.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050839848, 2024-09-14T05:30:44.476 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.476 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.476 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.476 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4004 2024-09-14T05:30:44.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.477 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:44.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:44.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358613+0000", 2024-09-14T05:30:44.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:30:44.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358618+0000", 2024-09-14T05:30:44.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6009999999999997e-06 2024-09-14T05:30:44.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358630+0000", 2024-09-14T05:30:44.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2122e-05 2024-09-14T05:30:44.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358638+0000", 2024-09-14T05:30:44.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3969999999999994e-06 2024-09-14T05:30:44.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409088+0000", 2024-09-14T05:30:44.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050449565000000002 2024-09-14T05:30:44.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409153+0000", 2024-09-14T05:30:44.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4728999999999996e-05 2024-09-14T05:30:44.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409452+0000", 2024-09-14T05:30:44.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029958900000000001 2024-09-14T05:30:44.480 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.480 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.480 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.480 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4005 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6160384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.481 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:44.481 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.047265929999995, 2024-09-14T05:30:44.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051054752000000002, 2024-09-14T05:30:44.481 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.481 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.481 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.481 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.481 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.481 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.481 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4005 2024-09-14T05:30:44.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.481 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:44.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:44.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358830+0000", 2024-09-14T05:30:44.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-14T05:30:44.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358833+0000", 2024-09-14T05:30:44.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9529999999999999e-06 2024-09-14T05:30:44.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358843+0000", 2024-09-14T05:30:44.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8069999999999994e-06 2024-09-14T05:30:44.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358849+0000", 2024-09-14T05:30:44.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1040000000000003e-06 2024-09-14T05:30:44.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409487+0000", 2024-09-14T05:30:44.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050638626999999999 2024-09-14T05:30:44.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409535+0000", 2024-09-14T05:30:44.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7525000000000003e-05 2024-09-14T05:30:44.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409884+0000", 2024-09-14T05:30:44.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034914800000000003 2024-09-14T05:30:44.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.485 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.485 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4006 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.485 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:44.485 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.047228817999994, 2024-09-14T05:30:44.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051425323000000002, 2024-09-14T05:30:44.485 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.485 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.486 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.486 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.486 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.486 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.486 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4006 2024-09-14T05:30:44.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.486 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:44.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:44.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358867+0000", 2024-09-14T05:30:44.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2900000000000004e-07 2024-09-14T05:30:44.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358869+0000", 2024-09-14T05:30:44.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4210000000000002e-06 2024-09-14T05:30:44.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358876+0000", 2024-09-14T05:30:44.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4980000000000003e-06 2024-09-14T05:30:44.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358881+0000", 2024-09-14T05:30:44.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3090000000000003e-06 2024-09-14T05:30:44.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409915+0000", 2024-09-14T05:30:44.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051033348999999999 2024-09-14T05:30:44.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409964+0000", 2024-09-14T05:30:44.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9858000000000003e-05 2024-09-14T05:30:44.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410292+0000", 2024-09-14T05:30:44.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032735899999999999 2024-09-14T05:30:44.489 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.490 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.490 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.490 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4007 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.490 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:44.490 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.047206224999996, 2024-09-14T05:30:44.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05184076, 2024-09-14T05:30:44.490 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.490 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.490 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.490 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.490 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.491 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.491 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4007 2024-09-14T05:30:44.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.491 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:44.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:44.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:44.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:30:44.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358891+0000", 2024-09-14T05:30:44.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9819999999999998e-06 2024-09-14T05:30:44.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358897+0000", 2024-09-14T05:30:44.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9390000000000002e-06 2024-09-14T05:30:44.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358902+0000", 2024-09-14T05:30:44.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6770000000000003e-06 2024-09-14T05:30:44.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410335+0000", 2024-09-14T05:30:44.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051432991999999997 2024-09-14T05:30:44.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410416+0000", 2024-09-14T05:30:44.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1316999999999994e-05 2024-09-14T05:30:44.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410730+0000", 2024-09-14T05:30:44.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000313419 2024-09-14T05:30:44.494 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.494 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.494 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.494 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4008 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7634944~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.495 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:44.495 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.047181863999995, 2024-09-14T05:30:44.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052208082000000003, 2024-09-14T05:30:44.495 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.495 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.495 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.495 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.495 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.495 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.495 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4008 2024-09-14T05:30:44.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.496 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:44.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:44.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358914+0000", 2024-09-14T05:30:44.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6100000000000001e-07 2024-09-14T05:30:44.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358916+0000", 2024-09-14T05:30:44.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.902e-06 2024-09-14T05:30:44.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358923+0000", 2024-09-14T05:30:44.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4780000000000002e-06 2024-09-14T05:30:44.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358929+0000", 2024-09-14T05:30:44.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0970000000000002e-06 2024-09-14T05:30:44.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410761+0000", 2024-09-14T05:30:44.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051831436000000002 2024-09-14T05:30:44.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410829+0000", 2024-09-14T05:30:44.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7874999999999996e-05 2024-09-14T05:30:44.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411121+0000", 2024-09-14T05:30:44.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029283299999999999 2024-09-14T05:30:44.499 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.499 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.499 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.499 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4009 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8126464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:44.500 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:44.500 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.047034639000003, 2024-09-14T05:30:44.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052518169000000003, 2024-09-14T05:30:44.500 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:44.500 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:44.500 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:44.500 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:44.500 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:44.500 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.500 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4009 2024-09-14T05:30:44.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.500 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:44.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:44.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:44.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:44.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:44.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:44.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:44.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:44.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0200000000000002e-07 2024-09-14T05:30:44.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:44.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359066+0000", 2024-09-14T05:30:44.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7899999999999999e-06 2024-09-14T05:30:44.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:44.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359074+0000", 2024-09-14T05:30:44.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2579999999999995e-06 2024-09-14T05:30:44.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359082+0000", 2024-09-14T05:30:44.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.464e-06 2024-09-14T05:30:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411160+0000", 2024-09-14T05:30:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052078078 2024-09-14T05:30:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411244+0000", 2024-09-14T05:30:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4041000000000004e-05 2024-09-14T05:30:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411579+0000", 2024-09-14T05:30:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033493599999999998 2024-09-14T05:30:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:44.504 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:44.506 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:30:44.506 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:44.504+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:44.607 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:44.606+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:44.609 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:44.609 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:44.609 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:44.609 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:44.618 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:30:44.707 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:44.706+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:44.720 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:44.720 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:44.720 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:44.720 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:44.720 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:44.729 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:30:44.808 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:44.808+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:44.872 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74187 3.f 3:f707b2d9:::rbd_data.112d46b1ca94.0000000000000061:11 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.302841794000003, 2024-09-14T05:30:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072367828999999995, 2024-09-14T05:30:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74187 2024-09-14T05:30:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0900000000000002e-07 2024-09-14T05:30:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568166+0000", 2024-09-14T05:30:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.683e-06 2024-09-14T05:30:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568172+0000", 2024-09-14T05:30:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0639999999999999e-06 2024-09-14T05:30:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568177+0000", 2024-09-14T05:30:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.967e-06 2024-09-14T05:30:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640193+0000", 2024-09-14T05:30:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072016284 2024-09-14T05:30:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640282+0000", 2024-09-14T05:30:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9011999999999994e-05 2024-09-14T05:30:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640532+0000", 2024-09-14T05:30:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024940999999999999 2024-09-14T05:30:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74188 3.f 3:f707b2d9:::rbd_data.112d46b1ca94.0000000000000061:11 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.302822421000002, 2024-09-14T05:30:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072600425999999996, 2024-09-14T05:30:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74188 2024-09-14T05:30:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568184+0000", 2024-09-14T05:30:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8700000000000001e-07 2024-09-14T05:30:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568200+0000", 2024-09-14T05:30:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6603e-05 2024-09-14T05:30:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568208+0000", 2024-09-14T05:30:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4800000000000004e-06 2024-09-14T05:30:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.880 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568213+0000", 2024-09-14T05:30:44.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6860000000000003e-06 2024-09-14T05:30:44.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640563+0000", 2024-09-14T05:30:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072349438000000002 2024-09-14T05:30:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640607+0000", 2024-09-14T05:30:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4014999999999998e-05 2024-09-14T05:30:44.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640784+0000", 2024-09-14T05:30:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017681700000000001 2024-09-14T05:30:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74194 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.302784289000002, 2024-09-14T05:30:44.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072795199000000005, 2024-09-14T05:30:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74194 2024-09-14T05:30:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:44.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568222+0000", 2024-09-14T05:30:44.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1300000000000001e-07 2024-09-14T05:30:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568224+0000", 2024-09-14T05:30:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0660000000000002e-06 2024-09-14T05:30:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568230+0000", 2024-09-14T05:30:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4470000000000003e-06 2024-09-14T05:30:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568235+0000", 2024-09-14T05:30:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1220000000000003e-06 2024-09-14T05:30:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640824+0000", 2024-09-14T05:30:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072588707000000002 2024-09-14T05:30:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640915+0000", 2024-09-14T05:30:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1008999999999995e-05 2024-09-14T05:30:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641017+0000", 2024-09-14T05:30:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010143499999999999 2024-09-14T05:30:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74208 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2693120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.888 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.302763712999997, 2024-09-14T05:30:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072905555999999996, 2024-09-14T05:30:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74208 2024-09-14T05:30:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.889 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9799999999999999e-07 2024-09-14T05:30:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568244+0000", 2024-09-14T05:30:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0339999999999999e-06 2024-09-14T05:30:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568251+0000", 2024-09-14T05:30:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4060000000000002e-06 2024-09-14T05:30:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568255+0000", 2024-09-14T05:30:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6589999999999998e-06 2024-09-14T05:30:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641048+0000", 2024-09-14T05:30:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072792647000000002 2024-09-14T05:30:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641086+0000", 2024-09-14T05:30:44.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8031999999999998e-05 2024-09-14T05:30:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641147+0000", 2024-09-14T05:30:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1379999999999998e-05 2024-09-14T05:30:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74236 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2662400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.302130050000002, 2024-09-14T05:30:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072517776000000006, 2024-09-14T05:30:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74236 2024-09-14T05:30:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568881+0000", 2024-09-14T05:30:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2340000000000002e-06 2024-09-14T05:30:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568886+0000", 2024-09-14T05:30:44.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9450000000000001e-06 2024-09-14T05:30:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568898+0000", 2024-09-14T05:30:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2150000000000001e-05 2024-09-14T05:30:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568907+0000", 2024-09-14T05:30:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9239999999999996e-06 2024-09-14T05:30:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641185+0000", 2024-09-14T05:30:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.07227807 2024-09-14T05:30:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641265+0000", 2024-09-14T05:30:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0575999999999996e-05 2024-09-14T05:30:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641393+0000", 2024-09-14T05:30:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012787700000000001 2024-09-14T05:30:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.898 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74239 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2695168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.302085656999999, 2024-09-14T05:30:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072593816000000005, 2024-09-14T05:30:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74239 2024-09-14T05:30:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4300000000000003e-07 2024-09-14T05:30:44.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568923+0000", 2024-09-14T05:30:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2019999999999998e-06 2024-09-14T05:30:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568933+0000", 2024-09-14T05:30:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0376e-05 2024-09-14T05:30:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568938+0000", 2024-09-14T05:30:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2850000000000002e-06 2024-09-14T05:30:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641424+0000", 2024-09-14T05:30:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072485230999999997 2024-09-14T05:30:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641464+0000", 2024-09-14T05:30:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0574000000000002e-05 2024-09-14T05:30:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641514+0000", 2024-09-14T05:30:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9604999999999997e-05 2024-09-14T05:30:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74252 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2664448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.903 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.301905742999999, 2024-09-14T05:30:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072631237000000001, 2024-09-14T05:30:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74252 2024-09-14T05:30:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569101+0000", 2024-09-14T05:30:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8700000000000005e-07 2024-09-14T05:30:44.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569104+0000", 2024-09-14T05:30:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.163e-06 2024-09-14T05:30:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569115+0000", 2024-09-14T05:30:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1335000000000001e-05 2024-09-14T05:30:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569121+0000", 2024-09-14T05:30:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0800000000000002e-06 2024-09-14T05:30:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641552+0000", 2024-09-14T05:30:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072430743000000006 2024-09-14T05:30:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641631+0000", 2024-09-14T05:30:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9388999999999995e-05 2024-09-14T05:30:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641731+0000", 2024-09-14T05:30:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9840000000000006e-05 2024-09-14T05:30:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74256 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2697216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.301874687000002, 2024-09-14T05:30:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072721090000000002, 2024-09-14T05:30:44.908 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74256 2024-09-14T05:30:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:44.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0299999999999999e-07 2024-09-14T05:30:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569133+0000", 2024-09-14T05:30:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.046e-06 2024-09-14T05:30:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569139+0000", 2024-09-14T05:30:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0279999999999997e-06 2024-09-14T05:30:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569144+0000", 2024-09-14T05:30:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6929999999999998e-06 2024-09-14T05:30:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641762+0000", 2024-09-14T05:30:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072617904999999996 2024-09-14T05:30:44.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641801+0000", 2024-09-14T05:30:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8902999999999997e-05 2024-09-14T05:30:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641852+0000", 2024-09-14T05:30:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1011999999999999e-05 2024-09-14T05:30:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74285 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2672640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.301029251000003, 2024-09-14T05:30:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072422069000000006, 2024-09-14T05:30:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74285 2024-09-14T05:30:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569977+0000", 2024-09-14T05:30:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3099999999999998e-07 2024-09-14T05:30:44.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569978+0000", 2024-09-14T05:30:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.598e-06 2024-09-14T05:30:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569984+0000", 2024-09-14T05:30:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4689999999999996e-06 2024-09-14T05:30:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569989+0000", 2024-09-14T05:30:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8409999999999999e-06 2024-09-14T05:30:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642229+0000", 2024-09-14T05:30:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072240046000000002 2024-09-14T05:30:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642306+0000", 2024-09-14T05:30:44.917 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7347000000000003e-05 2024-09-14T05:30:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642398+0000", 2024-09-14T05:30:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2336999999999994e-05 2024-09-14T05:30:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74292 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2705408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.300774543000003, 2024-09-14T05:30:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072361018999999999, 2024-09-14T05:30:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74292 2024-09-14T05:30:44.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570232+0000", 2024-09-14T05:30:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.06e-07 2024-09-14T05:30:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570235+0000", 2024-09-14T05:30:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3189999999999999e-06 2024-09-14T05:30:44.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570248+0000", 2024-09-14T05:30:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3006e-05 2024-09-14T05:30:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570254+0000", 2024-09-14T05:30:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.908e-06 2024-09-14T05:30:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642433+0000", 2024-09-14T05:30:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072179192000000003 2024-09-14T05:30:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642500+0000", 2024-09-14T05:30:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7080000000000001e-05 2024-09-14T05:30:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642592+0000", 2024-09-14T05:30:44.923 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1807999999999999e-05 2024-09-14T05:30:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74304 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2707456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.300417877000001, 2024-09-14T05:30:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072444960000000003, 2024-09-14T05:30:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.924 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74304 2024-09-14T05:30:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.925 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9800000000000004e-07 2024-09-14T05:30:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570590+0000", 2024-09-14T05:30:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0810000000000001e-06 2024-09-14T05:30:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.926 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570596+0000", 2024-09-14T05:30:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9789999999999998e-06 2024-09-14T05:30:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570601+0000", 2024-09-14T05:30:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6750000000000001e-06 2024-09-14T05:30:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.927 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642835+0000", 2024-09-14T05:30:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.07223359 2024-09-14T05:30:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642917+0000", 2024-09-14T05:30:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2013000000000002e-05 2024-09-14T05:30:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643033+0000", 2024-09-14T05:30:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116124 2024-09-14T05:30:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.928 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74321 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2709504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.299935886999997, 2024-09-14T05:30:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072350611999999995, 2024-09-14T05:30:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.929 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74321 2024-09-14T05:30:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3e-07 2024-09-14T05:30:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571073+0000", 2024-09-14T05:30:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6879999999999999e-06 2024-09-14T05:30:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571080+0000", 2024-09-14T05:30:44.931 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0319999999999998e-06 2024-09-14T05:30:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571085+0000", 2024-09-14T05:30:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2109999999999997e-06 2024-09-14T05:30:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643309+0000", 2024-09-14T05:30:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072223812999999998 2024-09-14T05:30:44.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643351+0000", 2024-09-14T05:30:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2239e-05 2024-09-14T05:30:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643420+0000", 2024-09-14T05:30:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8998999999999994e-05 2024-09-14T05:30:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.933 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74332 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2678784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.299612723999999, 2024-09-14T05:30:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072259062999999998, 2024-09-14T05:30:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.934 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74332 2024-09-14T05:30:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571394+0000", 2024-09-14T05:30:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:30:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571397+0000", 2024-09-14T05:30:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8469999999999998e-06 2024-09-14T05:30:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571411+0000", 2024-09-14T05:30:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.34e-05 2024-09-14T05:30:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571418+0000", 2024-09-14T05:30:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9149999999999999e-06 2024-09-14T05:30:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643458+0000", 2024-09-14T05:30:44.937 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072039881 2024-09-14T05:30:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643539+0000", 2024-09-14T05:30:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1650000000000006e-05 2024-09-14T05:30:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643652+0000", 2024-09-14T05:30:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000112593 2024-09-14T05:30:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74336 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2711552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.299576274000003, 2024-09-14T05:30:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072354038999999995, 2024-09-14T05:30:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74336 2024-09-14T05:30:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.939 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.940 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571430+0000", 2024-09-14T05:30:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8999999999999996e-07 2024-09-14T05:30:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571432+0000", 2024-09-14T05:30:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4159999999999998e-06 2024-09-14T05:30:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571439+0000", 2024-09-14T05:30:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.657e-06 2024-09-14T05:30:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571444+0000", 2024-09-14T05:30:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1329999999999998e-06 2024-09-14T05:30:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643690+0000", 2024-09-14T05:30:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072246088 2024-09-14T05:30:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643731+0000", 2024-09-14T05:30:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0311000000000001e-05 2024-09-14T05:30:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643783+0000", 2024-09-14T05:30:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2843999999999998e-05 2024-09-14T05:30:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.943 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74349 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.299555265000002, 2024-09-14T05:30:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072554297000000004, 2024-09-14T05:30:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74349 2024-09-14T05:30:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571451+0000", 2024-09-14T05:30:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.41e-07 2024-09-14T05:30:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571453+0000", 2024-09-14T05:30:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8360000000000001e-06 2024-09-14T05:30:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571475+0000", 2024-09-14T05:30:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2452000000000001e-05 2024-09-14T05:30:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571480+0000", 2024-09-14T05:30:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4430000000000002e-06 2024-09-14T05:30:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643821+0000", 2024-09-14T05:30:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072340347999999999 2024-09-14T05:30:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643904+0000", 2024-09-14T05:30:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3534000000000004e-05 2024-09-14T05:30:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644005+0000", 2024-09-14T05:30:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100343 2024-09-14T05:30:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74357 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2713600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.299511232, 2024-09-14T05:30:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072651144000000001, 2024-09-14T05:30:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74357 2024-09-14T05:30:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571495+0000", 2024-09-14T05:30:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8899999999999999e-07 2024-09-14T05:30:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571498+0000", 2024-09-14T05:30:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5840000000000002e-06 2024-09-14T05:30:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571504+0000", 2024-09-14T05:30:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6549999999999998e-06 2024-09-14T05:30:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571509+0000", 2024-09-14T05:30:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6500000000000004e-06 2024-09-14T05:30:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644043+0000", 2024-09-14T05:30:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072533832000000006 2024-09-14T05:30:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644090+0000", 2024-09-14T05:30:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7426000000000003e-05 2024-09-14T05:30:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644146+0000", 2024-09-14T05:30:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5408000000000002e-05 2024-09-14T05:30:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74364 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2682880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.299174460000003, 2024-09-14T05:30:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072531227000000004, 2024-09-14T05:30:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74364 2024-09-14T05:30:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571832+0000", 2024-09-14T05:30:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4e-07 2024-09-14T05:30:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571838+0000", 2024-09-14T05:30:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0519999999999998e-06 2024-09-14T05:30:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571865+0000", 2024-09-14T05:30:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7109000000000001e-05 2024-09-14T05:30:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571883+0000", 2024-09-14T05:30:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7428e-05 2024-09-14T05:30:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644182+0000", 2024-09-14T05:30:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072299551000000004 2024-09-14T05:30:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644260+0000", 2024-09-14T05:30:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7551000000000003e-05 2024-09-14T05:30:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644362+0000", 2024-09-14T05:30:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010259600000000001 2024-09-14T05:30:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74367 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2715648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.298811153000003, 2024-09-14T05:30:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072368259000000004, 2024-09-14T05:30:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74367 2024-09-14T05:30:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572195+0000", 2024-09-14T05:30:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.78e-07 2024-09-14T05:30:44.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572200+0000", 2024-09-14T05:30:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2520000000000001e-06 2024-09-14T05:30:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572213+0000", 2024-09-14T05:30:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3618000000000001e-05 2024-09-14T05:30:44.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572226+0000", 2024-09-14T05:30:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3040999999999999e-05 2024-09-14T05:30:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644409+0000", 2024-09-14T05:30:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072182982000000007 2024-09-14T05:30:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644489+0000", 2024-09-14T05:30:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9429000000000001e-05 2024-09-14T05:30:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644563+0000", 2024-09-14T05:30:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4059000000000004e-05 2024-09-14T05:30:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74380 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2684928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.298677374999997, 2024-09-14T05:30:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072442180999999994, 2024-09-14T05:30:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74380 2024-09-14T05:30:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:44.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572329+0000", 2024-09-14T05:30:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3300000000000001e-07 2024-09-14T05:30:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572333+0000", 2024-09-14T05:30:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8e-06 2024-09-14T05:30:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572346+0000", 2024-09-14T05:30:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.307e-05 2024-09-14T05:30:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572353+0000", 2024-09-14T05:30:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9e-06 2024-09-14T05:30:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644597+0000", 2024-09-14T05:30:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072244770999999999 2024-09-14T05:30:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644672+0000", 2024-09-14T05:30:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4945999999999997e-05 2024-09-14T05:30:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644770+0000", 2024-09-14T05:30:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7961000000000006e-05 2024-09-14T05:30:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74383 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2717696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:44.970 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 55.298421380999997, 2024-09-14T05:30:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072397258000000006, 2024-09-14T05:30:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74383 2024-09-14T05:30:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:44.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.972 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572585+0000", 2024-09-14T05:30:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:30:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:44.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572589+0000", 2024-09-14T05:30:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:30:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572602+0000", 2024-09-14T05:30:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2917e-05 2024-09-14T05:30:44.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572610+0000", 2024-09-14T05:30:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.943e-06 2024-09-14T05:30:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644791+0000", 2024-09-14T05:30:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072181113000000005 2024-09-14T05:30:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:44.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644861+0000", 2024-09-14T05:30:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0650999999999998e-05 2024-09-14T05:30:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644981+0000", 2024-09-14T05:30:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000120185 2024-09-14T05:30:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:44.976 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:44.976 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:44.978 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:30:44.978 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:44.977+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:45.059 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T05:30:45.079 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T05:30:45.079 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:45.078+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:45.143 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:45.143 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:45.144 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:45.144 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:45.154 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:30:45.179 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:45.179+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:45.280 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:45.280+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:45.300 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:45.300 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:45.300 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:45.300 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:45.300 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:45.309 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:30:45.380 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:45.379+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:45.414 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:30:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:30:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 97.636080613999994, 2024-09-14T05:30:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.24562984099999999, 2024-09-14T05:30:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:30:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:30:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1 2024-09-14T05:30:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:30:45.415 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:30:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776565+0000", 2024-09-14T05:30:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4780000000000001e-06 2024-09-14T05:30:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.416 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776571+0000", 2024-09-14T05:30:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7669999999999998e-06 2024-09-14T05:30:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776580+0000", 2024-09-14T05:30:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9830000000000002e-06 2024-09-14T05:30:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776589+0000", 2024-09-14T05:30:45.417 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7579999999999998e-06 2024-09-14T05:30:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776662+0000", 2024-09-14T05:30:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3332999999999997e-05 2024-09-14T05:30:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:30:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776670+0000", 2024-09-14T05:30:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9850000000000006e-06 2024-09-14T05:30:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.418 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.020867+0000", 2024-09-14T05:30:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.244196265 2024-09-14T05:30:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021022+0000", 2024-09-14T05:30:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000155699 2024-09-14T05:30:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021169+0000", 2024-09-14T05:30:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014661700000000001 2024-09-14T05:30:45.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:30:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021891+0000", 2024-09-14T05:30:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072146700000000005 2024-09-14T05:30:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022124+0000", 2024-09-14T05:30:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023354199999999999 2024-09-14T05:30:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:30:45.420 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022164+0000", 2024-09-14T05:30:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9897000000000001e-05 2024-09-14T05:30:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022193+0000", 2024-09-14T05:30:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9050000000000001e-05 2024-09-14T05:30:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:1989 3.1f e30/24 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head v 30'29, mlcod=30'27)", 2024-09-14T05:30:45.421 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:30:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.819309626999996, 2024-09-14T05:30:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.064242492999999998, 2024-09-14T05:30:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:30:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.422 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:30:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593336+0000", 2024-09-14T05:30:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3909999999999999e-06 2024-09-14T05:30:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616632+0000", 2024-09-14T05:30:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.023296598000000002 2024-09-14T05:30:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616657+0000", 2024-09-14T05:30:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5228e-05 2024-09-14T05:30:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616684+0000", 2024-09-14T05:30:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6441999999999999e-05 2024-09-14T05:30:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.424 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616755+0000", 2024-09-14T05:30:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1452000000000006e-05 2024-09-14T05:30:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616780+0000", 2024-09-14T05:30:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4436000000000001e-05 2024-09-14T05:30:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:30:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657527+0000", 2024-09-14T05:30:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.040747323000000002 2024-09-14T05:30:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657577+0000", 2024-09-14T05:30:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9623000000000002e-05 2024-09-14T05:30:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124262 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4481024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867845+0000", 2024-09-14T05:30:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.544798428, 2024-09-14T05:30:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048759513999999997, 2024-09-14T05:30:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.426 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124262 2024-09-14T05:30:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867845+0000", 2024-09-14T05:30:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867845+0000", 2024-09-14T05:30:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867846+0000", 2024-09-14T05:30:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2e-07 2024-09-14T05:30:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.428 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867848+0000", 2024-09-14T05:30:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0339999999999999e-06 2024-09-14T05:30:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867854+0000", 2024-09-14T05:30:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7350000000000001e-06 2024-09-14T05:30:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867859+0000", 2024-09-14T05:30:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9690000000000002e-06 2024-09-14T05:30:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916436+0000", 2024-09-14T05:30:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048577694999999997 2024-09-14T05:30:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916515+0000", 2024-09-14T05:30:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8982999999999999e-05 2024-09-14T05:30:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916605+0000", 2024-09-14T05:30:45.430 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9578000000000002e-05 2024-09-14T05:30:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124263 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4587520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:30:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.544778801, 2024-09-14T05:30:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048866746000000003, 2024-09-14T05:30:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.431 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124263 2024-09-14T05:30:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:30:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:30:45.432 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:30:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2199999999999999e-07 2024-09-14T05:30:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867867+0000", 2024-09-14T05:30:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.091e-06 2024-09-14T05:30:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.433 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867873+0000", 2024-09-14T05:30:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8869999999999997e-06 2024-09-14T05:30:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867884+0000", 2024-09-14T05:30:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0784999999999999e-05 2024-09-14T05:30:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916636+0000", 2024-09-14T05:30:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048752141999999998 2024-09-14T05:30:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916680+0000", 2024-09-14T05:30:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3621000000000001e-05 2024-09-14T05:30:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916732+0000", 2024-09-14T05:30:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1798000000000001e-05 2024-09-14T05:30:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.436 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124264 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4694016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:45.436 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867891+0000", 2024-09-14T05:30:45.436 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.544752446, 2024-09-14T05:30:45.436 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049046027999999998, 2024-09-14T05:30:45.436 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124264 2024-09-14T05:30:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867891+0000", 2024-09-14T05:30:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867891+0000", 2024-09-14T05:30:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867892+0000", 2024-09-14T05:30:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.39e-07 2024-09-14T05:30:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.438 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867894+0000", 2024-09-14T05:30:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.278e-06 2024-09-14T05:30:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867900+0000", 2024-09-14T05:30:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2469999999999997e-06 2024-09-14T05:30:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867905+0000", 2024-09-14T05:30:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0209999999999999e-06 2024-09-14T05:30:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.439 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916764+0000", 2024-09-14T05:30:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048859163999999997 2024-09-14T05:30:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916845+0000", 2024-09-14T05:30:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0309999999999995e-05 2024-09-14T05:30:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916937+0000", 2024-09-14T05:30:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2569000000000001e-05 2024-09-14T05:30:45.440 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124265 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:30:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.544732240999998, 2024-09-14T05:30:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049155040999999997, 2024-09-14T05:30:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.441 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124265 2024-09-14T05:30:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:30:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:30:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:30:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7099999999999997e-07 2024-09-14T05:30:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867914+0000", 2024-09-14T05:30:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9810000000000002e-06 2024-09-14T05:30:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867920+0000", 2024-09-14T05:30:45.443 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.643e-06 2024-09-14T05:30:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867924+0000", 2024-09-14T05:30:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.605e-06 2024-09-14T05:30:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916970+0000", 2024-09-14T05:30:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049045501999999998 2024-09-14T05:30:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917013+0000", 2024-09-14T05:30:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3857999999999999e-05 2024-09-14T05:30:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.445 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917067+0000", 2024-09-14T05:30:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3081000000000002e-05 2024-09-14T05:30:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124266 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4907008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867930+0000", 2024-09-14T05:30:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.544713382000001, 2024-09-14T05:30:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049337389000000002, 2024-09-14T05:30:45.446 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124266 2024-09-14T05:30:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867930+0000", 2024-09-14T05:30:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867930+0000", 2024-09-14T05:30:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867931+0000", 2024-09-14T05:30:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9999999999999998e-07 2024-09-14T05:30:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.448 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867933+0000", 2024-09-14T05:30:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.046e-06 2024-09-14T05:30:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867938+0000", 2024-09-14T05:30:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5589999999999996e-06 2024-09-14T05:30:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867941+0000", 2024-09-14T05:30:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.791e-06 2024-09-14T05:30:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.449 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917100+0000", 2024-09-14T05:30:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049158544999999998 2024-09-14T05:30:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917179+0000", 2024-09-14T05:30:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9350000000000004e-05 2024-09-14T05:30:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917268+0000", 2024-09-14T05:30:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8697999999999997e-05 2024-09-14T05:30:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.450 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124267 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5013504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:30:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.544698826000001, 2024-09-14T05:30:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049449223, 2024-09-14T05:30:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.451 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124267 2024-09-14T05:30:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:30:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.452 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:30:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:30:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.28e-07 2024-09-14T05:30:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867946+0000", 2024-09-14T05:30:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.116e-06 2024-09-14T05:30:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.453 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867950+0000", 2024-09-14T05:30:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2049999999999998e-06 2024-09-14T05:30:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867952+0000", 2024-09-14T05:30:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6800000000000002e-06 2024-09-14T05:30:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917300+0000", 2024-09-14T05:30:45.454 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049347633000000002 2024-09-14T05:30:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917343+0000", 2024-09-14T05:30:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3022000000000002e-05 2024-09-14T05:30:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917394+0000", 2024-09-14T05:30:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1338999999999998e-05 2024-09-14T05:30:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124268 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5120000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867955+0000", 2024-09-14T05:30:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.544688419, 2024-09-14T05:30:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049638350999999997, 2024-09-14T05:30:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.456 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124268 2024-09-14T05:30:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867955+0000", 2024-09-14T05:30:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867955+0000", 2024-09-14T05:30:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867956+0000", 2024-09-14T05:30:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1299999999999999e-07 2024-09-14T05:30:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867961+0000", 2024-09-14T05:30:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4269999999999999e-06 2024-09-14T05:30:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.458 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867965+0000", 2024-09-14T05:30:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.563e-06 2024-09-14T05:30:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867967+0000", 2024-09-14T05:30:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6469999999999998e-06 2024-09-14T05:30:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917427+0000", 2024-09-14T05:30:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049460102999999998 2024-09-14T05:30:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917506+0000", 2024-09-14T05:30:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8246999999999998e-05 2024-09-14T05:30:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917594+0000", 2024-09-14T05:30:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8151000000000004e-05 2024-09-14T05:30:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124269 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:30:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.544672802000001, 2024-09-14T05:30:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049749122999999999, 2024-09-14T05:30:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.461 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124269 2024-09-14T05:30:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:30:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:30:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:30:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2499999999999999e-07 2024-09-14T05:30:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867972+0000", 2024-09-14T05:30:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0440000000000001e-06 2024-09-14T05:30:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867975+0000", 2024-09-14T05:30:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9699999999999999e-06 2024-09-14T05:30:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.464 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867978+0000", 2024-09-14T05:30:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5909999999999998e-06 2024-09-14T05:30:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917625+0000", 2024-09-14T05:30:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049647450000000003 2024-09-14T05:30:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917668+0000", 2024-09-14T05:30:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3108000000000001e-05 2024-09-14T05:30:45.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917720+0000", 2024-09-14T05:30:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1734999999999998e-05 2024-09-14T05:30:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124270 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5292032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:30:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.544662861999999, 2024-09-14T05:30:45.466 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049946404, 2024-09-14T05:30:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124270 2024-09-14T05:30:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:30:45.467 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:30:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:30:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5600000000000002e-07 2024-09-14T05:30:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.468 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867982+0000", 2024-09-14T05:30:45.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.95e-07 2024-09-14T05:30:45.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867985+0000", 2024-09-14T05:30:45.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8890000000000002e-06 2024-09-14T05:30:45.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.469 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.469 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867988+0000", 2024-09-14T05:30:45.469 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5900000000000002e-06 2024-09-14T05:30:45.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.470 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.470 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917753+0000", 2024-09-14T05:30:45.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049765113 2024-09-14T05:30:45.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.470 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.470 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917833+0000", 2024-09-14T05:30:45.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0055000000000005e-05 2024-09-14T05:30:45.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.470 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.470 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917927+0000", 2024-09-14T05:30:45.470 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4506000000000006e-05 2024-09-14T05:30:45.471 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.471 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.471 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.471 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124271 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5398528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:45.471 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868010+0000", 2024-09-14T05:30:45.471 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.544633666999999, 2024-09-14T05:30:45.471 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050049336, 2024-09-14T05:30:45.471 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.471 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.471 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.471 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.472 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.472 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.472 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124271 2024-09-14T05:30:45.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.472 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868010+0000", 2024-09-14T05:30:45.472 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.472 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.472 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868010+0000", 2024-09-14T05:30:45.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868011+0000", 2024-09-14T05:30:45.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:45.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.473 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868012+0000", 2024-09-14T05:30:45.473 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5719999999999999e-06 2024-09-14T05:30:45.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.473 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.474 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868016+0000", 2024-09-14T05:30:45.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.861e-06 2024-09-14T05:30:45.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.474 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.474 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868019+0000", 2024-09-14T05:30:45.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7580000000000001e-06 2024-09-14T05:30:45.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.474 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.474 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917961+0000", 2024-09-14T05:30:45.474 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049941885999999998 2024-09-14T05:30:45.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.475 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918007+0000", 2024-09-14T05:30:45.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6187999999999999e-05 2024-09-14T05:30:45.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.475 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.475 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918059+0000", 2024-09-14T05:30:45.475 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2645000000000002e-05 2024-09-14T05:30:45.475 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.475 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.475 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.475 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124272 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:45.476 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:30:45.476 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.544621639999999, 2024-09-14T05:30:45.476 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050239936999999998, 2024-09-14T05:30:45.476 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.476 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.476 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.476 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.476 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.476 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.476 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124272 2024-09-14T05:30:45.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.476 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.476 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:30:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:30:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:30:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.16e-07 2024-09-14T05:30:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868023+0000", 2024-09-14T05:30:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7999999999999993e-07 2024-09-14T05:30:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868026+0000", 2024-09-14T05:30:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9859999999999999e-06 2024-09-14T05:30:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868029+0000", 2024-09-14T05:30:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.039e-06 2024-09-14T05:30:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.479 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.479 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918093+0000", 2024-09-14T05:30:45.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050063689000000001 2024-09-14T05:30:45.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.479 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.479 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918173+0000", 2024-09-14T05:30:45.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0131999999999998e-05 2024-09-14T05:30:45.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.479 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.479 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918262+0000", 2024-09-14T05:30:45.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8894999999999995e-05 2024-09-14T05:30:45.480 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.480 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.480 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.480 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124273 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:45.480 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868032+0000", 2024-09-14T05:30:45.480 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.544611491000001, 2024-09-14T05:30:45.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050357587000000002, 2024-09-14T05:30:45.480 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.480 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.480 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.480 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.480 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.481 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.481 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124273 2024-09-14T05:30:45.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.481 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868032+0000", 2024-09-14T05:30:45.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868032+0000", 2024-09-14T05:30:45.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868033+0000", 2024-09-14T05:30:45.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9799999999999999e-07 2024-09-14T05:30:45.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868034+0000", 2024-09-14T05:30:45.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1119999999999999e-06 2024-09-14T05:30:45.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868037+0000", 2024-09-14T05:30:45.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0189999999999998e-06 2024-09-14T05:30:45.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.483 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.483 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868039+0000", 2024-09-14T05:30:45.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.57e-06 2024-09-14T05:30:45.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.483 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.483 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918294+0000", 2024-09-14T05:30:45.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050254778999999999 2024-09-14T05:30:45.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.483 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918338+0000", 2024-09-14T05:30:45.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4288e-05 2024-09-14T05:30:45.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918390+0000", 2024-09-14T05:30:45.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1520999999999997e-05 2024-09-14T05:30:45.484 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.484 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.484 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.484 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124274 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:45.484 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:30:45.484 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.544601649000001, 2024-09-14T05:30:45.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050548334, 2024-09-14T05:30:45.485 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.485 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.485 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.485 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.485 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.485 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.485 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124274 2024-09-14T05:30:45.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.485 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:30:45.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.486 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.486 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:30:45.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.486 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.486 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:30:45.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1799999999999999e-07 2024-09-14T05:30:45.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.486 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.486 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868043+0000", 2024-09-14T05:30:45.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9999999999999995e-07 2024-09-14T05:30:45.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868046+0000", 2024-09-14T05:30:45.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8880000000000001e-06 2024-09-14T05:30:45.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868049+0000", 2024-09-14T05:30:45.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7e-06 2024-09-14T05:30:45.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918423+0000", 2024-09-14T05:30:45.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050374249000000003 2024-09-14T05:30:45.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918503+0000", 2024-09-14T05:30:45.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0049000000000006e-05 2024-09-14T05:30:45.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.488 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.488 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918590+0000", 2024-09-14T05:30:45.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7230000000000003e-05 2024-09-14T05:30:45.488 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.489 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.489 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.489 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124275 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5824512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:45.489 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:30:45.489 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.544592063, 2024-09-14T05:30:45.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050665241999999999, 2024-09-14T05:30:45.489 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.489 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.489 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.489 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.489 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.490 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.490 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124275 2024-09-14T05:30:45.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.490 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:30:45.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:30:45.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:30:45.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1e-07 2024-09-14T05:30:45.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.492 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.492 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868053+0000", 2024-09-14T05:30:45.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.049e-06 2024-09-14T05:30:45.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.493 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868056+0000", 2024-09-14T05:30:45.493 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9519999999999999e-06 2024-09-14T05:30:45.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.493 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868058+0000", 2024-09-14T05:30:45.493 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4789999999999999e-06 2024-09-14T05:30:45.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918622+0000", 2024-09-14T05:30:45.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050563679 2024-09-14T05:30:45.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918666+0000", 2024-09-14T05:30:45.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3600000000000003e-05 2024-09-14T05:30:45.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918717+0000", 2024-09-14T05:30:45.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1273000000000002e-05 2024-09-14T05:30:45.495 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.495 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.495 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.495 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124276 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:45.495 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:30:45.495 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.544582518999999, 2024-09-14T05:30:45.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050860996999999998, 2024-09-14T05:30:45.495 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.495 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.495 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.496 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.496 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.496 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.496 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124276 2024-09-14T05:30:45.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.496 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.496 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.496 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:30:45.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:30:45.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:30:45.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1e-07 2024-09-14T05:30:45.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868062+0000", 2024-09-14T05:30:45.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2099999999999995e-07 2024-09-14T05:30:45.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868065+0000", 2024-09-14T05:30:45.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9500000000000001e-06 2024-09-14T05:30:45.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868068+0000", 2024-09-14T05:30:45.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4760000000000001e-06 2024-09-14T05:30:45.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918750+0000", 2024-09-14T05:30:45.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050681926000000002 2024-09-14T05:30:45.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918829+0000", 2024-09-14T05:30:45.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9264000000000005e-05 2024-09-14T05:30:45.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918922+0000", 2024-09-14T05:30:45.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3250000000000003e-05 2024-09-14T05:30:45.500 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.500 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.500 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.500 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124277 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 6037504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:45.500 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:30:45.500 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.544572910999999, 2024-09-14T05:30:45.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050980939000000003, 2024-09-14T05:30:45.501 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.501 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.501 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.501 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.501 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.501 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.501 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124277 2024-09-14T05:30:45.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.501 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.501 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:30:45.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:30:45.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:30:45.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.11e-07 2024-09-14T05:30:45.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868072+0000", 2024-09-14T05:30:45.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9999999999999995e-07 2024-09-14T05:30:45.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868075+0000", 2024-09-14T05:30:45.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9969999999999999e-06 2024-09-14T05:30:45.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868078+0000", 2024-09-14T05:30:45.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5349999999999999e-06 2024-09-14T05:30:45.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918956+0000", 2024-09-14T05:30:45.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050878067999999999 2024-09-14T05:30:45.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919000+0000", 2024-09-14T05:30:45.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3837000000000001e-05 2024-09-14T05:30:45.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919052+0000", 2024-09-14T05:30:45.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2290999999999998e-05 2024-09-14T05:30:45.505 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.505 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.505 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124278 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 6144000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868080+0000", 2024-09-14T05:30:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.544563473, 2024-09-14T05:30:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.051170446000000001, 2024-09-14T05:30:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.506 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124278 2024-09-14T05:30:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868080+0000", 2024-09-14T05:30:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868080+0000", 2024-09-14T05:30:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868081+0000", 2024-09-14T05:30:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1500000000000001e-07 2024-09-14T05:30:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868081+0000", 2024-09-14T05:30:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9400000000000004e-07 2024-09-14T05:30:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868084+0000", 2024-09-14T05:30:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.943e-06 2024-09-14T05:30:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868087+0000", 2024-09-14T05:30:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6929999999999999e-06 2024-09-14T05:30:45.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919084+0000", 2024-09-14T05:30:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050996581999999999 2024-09-14T05:30:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919163+0000", 2024-09-14T05:30:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8931999999999996e-05 2024-09-14T05:30:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919251+0000", 2024-09-14T05:30:45.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8187000000000001e-05 2024-09-14T05:30:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124279 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 6250496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:30:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 27.544553863000001, 2024-09-14T05:30:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.051284430999999998, 2024-09-14T05:30:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.514 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124279 2024-09-14T05:30:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:30:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:30:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:30:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.11e-07 2024-09-14T05:30:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868091+0000", 2024-09-14T05:30:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8299999999999995e-07 2024-09-14T05:30:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868094+0000", 2024-09-14T05:30:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8870000000000001e-06 2024-09-14T05:30:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868097+0000", 2024-09-14T05:30:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5330000000000001e-06 2024-09-14T05:30:45.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919276+0000", 2024-09-14T05:30:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.051179740000000001 2024-09-14T05:30:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919322+0000", 2024-09-14T05:30:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5475000000000001e-05 2024-09-14T05:30:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919374+0000", 2024-09-14T05:30:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2602000000000002e-05 2024-09-14T05:30:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.519 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.519 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:45.521 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:30:45.521 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:45.520+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:45.621 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:45.619+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:45.665 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:45.665 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:45.665 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:45.665 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:45.673 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:30:45.762 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:45.760+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:45.820 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:45.821 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:45.821 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:45.821 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:45.821 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:45.822 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:45.822+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:45.828 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:30:45.923 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:45.922+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:45.965 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:30:45.973 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:45.973 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:45.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:45.973 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:45.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.974 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73760 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2848768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:45.974 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:45.974 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.412465672000003, 2024-09-14T05:30:45.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.04469911, 2024-09-14T05:30:45.974 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.974 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.974 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.974 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.974 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.974 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.974 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73760 2024-09-14T05:30:45.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.975 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:45.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:45.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:45.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:45.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559701+0000", 2024-09-14T05:30:45.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1909999999999999e-06 2024-09-14T05:30:45.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.976 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.976 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559708+0000", 2024-09-14T05:30:45.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7279999999999998e-06 2024-09-14T05:30:45.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559713+0000", 2024-09-14T05:30:45.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8389999999999997e-06 2024-09-14T05:30:45.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604310+0000", 2024-09-14T05:30:45.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044597664000000002 2024-09-14T05:30:45.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604352+0000", 2024-09-14T05:30:45.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1236000000000003e-05 2024-09-14T05:30:45.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604398+0000", 2024-09-14T05:30:45.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6091000000000003e-05 2024-09-14T05:30:45.978 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.978 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.978 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.978 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73773 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2801664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:45.979 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:45.979 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.412445503000001, 2024-09-14T05:30:45.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941026000000002, 2024-09-14T05:30:45.979 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.979 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.979 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.979 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.979 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.979 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.979 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73773 2024-09-14T05:30:45.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.979 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:45.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:45.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:45.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8599999999999999e-07 2024-09-14T05:30:45.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559721+0000", 2024-09-14T05:30:45.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0540000000000002e-06 2024-09-14T05:30:45.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559729+0000", 2024-09-14T05:30:45.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1359999999999997e-06 2024-09-14T05:30:45.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.981 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.981 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559734+0000", 2024-09-14T05:30:45.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2140000000000004e-06 2024-09-14T05:30:45.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604441+0000", 2024-09-14T05:30:45.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044706828999999997 2024-09-14T05:30:45.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604522+0000", 2024-09-14T05:30:45.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0603000000000001e-05 2024-09-14T05:30:45.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604660+0000", 2024-09-14T05:30:45.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000137804 2024-09-14T05:30:45.983 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.983 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.983 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.983 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73777 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2834432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:45.983 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:45.983 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.412423101999998, 2024-09-14T05:30:45.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045116270999999999, 2024-09-14T05:30:45.983 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.983 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.984 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.984 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.984 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.984 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.984 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73777 2024-09-14T05:30:45.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.984 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:45.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:45.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559742+0000", 2024-09-14T05:30:45.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4099999999999999e-07 2024-09-14T05:30:45.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559744+0000", 2024-09-14T05:30:45.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0040000000000002e-06 2024-09-14T05:30:45.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559749+0000", 2024-09-14T05:30:45.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9739999999999999e-06 2024-09-14T05:30:45.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559754+0000", 2024-09-14T05:30:45.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9540000000000003e-06 2024-09-14T05:30:45.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604695+0000", 2024-09-14T05:30:45.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941003 2024-09-14T05:30:45.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604766+0000", 2024-09-14T05:30:45.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0296999999999995e-05 2024-09-14T05:30:45.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604857+0000", 2024-09-14T05:30:45.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1598e-05 2024-09-14T05:30:45.987 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.987 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.987 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.987 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73790 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:45.987 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:45.988 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.412403167000001, 2024-09-14T05:30:45.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316437000000001, 2024-09-14T05:30:45.988 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.988 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.988 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.988 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.988 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.988 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.988 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73790 2024-09-14T05:30:45.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.988 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:45.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:45.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:45.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7599999999999998e-07 2024-09-14T05:30:45.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559763+0000", 2024-09-14T05:30:45.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0880000000000002e-06 2024-09-14T05:30:45.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559769+0000", 2024-09-14T05:30:45.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6740000000000002e-06 2024-09-14T05:30:45.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559774+0000", 2024-09-14T05:30:45.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6110000000000004e-06 2024-09-14T05:30:45.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604905+0000", 2024-09-14T05:30:45.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045130827999999998 2024-09-14T05:30:45.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604982+0000", 2024-09-14T05:30:45.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7410999999999995e-05 2024-09-14T05:30:45.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605077+0000", 2024-09-14T05:30:45.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5449e-05 2024-09-14T05:30:45.992 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.992 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.992 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.992 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73793 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2836480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:45.992 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:45.992 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.412383963000003, 2024-09-14T05:30:45.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045441212000000002, 2024-09-14T05:30:45.992 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.992 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.993 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.993 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.993 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.993 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.993 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73793 2024-09-14T05:30:45.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.993 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:45.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:45.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559781+0000", 2024-09-14T05:30:45.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2899999999999999e-07 2024-09-14T05:30:45.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559783+0000", 2024-09-14T05:30:45.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.075e-06 2024-09-14T05:30:45.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559788+0000", 2024-09-14T05:30:45.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5369999999999997e-06 2024-09-14T05:30:45.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:45.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559793+0000", 2024-09-14T05:30:45.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:30:45.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:45.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605109+0000", 2024-09-14T05:30:45.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316229999999999 2024-09-14T05:30:45.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:45.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605168+0000", 2024-09-14T05:30:45.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8949000000000001e-05 2024-09-14T05:30:45.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:45.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605221+0000", 2024-09-14T05:30:45.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3260000000000002e-05 2024-09-14T05:30:45.996 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.996 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:45.997 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:45.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.997 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73806 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:45.997 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:45.997 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.412365336999997, 2024-09-14T05:30:45.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045526189000000002, 2024-09-14T05:30:45.997 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:45.997 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:45.997 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:45.997 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:45.997 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:45.998 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:45.998 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73806 2024-09-14T05:30:45.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.998 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:45.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:45.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:45.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:45.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:45.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:45.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:45.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:45.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:30:45.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:45.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559801+0000", 2024-09-14T05:30:45.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.995e-06 2024-09-14T05:30:45.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:45.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:45.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:45.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559807+0000", 2024-09-14T05:30:46.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5450000000000003e-06 2024-09-14T05:30:46.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:46.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559811+0000", 2024-09-14T05:30:46.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5090000000000004e-06 2024-09-14T05:30:46.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:46.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605237+0000", 2024-09-14T05:30:46.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045426133 2024-09-14T05:30:46.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:46.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605278+0000", 2024-09-14T05:30:46.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1084999999999997e-05 2024-09-14T05:30:46.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:46.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605325+0000", 2024-09-14T05:30:46.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6582999999999998e-05 2024-09-14T05:30:46.001 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.001 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:46.001 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.002 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73809 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2838528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:46.002 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:46.002 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.412341832000003, 2024-09-14T05:30:46.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045596062999999999, 2024-09-14T05:30:46.002 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:46.002 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:46.002 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:46.002 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:46.002 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:46.002 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:46.002 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73809 2024-09-14T05:30:46.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.002 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:46.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:46.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:46.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:46.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:46.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:46.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559823+0000", 2024-09-14T05:30:46.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.08e-07 2024-09-14T05:30:46.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:46.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559825+0000", 2024-09-14T05:30:46.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.26e-06 2024-09-14T05:30:46.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:46.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559831+0000", 2024-09-14T05:30:46.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6690000000000003e-06 2024-09-14T05:30:46.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:46.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559835+0000", 2024-09-14T05:30:46.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5120000000000002e-06 2024-09-14T05:30:46.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:46.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605342+0000", 2024-09-14T05:30:46.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045506957000000001 2024-09-14T05:30:46.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:46.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605375+0000", 2024-09-14T05:30:46.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3250999999999997e-05 2024-09-14T05:30:46.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:46.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605418+0000", 2024-09-14T05:30:46.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3006000000000001e-05 2024-09-14T05:30:46.006 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.006 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:46.006 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.006 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73822 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2807808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:46.006 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:46.006 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.412314178999999, 2024-09-14T05:30:46.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045661405000000002, 2024-09-14T05:30:46.006 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:46.006 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:46.006 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:46.007 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:46.007 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:46.007 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:46.007 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73822 2024-09-14T05:30:46.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.007 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:46.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:46.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:46.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:46.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:46.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:46.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559851+0000", 2024-09-14T05:30:46.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4000000000000002e-07 2024-09-14T05:30:46.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:46.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559853+0000", 2024-09-14T05:30:46.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5380000000000001e-06 2024-09-14T05:30:46.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:46.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559859+0000", 2024-09-14T05:30:46.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0619999999999997e-06 2024-09-14T05:30:46.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:46.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559864+0000", 2024-09-14T05:30:46.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.977e-06 2024-09-14T05:30:46.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:46.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605435+0000", 2024-09-14T05:30:46.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045570672 2024-09-14T05:30:46.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:46.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605467+0000", 2024-09-14T05:30:46.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2486e-05 2024-09-14T05:30:46.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:46.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605511+0000", 2024-09-14T05:30:46.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4129999999999999e-05 2024-09-14T05:30:46.010 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.011 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:46.011 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.011 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73825 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2840576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:46.011 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:46.011 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.412290814999999, 2024-09-14T05:30:46.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045728694, 2024-09-14T05:30:46.011 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:46.011 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:46.011 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:46.011 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:46.011 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:46.011 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:46.012 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73825 2024-09-14T05:30:46.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.012 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:46.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:46.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:46.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:46.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:46.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:46.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559874+0000", 2024-09-14T05:30:46.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5400000000000002e-07 2024-09-14T05:30:46.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:46.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559876+0000", 2024-09-14T05:30:46.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.131e-06 2024-09-14T05:30:46.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:46.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559888+0000", 2024-09-14T05:30:46.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2216e-05 2024-09-14T05:30:46.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:46.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559893+0000", 2024-09-14T05:30:46.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6539999999999998e-06 2024-09-14T05:30:46.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:46.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605526+0000", 2024-09-14T05:30:46.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045633119 2024-09-14T05:30:46.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:46.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605560+0000", 2024-09-14T05:30:46.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3825999999999998e-05 2024-09-14T05:30:46.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:46.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605602+0000", 2024-09-14T05:30:46.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2293999999999999e-05 2024-09-14T05:30:46.015 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.015 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:46.015 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.015 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74023 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2863104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:46.015 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:46.016 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.41110484, 2024-09-14T05:30:46.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044635385999999999, 2024-09-14T05:30:46.016 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:46.016 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:46.016 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:46.016 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:46.016 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:46.016 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:46.016 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74023 2024-09-14T05:30:46.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.016 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:46.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:46.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:46.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:46.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:46.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:46.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561060+0000", 2024-09-14T05:30:46.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0699999999999998e-07 2024-09-14T05:30:46.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:46.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561072+0000", 2024-09-14T05:30:46.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2261e-05 2024-09-14T05:30:46.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:46.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561079+0000", 2024-09-14T05:30:46.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8399999999999997e-06 2024-09-14T05:30:46.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:46.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561084+0000", 2024-09-14T05:30:46.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7380000000000002e-06 2024-09-14T05:30:46.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:46.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605620+0000", 2024-09-14T05:30:46.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044536146999999998 2024-09-14T05:30:46.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:46.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605653+0000", 2024-09-14T05:30:46.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2969000000000002e-05 2024-09-14T05:30:46.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:46.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605695+0000", 2024-09-14T05:30:46.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2024000000000003e-05 2024-09-14T05:30:46.019 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.020 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:46.020 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.020 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74026 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2895872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:46.020 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:46.020 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.411073377999998, 2024-09-14T05:30:46.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044692717, 2024-09-14T05:30:46.020 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:46.020 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:46.020 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:46.020 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:46.020 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:46.020 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:46.021 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74026 2024-09-14T05:30:46.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.021 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:46.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:46.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:46.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:46.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:46.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:46.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:46.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:46.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:46.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561093+0000", 2024-09-14T05:30:46.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-14T05:30:46.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:46.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561099+0000", 2024-09-14T05:30:46.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0399999999999998e-06 2024-09-14T05:30:46.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:46.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561104+0000", 2024-09-14T05:30:46.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8180000000000003e-06 2024-09-14T05:30:46.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:46.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605710+0000", 2024-09-14T05:30:46.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044605657 2024-09-14T05:30:46.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:46.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605742+0000", 2024-09-14T05:30:46.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1949000000000002e-05 2024-09-14T05:30:46.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:46.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605784+0000", 2024-09-14T05:30:46.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1884000000000001e-05 2024-09-14T05:30:46.024 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.024 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:46.024 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.024 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74039 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:46.025 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:46.025 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.410946780000003, 2024-09-14T05:30:46.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044654905000000002, 2024-09-14T05:30:46.025 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:46.025 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:46.025 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:46.025 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:46.025 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:46.025 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:46.025 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74039 2024-09-14T05:30:46.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.026 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:46.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:46.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:46.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:46.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:46.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:46.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561218+0000", 2024-09-14T05:30:46.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4000000000000001e-07 2024-09-14T05:30:46.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:46.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561221+0000", 2024-09-14T05:30:46.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9730000000000002e-06 2024-09-14T05:30:46.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:46.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561229+0000", 2024-09-14T05:30:46.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1759999999999993e-06 2024-09-14T05:30:46.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:46.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561235+0000", 2024-09-14T05:30:46.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0109999999999997e-06 2024-09-14T05:30:46.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:46.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605798+0000", 2024-09-14T05:30:46.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044563285000000001 2024-09-14T05:30:46.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:46.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605831+0000", 2024-09-14T05:30:46.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2054999999999997e-05 2024-09-14T05:30:46.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:46.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605872+0000", 2024-09-14T05:30:46.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1764999999999997e-05 2024-09-14T05:30:46.028 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.029 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:46.029 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.029 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74042 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:46.029 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:46.029 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.410898326999998, 2024-09-14T05:30:46.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044773234000000002, 2024-09-14T05:30:46.029 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:46.029 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:46.029 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:46.029 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:46.030 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:46.030 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:46.030 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74042 2024-09-14T05:30:46.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.030 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:46.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:46.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:46.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:46.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:46.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:46.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:46.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4099999999999998e-07 2024-09-14T05:30:46.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:46.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561269+0000", 2024-09-14T05:30:46.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7930000000000002e-06 2024-09-14T05:30:46.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:46.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561278+0000", 2024-09-14T05:30:46.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5150000000000006e-06 2024-09-14T05:30:46.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:46.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561284+0000", 2024-09-14T05:30:46.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2029999999999997e-06 2024-09-14T05:30:46.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:46.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605934+0000", 2024-09-14T05:30:46.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044650449000000002 2024-09-14T05:30:46.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:46.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605983+0000", 2024-09-14T05:30:46.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8943999999999998e-05 2024-09-14T05:30:46.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:46.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606039+0000", 2024-09-14T05:30:46.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5689000000000002e-05 2024-09-14T05:30:46.033 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.033 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:46.033 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.034 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74052 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:46.034 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:46.034 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.410871452000002, 2024-09-14T05:30:46.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044830474000000002, 2024-09-14T05:30:46.034 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:46.034 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:46.034 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:46.034 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:46.034 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:46.034 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:46.034 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74052 2024-09-14T05:30:46.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.035 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:46.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:46.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:46.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:46.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:46.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:46.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:46.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1099999999999996e-07 2024-09-14T05:30:46.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:46.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561295+0000", 2024-09-14T05:30:46.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7770000000000001e-06 2024-09-14T05:30:46.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:46.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561303+0000", 2024-09-14T05:30:46.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6860000000000006e-06 2024-09-14T05:30:46.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:46.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561308+0000", 2024-09-14T05:30:46.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.023e-06 2024-09-14T05:30:46.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:46.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605893+0000", 2024-09-14T05:30:46.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044585458000000001 2024-09-14T05:30:46.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:46.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605933+0000", 2024-09-14T05:30:46.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9551999999999997e-05 2024-09-14T05:30:46.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:46.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606123+0000", 2024-09-14T05:30:46.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019046699999999999 2024-09-14T05:30:46.038 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.038 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:46.038 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.038 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74054 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:46.038 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:46.038 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.410849425000002, 2024-09-14T05:30:46.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093610999999999, 2024-09-14T05:30:46.039 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:46.039 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:46.039 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:46.039 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:46.039 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:46.039 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:46.039 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74054 2024-09-14T05:30:46.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.039 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:46.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:46.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:46.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9699999999999996e-07 2024-09-14T05:30:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561317+0000", 2024-09-14T05:30:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9740000000000001e-06 2024-09-14T05:30:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561324+0000", 2024-09-14T05:30:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4590000000000003e-06 2024-09-14T05:30:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561329+0000", 2024-09-14T05:30:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5969999999999999e-06 2024-09-14T05:30:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:46.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606142+0000", 2024-09-14T05:30:46.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044812347000000002 2024-09-14T05:30:46.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:46.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606184+0000", 2024-09-14T05:30:46.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2441999999999998e-05 2024-09-14T05:30:46.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:46.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606408+0000", 2024-09-14T05:30:46.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022429500000000001 2024-09-14T05:30:46.042 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.042 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:46.043 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.043 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74056 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:46.043 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:46.043 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.410654721999997, 2024-09-14T05:30:46.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045191685000000002, 2024-09-14T05:30:46.043 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:46.043 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:46.043 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:46.043 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:46.043 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:46.043 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:46.043 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74056 2024-09-14T05:30:46.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.044 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:46.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:46.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:46.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:46.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:46.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.045 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:46.045 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561510+0000", 2024-09-14T05:30:46.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2399999999999998e-07 2024-09-14T05:30:46.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.045 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:46.045 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561513+0000", 2024-09-14T05:30:46.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7219999999999999e-06 2024-09-14T05:30:46.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.045 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:46.045 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561524+0000", 2024-09-14T05:30:46.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1357e-05 2024-09-14T05:30:46.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:46.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561530+0000", 2024-09-14T05:30:46.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2890000000000003e-06 2024-09-14T05:30:46.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:46.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606438+0000", 2024-09-14T05:30:46.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044907326999999997 2024-09-14T05:30:46.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:46.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606520+0000", 2024-09-14T05:30:46.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1924999999999999e-05 2024-09-14T05:30:46.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:46.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606701+0000", 2024-09-14T05:30:46.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00018144100000000001 2024-09-14T05:30:46.047 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.047 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:46.047 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.047 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74058 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:46.048 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:46.048 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.410176049, 2024-09-14T05:30:46.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093368000000002, 2024-09-14T05:30:46.048 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:46.048 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:46.048 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:46.048 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:46.048 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:46.048 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:46.048 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74058 2024-09-14T05:30:46.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.048 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:46.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.048 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:46.048 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:46.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:46.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:46.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:46.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561989+0000", 2024-09-14T05:30:46.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3300000000000001e-07 2024-09-14T05:30:46.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:46.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561993+0000", 2024-09-14T05:30:46.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1790000000000001e-06 2024-09-14T05:30:46.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:46.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562007+0000", 2024-09-14T05:30:46.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3434e-05 2024-09-14T05:30:46.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:46.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562014+0000", 2024-09-14T05:30:46.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.537e-06 2024-09-14T05:30:46.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:46.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606736+0000", 2024-09-14T05:30:46.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044722117999999998 2024-09-14T05:30:46.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:46.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606829+0000", 2024-09-14T05:30:46.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2410999999999993e-05 2024-09-14T05:30:46.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:46.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607082+0000", 2024-09-14T05:30:46.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025285599999999998 2024-09-14T05:30:46.052 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.052 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:46.052 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.052 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74059 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:46.052 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:46.052 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.410118058000002, 2024-09-14T05:30:46.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045479928000000003, 2024-09-14T05:30:46.052 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:46.052 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:46.053 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:46.053 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:46.053 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:46.053 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:46.053 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74059 2024-09-14T05:30:46.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.053 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:46.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.053 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:46.053 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:46.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.053 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:46.053 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:46.054 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.054 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:46.054 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562047+0000", 2024-09-14T05:30:46.054 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6899999999999997e-07 2024-09-14T05:30:46.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.054 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:46.054 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562050+0000", 2024-09-14T05:30:46.054 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.943e-06 2024-09-14T05:30:46.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.054 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:46.054 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562058+0000", 2024-09-14T05:30:46.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0120000000000005e-06 2024-09-14T05:30:46.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:46.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562063+0000", 2024-09-14T05:30:46.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:30:46.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:46.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607186+0000", 2024-09-14T05:30:46.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045122348999999999 2024-09-14T05:30:46.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:46.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607265+0000", 2024-09-14T05:30:46.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9462999999999995e-05 2024-09-14T05:30:46.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:46.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607526+0000", 2024-09-14T05:30:46.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026090400000000001 2024-09-14T05:30:46.056 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.056 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:46.056 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.056 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74060 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2850816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:46.057 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:46.057 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 56.410092681000002, 2024-09-14T05:30:46.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045200092999999997, 2024-09-14T05:30:46.057 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:46.057 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:46.057 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:46.057 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:46.057 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:46.057 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:46.057 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74060 2024-09-14T05:30:46.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.057 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:46.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:46.058 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:46.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.058 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.058 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.058 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:46.058 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:46.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.058 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.058 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.058 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:46.058 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:46.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5200000000000002e-07 2024-09-14T05:30:46.058 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.059 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:46.059 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562075+0000", 2024-09-14T05:30:46.059 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5200000000000002e-06 2024-09-14T05:30:46.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.059 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:46.059 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562082+0000", 2024-09-14T05:30:46.059 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8929999999999999e-06 2024-09-14T05:30:46.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.059 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:46.059 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562088+0000", 2024-09-14T05:30:46.059 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5570000000000003e-06 2024-09-14T05:30:46.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:46.060 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607111+0000", 2024-09-14T05:30:46.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045023464999999999 2024-09-14T05:30:46.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:46.060 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607190+0000", 2024-09-14T05:30:46.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8721999999999996e-05 2024-09-14T05:30:46.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:46.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607272+0000", 2024-09-14T05:30:46.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1483999999999994e-05 2024-09-14T05:30:46.061 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.061 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:46.061 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.061 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:136816 3.7 3:e5cc52c3:::rbd_id.image_client.0-clone31:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-14T05:30:46.061 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:30.438431+0000", 2024-09-14T05:30:46.061 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 15.533732978, 2024-09-14T05:30:46.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.96261680299999997, 2024-09-14T05:30:46.061 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:46.061 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:46.061 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:46.062 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:46.062 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:46.062 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:46.062 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 136816 2024-09-14T05:30:46.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.062 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:46.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:46.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438431+0000", 2024-09-14T05:30:46.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:46.063 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438431+0000", 2024-09-14T05:30:46.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:46.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.063 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:46.063 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438432+0000", 2024-09-14T05:30:46.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0419999999999999e-06 2024-09-14T05:30:46.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.063 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:46.063 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438437+0000", 2024-09-14T05:30:46.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5000000000000001e-06 2024-09-14T05:30:46.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:46.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438446+0000", 2024-09-14T05:30:46.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5740000000000002e-06 2024-09-14T05:30:46.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:46.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438456+0000", 2024-09-14T05:30:46.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0056e-05 2024-09-14T05:30:46.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:46.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438500+0000", 2024-09-14T05:30:46.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3738e-05 2024-09-14T05:30:46.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:30:46.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438506+0000", 2024-09-14T05:30:46.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1659999999999999e-06 2024-09-14T05:30:46.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:46.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.399222+0000", 2024-09-14T05:30:46.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.96071578999999996 2024-09-14T05:30:46.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:46.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.399381+0000", 2024-09-14T05:30:46.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000158834 2024-09-14T05:30:46.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-14T05:30:46.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.399530+0000", 2024-09-14T05:30:46.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014862400000000001 2024-09-14T05:30:46.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:30:46.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.400507+0000", 2024-09-14T05:30:46.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0009779369999999999 2024-09-14T05:30:46.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:46.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.400969+0000", 2024-09-14T05:30:46.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046119800000000001 2024-09-14T05:30:46.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.067 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:30:46.067 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.401006+0000", 2024-09-14T05:30:46.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7594e-05 2024-09-14T05:30:46.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:46.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:46.067 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:46.067 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.401048+0000", 2024-09-14T05:30:46.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1749999999999998e-05 2024-09-14T05:30:46.067 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.067 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:46.067 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.067 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:46.067 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:46.067 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:46.068 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T05:30:46.069 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:30:46.070 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T05:30:46.073 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:46.070+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:46.078 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T05:30:46.099 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:46.100 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:46.135 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:46.135 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:46.169 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:46.170 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:46.170 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:46.168+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:46.199 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:46.199 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:46.203 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:46.203 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:46.216 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:30:46.220 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:46.221 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:46.243 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:46.244 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:46.260 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:46.260 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:46.270 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:46.269+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:46.277 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:46.277 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:46.328 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:46.328 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:46.337 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:30:46.378 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:46.376+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:46.472 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:46.472+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:46.481 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:46.481 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:46.490 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:30:46.573 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:46.572+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:46.594 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:46.595 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:46.603 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:30:46.674 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:46.673+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:46.706 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:46.706 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:46.715 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:30:46.800 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:46.799+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:46.860 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:46.860 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:46.869 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:30:46.875 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:46.874+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:46.976 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:46.976+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:47.013 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:47.013 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:47.022 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:30:47.068 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T05:30:47.071 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-14T05:30:47.078 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:47.076+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:47.093 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T05:30:47.093 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:30:47.179 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:47.177+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:47.180 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:47.180 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:47.189 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:30:47.281 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:47.279+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:47.293 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:47.293 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:47.302 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:30:47.381 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:47.379+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:47.405 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:47.405 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:47.414 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:30:47.426 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:30:47.427 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:30:47.438 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-14T05:30:47.070478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":542936253,"num_objects":1496,"num_object_clones":363,"num_object_copies":2992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1496,"num_whiteouts":909,"num_read":178264,"num_read_kb":561933,"num_write":20095,"num_write_kb":282222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17714,"ondisk_log_size":17714,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":521840,"kb_used_data":207832,"kb_used_omap":12,"kb_used_meta":313203,"kb_avail":754452880,"statfs":{"total":773094113280,"available":772559749120,"internally_reserved":0,"allocated":212819968,"data_stored":215234354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":320720432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-290148000,"num_objects":-118,"num_object_clones":-31,"num_object_copies":-236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-118,"num_whiteouts":-11,"num_read":1577,"num_read_kb":1346,"num_write":231,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022921"},"pg_stats":[{"pgid":"3.1f","version":"97'705","reported_seq":10691,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.181096+0000","last_change":"2024-09-14T05:30:46.070790+0000","last_active":"2024-09-14T05:30:46.181096+0000","last_peered":"2024-09-14T05:30:46.181096+0000","last_clean":"2024-09-14T05:30:46.181096+0000","last_became_active":"2024-09-14T05:29:24.018892+0000","last_became_peered":"2024-09-14T05:29:24.018892+0000","last_unstale":"2024-09-14T05:30:46.181096+0000","last_undegraded":"2024-09-14T05:30:46.181096+0000","last_fullsized":"2024-09-14T05:30:46.181096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:07:32.134157+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0054684850000000004,"stat_sum":{"num_bytes":34461912,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":30,"num_read":12069,"num_read_kb":37227,"num_write":795,"num_write_kb":31144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.1e","version":"97'375","reported_seq":3118,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.071407+0000","last_change":"2024-09-14T05:30:46.071407+0000","last_active":"2024-09-14T05:30:46.071407+0000","last_peered":"2024-09-14T05:30:46.071407+0000","last_clean":"2024-09-14T05:30:46.071407+0000","last_became_active":"2024-09-14T05:29:24.014468+0000","last_became_peered":"2024-09-14T05:29:24.014468+0000","last_unstale":"2024-09-14T05:30:46.071407+0000","last_undegraded":"2024-09-14T05:30:46.071407+0000","last_fullsized":"2024-09-14T05:30:46.071407+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:01:23.775510+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0063721719999999997,"stat_sum":{"num_bytes":22845168,"num_objects":45,"num_object_clones":13,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":26,"num_read":2771,"num_read_kb":32473,"num_write":390,"num_write_kb":21093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.1d","version":"97'1016","reported_seq":9700,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.068353+0000","last_change":"2024-09-14T05:30:46.069109+0000","last_active":"2024-09-14T05:30:46.068353+0000","last_peered":"2024-09-14T05:30:46.068353+0000","last_clean":"2024-09-14T05:30:46.068353+0000","last_became_active":"2024-09-14T05:29:24.014251+0000","last_became_peered":"2024-09-14T05:29:24.014251+0000","last_unstale":"2024-09-14T05:30:46.068353+0000","last_undegraded":"2024-09-14T05:30:46.068353+0000","last_fullsized":"2024-09-14T05:30:46.068353+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:01:01.591833+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0081406800000000008,"stat_sum":{"num_bytes":33229696,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":23,"num_read":12251,"num_read_kb":23813,"num_write":1546,"num_write_kb":7077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.1c","version":"97'534","reported_seq":10220,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.183951+0000","last_change":"2024-09-14T05:30:46.066151+0000","last_active":"2024-09-14T05:30:46.183951+0000","last_peered":"2024-09-14T05:30:46.183951+0000","last_clean":"2024-09-14T05:30:46.183951+0000","last_became_active":"2024-09-14T05:29:24.012427+0000","last_became_peered":"2024-09-14T05:29:24.012427+0000","last_unstale":"2024-09-14T05:30:46.183951+0000","last_undegraded":"2024-09-14T05:30:46.183951+0000","last_fullsized":"2024-09-14T05:30:46.183951+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:46:55.814565+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0047000719999999996,"stat_sum":{"num_bytes":33076136,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":33,"num_read":10673,"num_read_kb":34957,"num_write":513,"num_write_kb":4632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.1b","version":"97'521","reported_seq":3311,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.215051+0000","last_change":"2024-09-14T05:30:46.059132+0000","last_active":"2024-09-14T05:30:46.215051+0000","last_peered":"2024-09-14T05:30:46.215051+0000","last_clean":"2024-09-14T05:30:46.215051+0000","last_became_active":"2024-09-14T05:29:24.012079+0000","last_became_peered":"2024-09-14T05:29:24.012079+0000","last_unstale":"2024-09-14T05:30:46.215051+0000","last_undegraded":"2024-09-14T05:30:46.215051+0000","last_fullsized":"2024-09-14T05:30:46.215051+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:39:10.465754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0004929,"stat_sum":{"num_bytes":24215809,"num_objects":48,"num_object_clones":12,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":31,"num_read":4071,"num_read_kb":25920,"num_write":545,"num_write_kb":21860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.1a","version":"97'967","reported_seq":6186,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.196505+0000","last_change":"2024-09-14T05:30:46.069899+0000","last_active":"2024-09-14T05:30:46.196505+0000","last_peered":"2024-09-14T05:30:46.196505+0000","last_clean":"2024-09-14T05:30:46.196505+0000","last_became_active":"2024-09-14T05:29:24.012461+0000","last_became_peered":"2024-09-14T05:29:24.012461+0000","last_unstale":"2024-09-14T05:30:46.196505+0000","last_undegraded":"2024-09-14T05:30:46.196505+0000","last_fullsized":"2024-09-14T05:30:46.196505+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:55.157830+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0081011150000000007,"stat_sum":{"num_bytes":5611490,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":30,"num_read":10379,"num_read_kb":34350,"num_write":1259,"num_write_kb":8459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.19","version":"97'543","reported_seq":2046,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.169525+0000","last_change":"2024-09-14T05:30:46.068521+0000","last_active":"2024-09-14T05:30:46.169525+0000","last_peered":"2024-09-14T05:30:46.169525+0000","last_clean":"2024-09-14T05:30:46.169525+0000","last_became_active":"2024-09-14T05:29:24.018753+0000","last_became_peered":"2024-09-14T05:29:24.018753+0000","last_unstale":"2024-09-14T05:30:46.169525+0000","last_undegraded":"2024-09-14T05:30:46.169525+0000","last_fullsized":"2024-09-14T05:30:46.169525+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:47:08.137099+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0098771780000000003,"stat_sum":{"num_bytes":6018322,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":26,"num_read":2571,"num_read_kb":13570,"num_write":671,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.18","version":"97'643","reported_seq":2641,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.072986+0000","last_change":"2024-09-14T05:30:46.072986+0000","last_active":"2024-09-14T05:30:46.072986+0000","last_peered":"2024-09-14T05:30:46.072986+0000","last_clean":"2024-09-14T05:30:46.072986+0000","last_became_active":"2024-09-14T05:29:24.011176+0000","last_became_peered":"2024-09-14T05:29:24.011176+0000","last_unstale":"2024-09-14T05:30:46.072986+0000","last_undegraded":"2024-09-14T05:30:46.072986+0000","last_fullsized":"2024-09-14T05:30:46.072986+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:39:09.875678+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0075706009999999997,"stat_sum":{"num_bytes":3419100,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":22,"num_read":4441,"num_read_kb":7267,"num_write":666,"num_write_kb":3416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.17","version":"94'307","reported_seq":2905,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.071470+0000","last_change":"2024-09-14T05:30:46.072148+0000","last_active":"2024-09-14T05:30:46.071470+0000","last_peered":"2024-09-14T05:30:46.071470+0000","last_clean":"2024-09-14T05:30:46.071470+0000","last_became_active":"2024-09-14T05:29:24.014005+0000","last_became_peered":"2024-09-14T05:29:24.014005+0000","last_unstale":"2024-09-14T05:30:46.071470+0000","last_undegraded":"2024-09-14T05:30:46.071470+0000","last_fullsized":"2024-09-14T05:30:46.071470+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:03:38.962187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067189659999999998,"stat_sum":{"num_bytes":6954822,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":21,"num_read":2971,"num_read_kb":11191,"num_write":291,"num_write_kb":5639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.16","version":"97'308","reported_seq":9740,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.075750+0000","last_change":"2024-09-14T05:30:46.075750+0000","last_active":"2024-09-14T05:30:46.075750+0000","last_peered":"2024-09-14T05:30:46.075750+0000","last_clean":"2024-09-14T05:30:46.075750+0000","last_became_active":"2024-09-14T05:29:24.013434+0000","last_became_peered":"2024-09-14T05:29:24.013434+0000","last_unstale":"2024-09-14T05:30:46.075750+0000","last_undegraded":"2024-09-14T05:30:46.075750+0000","last_fullsized":"2024-09-14T05:30:46.075750+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:48:57.711176+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0052919129999999997,"stat_sum":{"num_bytes":18095362,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":35,"num_read":9226,"num_read_kb":22920,"num_write":292,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.15","version":"93'409","reported_seq":2535,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T05:30:42.112647+0000","last_change":"2024-09-14T05:30:42.090899+0000","last_active":"2024-09-14T05:30:42.112647+0000","last_peered":"2024-09-14T05:30:42.112647+0000","last_clean":"2024-09-14T05:30:42.112647+0000","last_became_active":"2024-09-14T05:29:24.014457+0000","last_became_peered":"2024-09-14T05:29:24.014457+0000","last_unstale":"2024-09-14T05:30:42.112647+0000","last_undegraded":"2024-09-14T05:30:42.112647+0000","last_fullsized":"2024-09-14T05:30:42.112647+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:43:23.516187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001222351,"stat_sum":{"num_bytes":17723019,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":23,"num_read":2602,"num_read_kb":9344,"num_write":409,"num_write_kb":12875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.14","version":"96'790","reported_seq":2309,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061838+0000","last_change":"2024-09-14T05:30:46.062287+0000","last_active":"2024-09-14T05:30:46.061838+0000","last_peered":"2024-09-14T05:30:46.061838+0000","last_clean":"2024-09-14T05:30:46.061838+0000","last_became_active":"2024-09-14T05:29:24.013266+0000","last_became_peered":"2024-09-14T05:29:24.013266+0000","last_unstale":"2024-09-14T05:30:46.061838+0000","last_undegraded":"2024-09-14T05:30:46.061838+0000","last_fullsized":"2024-09-14T05:30:46.061838+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:47:09.025200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018018629999999999,"stat_sum":{"num_bytes":12290463,"num_objects":46,"num_object_clones":7,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":32,"num_read":4166,"num_read_kb":5461,"num_write":1039,"num_write_kb":4015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.7","version":"0'0","reported_seq":170,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.060889+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:30:46.060889+0000","last_peered":"2024-09-14T05:30:46.060889+0000","last_clean":"2024-09-14T05:30:46.060889+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:30:46.060889+0000","last_undegraded":"2024-09-14T05:30:46.060889+0000","last_fullsized":"2024-09-14T05:30:46.060889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"97'318","reported_seq":2032,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.067950+0000","last_change":"2024-09-14T05:30:46.067950+0000","last_active":"2024-09-14T05:30:46.067950+0000","last_peered":"2024-09-14T05:30:46.067950+0000","last_clean":"2024-09-14T05:30:46.067950+0000","last_became_active":"2024-09-14T05:29:24.012099+0000","last_became_peered":"2024-09-14T05:29:24.012099+0000","last_unstale":"2024-09-14T05:30:46.067950+0000","last_undegraded":"2024-09-14T05:30:46.067950+0000","last_fullsized":"2024-09-14T05:30:46.067950+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:09:28.237972+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0092655359999999996,"stat_sum":{"num_bytes":2982656,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":22,"num_read":1942,"num_read_kb":3550,"num_write":265,"num_write_kb":5452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.058905+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:30:46.058905+0000","last_peered":"2024-09-14T05:30:46.058905+0000","last_clean":"2024-09-14T05:30:46.058905+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:30:46.058905+0000","last_undegraded":"2024-09-14T05:30:46.058905+0000","last_fullsized":"2024-09-14T05:30:46.058905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"96'865","reported_seq":3643,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.071611+0000","last_change":"2024-09-14T05:30:46.072251+0000","last_active":"2024-09-14T05:30:46.071611+0000","last_peered":"2024-09-14T05:30:46.071611+0000","last_clean":"2024-09-14T05:30:46.071611+0000","last_became_active":"2024-09-14T05:30:31.398064+0000","last_became_peered":"2024-09-14T05:30:31.398064+0000","last_unstale":"2024-09-14T05:30:46.071611+0000","last_undegraded":"2024-09-14T05:30:46.071611+0000","last_fullsized":"2024-09-14T05:30:46.071611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:09:33.638434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001881445,"stat_sum":{"num_bytes":30308876,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":27,"num_read":6380,"num_read_kb":8075,"num_write":1040,"num_write_kb":5150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":648894,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.5","version":"0'0","reported_seq":170,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061062+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:30:46.061062+0000","last_peered":"2024-09-14T05:30:46.061062+0000","last_clean":"2024-09-14T05:30:46.061062+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:30:46.061062+0000","last_undegraded":"2024-09-14T05:30:46.061062+0000","last_fullsized":"2024-09-14T05:30:46.061062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"94'537","reported_seq":2574,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.065100+0000","last_change":"2024-09-14T05:30:46.065871+0000","last_active":"2024-09-14T05:30:46.065100+0000","last_peered":"2024-09-14T05:30:46.065100+0000","last_clean":"2024-09-14T05:30:46.065100+0000","last_became_active":"2024-09-14T05:29:40.095441+0000","last_became_peered":"2024-09-14T05:29:40.095441+0000","last_unstale":"2024-09-14T05:30:46.065100+0000","last_undegraded":"2024-09-14T05:30:46.065100+0000","last_fullsized":"2024-09-14T05:30:46.065100+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:43.778442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010106340000000001,"stat_sum":{"num_bytes":7847118,"num_objects":53,"num_object_clones":13,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":34,"num_read":3487,"num_read_kb":12454,"num_write":608,"num_write_kb":11389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.4","version":"0'0","reported_seq":170,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.064808+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:30:46.064808+0000","last_peered":"2024-09-14T05:30:46.064808+0000","last_clean":"2024-09-14T05:30:46.064808+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:30:46.064808+0000","last_undegraded":"2024-09-14T05:30:46.064808+0000","last_fullsized":"2024-09-14T05:30:46.064808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"97'311","reported_seq":1755,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.073251+0000","last_change":"2024-09-14T05:30:46.073251+0000","last_active":"2024-09-14T05:30:46.073251+0000","last_peered":"2024-09-14T05:30:46.073251+0000","last_clean":"2024-09-14T05:30:46.073251+0000","last_became_active":"2024-09-14T05:29:24.016940+0000","last_became_peered":"2024-09-14T05:29:24.016940+0000","last_unstale":"2024-09-14T05:30:46.073251+0000","last_undegraded":"2024-09-14T05:30:46.073251+0000","last_fullsized":"2024-09-14T05:30:46.073251+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:03:23.026285+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0036575190000000001,"stat_sum":{"num_bytes":10220890,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":25,"num_read":1560,"num_read_kb":4382,"num_write":313,"num_write_kb":3996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.3","version":"93'512","reported_seq":2633,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T05:30:42.127680+0000","last_change":"2024-09-14T05:30:42.090823+0000","last_active":"2024-09-14T05:30:42.127680+0000","last_peered":"2024-09-14T05:30:42.127680+0000","last_clean":"2024-09-14T05:30:42.127680+0000","last_became_active":"2024-09-14T05:29:24.015905+0000","last_became_peered":"2024-09-14T05:29:24.015905+0000","last_unstale":"2024-09-14T05:30:42.127680+0000","last_undegraded":"2024-09-14T05:30:42.127680+0000","last_fullsized":"2024-09-14T05:30:42.127680+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:28:16.963101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262345,"stat_sum":{"num_bytes":12367677,"num_objects":68,"num_object_clones":20,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":41,"num_read":2406,"num_read_kb":9583,"num_write":547,"num_write_kb":7709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"2.2","version":"22'2","reported_seq":172,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.060834+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:30:46.060834+0000","last_peered":"2024-09-14T05:30:46.060834+0000","last_clean":"2024-09-14T05:30:46.060834+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:30:46.060834+0000","last_undegraded":"2024-09-14T05:30:46.060834+0000","last_fullsized":"2024-09-14T05:30:46.060834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"97'418","reported_seq":10658,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.215588+0000","last_change":"2024-09-14T05:30:46.067741+0000","last_active":"2024-09-14T05:30:46.215588+0000","last_peered":"2024-09-14T05:30:46.215588+0000","last_clean":"2024-09-14T05:30:46.215588+0000","last_became_active":"2024-09-14T05:29:24.011764+0000","last_became_peered":"2024-09-14T05:29:24.011764+0000","last_unstale":"2024-09-14T05:30:46.215588+0000","last_undegraded":"2024-09-14T05:30:46.215588+0000","last_fullsized":"2024-09-14T05:30:46.215588+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:52:30.014697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002615062,"stat_sum":{"num_bytes":15504609,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":29,"num_read":11558,"num_read_kb":38786,"num_write":374,"num_write_kb":4949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.1","version":"0'0","reported_seq":170,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.060986+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:30:46.060986+0000","last_peered":"2024-09-14T05:30:46.060986+0000","last_clean":"2024-09-14T05:30:46.060986+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:30:46.060986+0000","last_undegraded":"2024-09-14T05:30:46.060986+0000","last_fullsized":"2024-09-14T05:30:46.060986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"96'677","reported_seq":1849,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.075537+0000","last_change":"2024-09-14T05:30:46.075965+0000","last_active":"2024-09-14T05:30:46.075537+0000","last_peered":"2024-09-14T05:30:46.075537+0000","last_clean":"2024-09-14T05:30:46.075537+0000","last_became_active":"2024-09-14T05:29:40.095255+0000","last_became_peered":"2024-09-14T05:29:40.095255+0000","last_unstale":"2024-09-14T05:30:46.075537+0000","last_undegraded":"2024-09-14T05:30:46.075537+0000","last_fullsized":"2024-09-14T05:30:46.075537+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:48:24.243895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046994810000000001,"stat_sum":{"num_bytes":2523174,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":22,"num_read":3109,"num_read_kb":7242,"num_write":912,"num_write_kb":10028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.0","version":"0'0","reported_seq":170,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061943+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:30:46.061943+0000","last_peered":"2024-09-14T05:30:46.061943+0000","last_clean":"2024-09-14T05:30:46.061943+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:30:46.061943+0000","last_undegraded":"2024-09-14T05:30:46.061943+0000","last_fullsized":"2024-09-14T05:30:46.061943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"94'540","reported_seq":2566,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.068389+0000","last_change":"2024-09-14T05:30:46.068389+0000","last_active":"2024-09-14T05:30:46.068389+0000","last_peered":"2024-09-14T05:30:46.068389+0000","last_clean":"2024-09-14T05:30:46.068389+0000","last_became_active":"2024-09-14T05:29:24.015792+0000","last_became_peered":"2024-09-14T05:29:24.015792+0000","last_unstale":"2024-09-14T05:30:46.068389+0000","last_undegraded":"2024-09-14T05:30:46.068389+0000","last_fullsized":"2024-09-14T05:30:46.068389+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:51:18.518863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009539657,"stat_sum":{"num_bytes":5561684,"num_objects":59,"num_object_clones":14,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":39,"num_read":3491,"num_read_kb":9434,"num_write":601,"num_write_kb":5626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.3","version":"20'1","reported_seq":171,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.060656+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:30:46.060656+0000","last_peered":"2024-09-14T05:30:46.060656+0000","last_clean":"2024-09-14T05:30:46.060656+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:30:46.060656+0000","last_undegraded":"2024-09-14T05:30:46.060656+0000","last_fullsized":"2024-09-14T05:30:46.060656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":261,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061019+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:30:46.061019+0000","last_peered":"2024-09-14T05:30:46.061019+0000","last_clean":"2024-09-14T05:30:46.061019+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:30:46.061019+0000","last_undegraded":"2024-09-14T05:30:46.061019+0000","last_fullsized":"2024-09-14T05:30:46.061019+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"97'370","reported_seq":2110,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.064083+0000","last_change":"2024-09-14T05:30:46.064083+0000","last_active":"2024-09-14T05:30:46.064083+0000","last_peered":"2024-09-14T05:30:46.064083+0000","last_clean":"2024-09-14T05:30:46.064083+0000","last_became_active":"2024-09-14T05:29:24.012222+0000","last_became_peered":"2024-09-14T05:29:24.012222+0000","last_unstale":"2024-09-14T05:30:46.064083+0000","last_undegraded":"2024-09-14T05:30:46.064083+0000","last_fullsized":"2024-09-14T05:30:46.064083+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:00:18.099857+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0054397630000000002,"stat_sum":{"num_bytes":11525349,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":24,"num_read":2826,"num_read_kb":6026,"num_write":320,"num_write_kb":3878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.9","version":"93'583","reported_seq":2258,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.069093+0000","last_change":"2024-09-14T05:30:46.069473+0000","last_active":"2024-09-14T05:30:46.069093+0000","last_peered":"2024-09-14T05:30:46.069093+0000","last_clean":"2024-09-14T05:30:46.069093+0000","last_became_active":"2024-09-14T05:29:24.012041+0000","last_became_peered":"2024-09-14T05:29:24.012041+0000","last_unstale":"2024-09-14T05:30:46.069093+0000","last_undegraded":"2024-09-14T05:30:46.069093+0000","last_fullsized":"2024-09-14T05:30:46.069093+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:03:27.411006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073498620000000004,"stat_sum":{"num_bytes":9771921,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":21,"num_read":2883,"num_read_kb":7330,"num_write":734,"num_write_kb":6418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.a","version":"97'589","reported_seq":2461,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.063548+0000","last_change":"2024-09-14T05:30:46.063548+0000","last_active":"2024-09-14T05:30:46.063548+0000","last_peered":"2024-09-14T05:30:46.063548+0000","last_clean":"2024-09-14T05:30:46.063548+0000","last_became_active":"2024-09-14T05:29:24.011403+0000","last_became_peered":"2024-09-14T05:29:24.011403+0000","last_unstale":"2024-09-14T05:30:46.063548+0000","last_undegraded":"2024-09-14T05:30:46.063548+0000","last_fullsized":"2024-09-14T05:30:46.063548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:31:01.164650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030322140000000001,"stat_sum":{"num_bytes":13791126,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":26,"num_read":3872,"num_read_kb":7972,"num_write":669,"num_write_kb":4971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.b","version":"94'516","reported_seq":9691,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061328+0000","last_change":"2024-09-14T05:30:46.061500+0000","last_active":"2024-09-14T05:30:46.061328+0000","last_peered":"2024-09-14T05:30:46.061328+0000","last_clean":"2024-09-14T05:30:46.061328+0000","last_became_active":"2024-09-14T05:29:24.006836+0000","last_became_peered":"2024-09-14T05:29:24.006836+0000","last_unstale":"2024-09-14T05:30:46.061328+0000","last_undegraded":"2024-09-14T05:30:46.061328+0000","last_fullsized":"2024-09-14T05:30:46.061328+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:35:39.708618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037603300000000001,"stat_sum":{"num_bytes":30696360,"num_objects":54,"num_object_clones":7,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":32,"num_read":11264,"num_read_kb":29169,"num_write":600,"num_write_kb":13044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.c","version":"92'420","reported_seq":3343,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T05:30:42.089804+0000","last_change":"2024-09-14T05:30:42.090049+0000","last_active":"2024-09-14T05:30:42.089804+0000","last_peered":"2024-09-14T05:30:42.089804+0000","last_clean":"2024-09-14T05:30:42.089804+0000","last_became_active":"2024-09-14T05:29:24.012574+0000","last_became_peered":"2024-09-14T05:29:24.012574+0000","last_unstale":"2024-09-14T05:30:42.089804+0000","last_undegraded":"2024-09-14T05:30:42.089804+0000","last_fullsized":"2024-09-14T05:30:42.089804+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:59:10.407456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049961899999999997,"stat_sum":{"num_bytes":10692287,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":25,"num_read":2807,"num_read_kb":13312,"num_write":357,"num_write_kb":9799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.d","version":"97'1100","reported_seq":11209,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.069371+0000","last_change":"2024-09-14T05:30:46.069601+0000","last_active":"2024-09-14T05:30:46.069371+0000","last_peered":"2024-09-14T05:30:46.069371+0000","last_clean":"2024-09-14T05:30:46.069371+0000","last_became_active":"2024-09-14T05:29:24.012401+0000","last_became_peered":"2024-09-14T05:29:24.012401+0000","last_unstale":"2024-09-14T05:30:46.069371+0000","last_undegraded":"2024-09-14T05:30:46.069371+0000","last_fullsized":"2024-09-14T05:30:46.069371+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1100,"log_dups_size":0,"ondisk_log_size":1100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:33:10.493428+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0085316369999999999,"stat_sum":{"num_bytes":39708826,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":34,"num_read":14901,"num_read_kb":22340,"num_write":1329,"num_write_kb":6495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.e","version":"97'333","reported_seq":2683,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.075233+0000","last_change":"2024-09-14T05:30:46.075460+0000","last_active":"2024-09-14T05:30:46.075233+0000","last_peered":"2024-09-14T05:30:46.075233+0000","last_clean":"2024-09-14T05:30:46.075233+0000","last_became_active":"2024-09-14T05:29:24.015315+0000","last_became_peered":"2024-09-14T05:29:24.015315+0000","last_unstale":"2024-09-14T05:30:46.075233+0000","last_undegraded":"2024-09-14T05:30:46.075233+0000","last_fullsized":"2024-09-14T05:30:46.075233+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:01:11.642896+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0047092430000000001,"stat_sum":{"num_bytes":9005641,"num_objects":53,"num_object_clones":17,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":36,"num_read":2150,"num_read_kb":8871,"num_write":290,"num_write_kb":3939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.f","version":"87'458","reported_seq":11513,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T05:30:42.089771+0000","last_change":"2024-09-14T05:30:42.089969+0000","last_active":"2024-09-14T05:30:42.089771+0000","last_peered":"2024-09-14T05:30:42.089771+0000","last_clean":"2024-09-14T05:30:42.089771+0000","last_became_active":"2024-09-14T05:29:24.018854+0000","last_became_peered":"2024-09-14T05:29:24.018854+0000","last_unstale":"2024-09-14T05:30:42.089771+0000","last_undegraded":"2024-09-14T05:30:42.089771+0000","last_fullsized":"2024-09-14T05:30:42.089771+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:19:02.691236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043059000000000001,"stat_sum":{"num_bytes":24608807,"num_objects":75,"num_object_clones":20,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":43,"num_read":11854,"num_read_kb":31740,"num_write":408,"num_write_kb":11837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.10","version":"94'358","reported_seq":2863,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061135+0000","last_change":"2024-09-14T05:30:46.061628+0000","last_active":"2024-09-14T05:30:46.061135+0000","last_peered":"2024-09-14T05:30:46.061135+0000","last_clean":"2024-09-14T05:30:46.061135+0000","last_became_active":"2024-09-14T05:29:24.010897+0000","last_became_peered":"2024-09-14T05:29:24.010897+0000","last_unstale":"2024-09-14T05:30:46.061135+0000","last_undegraded":"2024-09-14T05:30:46.061135+0000","last_fullsized":"2024-09-14T05:30:46.061135+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:19:57.056458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078010799999999995,"stat_sum":{"num_bytes":30209040,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":26,"num_read":2374,"num_read_kb":22456,"num_write":331,"num_write_kb":8939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.11","version":"96'531","reported_seq":3760,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.196994+0000","last_change":"2024-09-14T05:30:46.062429+0000","last_active":"2024-09-14T05:30:46.196994+0000","last_peered":"2024-09-14T05:30:46.196994+0000","last_clean":"2024-09-14T05:30:46.196994+0000","last_became_active":"2024-09-14T05:29:24.013792+0000","last_became_peered":"2024-09-14T05:29:24.013792+0000","last_unstale":"2024-09-14T05:30:46.196994+0000","last_undegraded":"2024-09-14T05:30:46.196994+0000","last_fullsized":"2024-09-14T05:30:46.196994+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:21.993766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001953354,"stat_sum":{"num_bytes":11256390,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":26,"num_read":4987,"num_read_kb":24302,"num_write":504,"num_write_kb":3330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.12","version":"96'448","reported_seq":2349,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061386+0000","last_change":"2024-09-14T05:30:46.061694+0000","last_active":"2024-09-14T05:30:46.061386+0000","last_peered":"2024-09-14T05:30:46.061386+0000","last_clean":"2024-09-14T05:30:46.061386+0000","last_became_active":"2024-09-14T05:30:38.013627+0000","last_became_peered":"2024-09-14T05:30:38.013627+0000","last_unstale":"2024-09-14T05:30:46.061386+0000","last_undegraded":"2024-09-14T05:30:46.061386+0000","last_fullsized":"2024-09-14T05:30:46.061386+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:26:00.217325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055527400000000002,"stat_sum":{"num_bytes":12733139,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":20,"num_read":2778,"num_read_kb":16473,"num_write":481,"num_write_kb":10860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2769134,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.13","version":"97'637","reported_seq":2287,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.212957+0000","last_change":"2024-09-14T05:30:46.062888+0000","last_active":"2024-09-14T05:30:46.212957+0000","last_peered":"2024-09-14T05:30:46.212957+0000","last_clean":"2024-09-14T05:30:46.212957+0000","last_became_active":"2024-09-14T05:29:24.014371+0000","last_became_peered":"2024-09-14T05:29:24.014371+0000","last_unstale":"2024-09-14T05:30:46.212957+0000","last_undegraded":"2024-09-14T05:30:46.212957+0000","last_fullsized":"2024-09-14T05:30:46.212957+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:08:22.644295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023503259999999998,"stat_sum":{"num_bytes":33088997,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":25,"num_read":3378,"num_read_kb":19888,"num_write":893,"num_write_kb":5499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":542345866,"num_objects":1492,"num_object_clones":363,"num_object_copies":2984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1492,"num_whiteouts":909,"num_read":178198,"num_read_kb":561878,"num_write":19992,"num_write_kb":280592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200396800,"data_stored":205772395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17639,"ondisk_log_size":17639,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476759,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91108,"kb_used_data":41856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94280732,"statfs":{"total":96636764160,"available":96543469568,"internally_reserved":0,"allocated":42860544,"data_stored":43268435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52480,"kb_used_data":17124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94319360,"statfs":{"total":96636764160,"available":96583024640,"internally_reserved":0,"allocated":17534976,"data_stored":17583619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509464,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44360,"kb_used_data":17328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94327480,"statfs":{"total":96636764160,"available":96591339520,"internally_reserved":0,"allocated":17743872,"data_stored":18412012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66492,"kb_used_data":39464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94305348,"statfs":{"total":96636764160,"available":96568676352,"internally_reserved":0,"allocated":40411136,"data_stored":41394432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49728,"kb_used_data":22408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94322112,"statfs":{"total":96636764160,"available":96585842688,"internally_reserved":0,"allocated":22945792,"data_stored":22996796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85912,"kb_used_data":14456,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94285928,"statfs":{"total":96636764160,"available":96548790272,"internally_reserved":0,"allocated":14802944,"data_stored":14771019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47744,"kb_used_data":20424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94324096,"statfs":{"total":96636764160,"available":96587874304,"internally_reserved":0,"allocated":20914176,"data_stored":21144190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84016,"kb_used_data":34772,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94287824,"statfs":{"total":96636764160,"available":96550731776,"internally_reserved":0,"allocated":35606528,"data_stored":35663851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":41766912,"data_stored":42758913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33869824,"data_stored":34538733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19193856,"data_stored":19534777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13656064,"data_stored":14236250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":21286912,"data_stored":21499204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":38166528,"data_stored":39272128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16224256,"data_stored":17175056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16232448,"data_stored":16757334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:30:47.440 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:30:47.482 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:47.481+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:47.517 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:47.517 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:47.525 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:30:47.582 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:47.581+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:47.669 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:47.669 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:47.677 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:30:47.684 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:47.682+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:47.759 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:30:47.760 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:30:47.771 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-14T05:30:47.070478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":542936253,"num_objects":1496,"num_object_clones":363,"num_object_copies":2992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1496,"num_whiteouts":909,"num_read":178264,"num_read_kb":561933,"num_write":20095,"num_write_kb":282222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17714,"ondisk_log_size":17714,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":521840,"kb_used_data":207832,"kb_used_omap":12,"kb_used_meta":313203,"kb_avail":754452880,"statfs":{"total":773094113280,"available":772559749120,"internally_reserved":0,"allocated":212819968,"data_stored":215234354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":320720432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-290148000,"num_objects":-118,"num_object_clones":-31,"num_object_copies":-236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-118,"num_whiteouts":-11,"num_read":1577,"num_read_kb":1346,"num_write":231,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022921"},"pg_stats":[{"pgid":"3.1f","version":"97'705","reported_seq":10691,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.181096+0000","last_change":"2024-09-14T05:30:46.070790+0000","last_active":"2024-09-14T05:30:46.181096+0000","last_peered":"2024-09-14T05:30:46.181096+0000","last_clean":"2024-09-14T05:30:46.181096+0000","last_became_active":"2024-09-14T05:29:24.018892+0000","last_became_peered":"2024-09-14T05:29:24.018892+0000","last_unstale":"2024-09-14T05:30:46.181096+0000","last_undegraded":"2024-09-14T05:30:46.181096+0000","last_fullsized":"2024-09-14T05:30:46.181096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:07:32.134157+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0054684850000000004,"stat_sum":{"num_bytes":34461912,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":30,"num_read":12069,"num_read_kb":37227,"num_write":795,"num_write_kb":31144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.1e","version":"97'375","reported_seq":3118,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.071407+0000","last_change":"2024-09-14T05:30:46.071407+0000","last_active":"2024-09-14T05:30:46.071407+0000","last_peered":"2024-09-14T05:30:46.071407+0000","last_clean":"2024-09-14T05:30:46.071407+0000","last_became_active":"2024-09-14T05:29:24.014468+0000","last_became_peered":"2024-09-14T05:29:24.014468+0000","last_unstale":"2024-09-14T05:30:46.071407+0000","last_undegraded":"2024-09-14T05:30:46.071407+0000","last_fullsized":"2024-09-14T05:30:46.071407+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:01:23.775510+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0063721719999999997,"stat_sum":{"num_bytes":22845168,"num_objects":45,"num_object_clones":13,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":26,"num_read":2771,"num_read_kb":32473,"num_write":390,"num_write_kb":21093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.1d","version":"97'1016","reported_seq":9700,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.068353+0000","last_change":"2024-09-14T05:30:46.069109+0000","last_active":"2024-09-14T05:30:46.068353+0000","last_peered":"2024-09-14T05:30:46.068353+0000","last_clean":"2024-09-14T05:30:46.068353+0000","last_became_active":"2024-09-14T05:29:24.014251+0000","last_became_peered":"2024-09-14T05:29:24.014251+0000","last_unstale":"2024-09-14T05:30:46.068353+0000","last_undegraded":"2024-09-14T05:30:46.068353+0000","last_fullsized":"2024-09-14T05:30:46.068353+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:01:01.591833+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0081406800000000008,"stat_sum":{"num_bytes":33229696,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":23,"num_read":12251,"num_read_kb":23813,"num_write":1546,"num_write_kb":7077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.1c","version":"97'534","reported_seq":10220,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.183951+0000","last_change":"2024-09-14T05:30:46.066151+0000","last_active":"2024-09-14T05:30:46.183951+0000","last_peered":"2024-09-14T05:30:46.183951+0000","last_clean":"2024-09-14T05:30:46.183951+0000","last_became_active":"2024-09-14T05:29:24.012427+0000","last_became_peered":"2024-09-14T05:29:24.012427+0000","last_unstale":"2024-09-14T05:30:46.183951+0000","last_undegraded":"2024-09-14T05:30:46.183951+0000","last_fullsized":"2024-09-14T05:30:46.183951+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:46:55.814565+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0047000719999999996,"stat_sum":{"num_bytes":33076136,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":33,"num_read":10673,"num_read_kb":34957,"num_write":513,"num_write_kb":4632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.1b","version":"97'521","reported_seq":3311,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.215051+0000","last_change":"2024-09-14T05:30:46.059132+0000","last_active":"2024-09-14T05:30:46.215051+0000","last_peered":"2024-09-14T05:30:46.215051+0000","last_clean":"2024-09-14T05:30:46.215051+0000","last_became_active":"2024-09-14T05:29:24.012079+0000","last_became_peered":"2024-09-14T05:29:24.012079+0000","last_unstale":"2024-09-14T05:30:46.215051+0000","last_undegraded":"2024-09-14T05:30:46.215051+0000","last_fullsized":"2024-09-14T05:30:46.215051+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:39:10.465754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0004929,"stat_sum":{"num_bytes":24215809,"num_objects":48,"num_object_clones":12,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":31,"num_read":4071,"num_read_kb":25920,"num_write":545,"num_write_kb":21860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.1a","version":"97'967","reported_seq":6186,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.196505+0000","last_change":"2024-09-14T05:30:46.069899+0000","last_active":"2024-09-14T05:30:46.196505+0000","last_peered":"2024-09-14T05:30:46.196505+0000","last_clean":"2024-09-14T05:30:46.196505+0000","last_became_active":"2024-09-14T05:29:24.012461+0000","last_became_peered":"2024-09-14T05:29:24.012461+0000","last_unstale":"2024-09-14T05:30:46.196505+0000","last_undegraded":"2024-09-14T05:30:46.196505+0000","last_fullsized":"2024-09-14T05:30:46.196505+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:55.157830+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0081011150000000007,"stat_sum":{"num_bytes":5611490,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":30,"num_read":10379,"num_read_kb":34350,"num_write":1259,"num_write_kb":8459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.19","version":"97'543","reported_seq":2046,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.169525+0000","last_change":"2024-09-14T05:30:46.068521+0000","last_active":"2024-09-14T05:30:46.169525+0000","last_peered":"2024-09-14T05:30:46.169525+0000","last_clean":"2024-09-14T05:30:46.169525+0000","last_became_active":"2024-09-14T05:29:24.018753+0000","last_became_peered":"2024-09-14T05:29:24.018753+0000","last_unstale":"2024-09-14T05:30:46.169525+0000","last_undegraded":"2024-09-14T05:30:46.169525+0000","last_fullsized":"2024-09-14T05:30:46.169525+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:47:08.137099+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0098771780000000003,"stat_sum":{"num_bytes":6018322,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":26,"num_read":2571,"num_read_kb":13570,"num_write":671,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.18","version":"97'643","reported_seq":2641,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.072986+0000","last_change":"2024-09-14T05:30:46.072986+0000","last_active":"2024-09-14T05:30:46.072986+0000","last_peered":"2024-09-14T05:30:46.072986+0000","last_clean":"2024-09-14T05:30:46.072986+0000","last_became_active":"2024-09-14T05:29:24.011176+0000","last_became_peered":"2024-09-14T05:29:24.011176+0000","last_unstale":"2024-09-14T05:30:46.072986+0000","last_undegraded":"2024-09-14T05:30:46.072986+0000","last_fullsized":"2024-09-14T05:30:46.072986+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:39:09.875678+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0075706009999999997,"stat_sum":{"num_bytes":3419100,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":22,"num_read":4441,"num_read_kb":7267,"num_write":666,"num_write_kb":3416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.17","version":"94'307","reported_seq":2905,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.071470+0000","last_change":"2024-09-14T05:30:46.072148+0000","last_active":"2024-09-14T05:30:46.071470+0000","last_peered":"2024-09-14T05:30:46.071470+0000","last_clean":"2024-09-14T05:30:46.071470+0000","last_became_active":"2024-09-14T05:29:24.014005+0000","last_became_peered":"2024-09-14T05:29:24.014005+0000","last_unstale":"2024-09-14T05:30:46.071470+0000","last_undegraded":"2024-09-14T05:30:46.071470+0000","last_fullsized":"2024-09-14T05:30:46.071470+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:03:38.962187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067189659999999998,"stat_sum":{"num_bytes":6954822,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":21,"num_read":2971,"num_read_kb":11191,"num_write":291,"num_write_kb":5639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.16","version":"97'308","reported_seq":9740,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.075750+0000","last_change":"2024-09-14T05:30:46.075750+0000","last_active":"2024-09-14T05:30:46.075750+0000","last_peered":"2024-09-14T05:30:46.075750+0000","last_clean":"2024-09-14T05:30:46.075750+0000","last_became_active":"2024-09-14T05:29:24.013434+0000","last_became_peered":"2024-09-14T05:29:24.013434+0000","last_unstale":"2024-09-14T05:30:46.075750+0000","last_undegraded":"2024-09-14T05:30:46.075750+0000","last_fullsized":"2024-09-14T05:30:46.075750+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:48:57.711176+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0052919129999999997,"stat_sum":{"num_bytes":18095362,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":35,"num_read":9226,"num_read_kb":22920,"num_write":292,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.15","version":"93'409","reported_seq":2535,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T05:30:42.112647+0000","last_change":"2024-09-14T05:30:42.090899+0000","last_active":"2024-09-14T05:30:42.112647+0000","last_peered":"2024-09-14T05:30:42.112647+0000","last_clean":"2024-09-14T05:30:42.112647+0000","last_became_active":"2024-09-14T05:29:24.014457+0000","last_became_peered":"2024-09-14T05:29:24.014457+0000","last_unstale":"2024-09-14T05:30:42.112647+0000","last_undegraded":"2024-09-14T05:30:42.112647+0000","last_fullsized":"2024-09-14T05:30:42.112647+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:43:23.516187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001222351,"stat_sum":{"num_bytes":17723019,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":23,"num_read":2602,"num_read_kb":9344,"num_write":409,"num_write_kb":12875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.14","version":"96'790","reported_seq":2309,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061838+0000","last_change":"2024-09-14T05:30:46.062287+0000","last_active":"2024-09-14T05:30:46.061838+0000","last_peered":"2024-09-14T05:30:46.061838+0000","last_clean":"2024-09-14T05:30:46.061838+0000","last_became_active":"2024-09-14T05:29:24.013266+0000","last_became_peered":"2024-09-14T05:29:24.013266+0000","last_unstale":"2024-09-14T05:30:46.061838+0000","last_undegraded":"2024-09-14T05:30:46.061838+0000","last_fullsized":"2024-09-14T05:30:46.061838+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:47:09.025200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018018629999999999,"stat_sum":{"num_bytes":12290463,"num_objects":46,"num_object_clones":7,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":32,"num_read":4166,"num_read_kb":5461,"num_write":1039,"num_write_kb":4015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.7","version":"0'0","reported_seq":170,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.060889+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:30:46.060889+0000","last_peered":"2024-09-14T05:30:46.060889+0000","last_clean":"2024-09-14T05:30:46.060889+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:30:46.060889+0000","last_undegraded":"2024-09-14T05:30:46.060889+0000","last_fullsized":"2024-09-14T05:30:46.060889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"97'318","reported_seq":2032,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.067950+0000","last_change":"2024-09-14T05:30:46.067950+0000","last_active":"2024-09-14T05:30:46.067950+0000","last_peered":"2024-09-14T05:30:46.067950+0000","last_clean":"2024-09-14T05:30:46.067950+0000","last_became_active":"2024-09-14T05:29:24.012099+0000","last_became_peered":"2024-09-14T05:29:24.012099+0000","last_unstale":"2024-09-14T05:30:46.067950+0000","last_undegraded":"2024-09-14T05:30:46.067950+0000","last_fullsized":"2024-09-14T05:30:46.067950+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:09:28.237972+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0092655359999999996,"stat_sum":{"num_bytes":2982656,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":22,"num_read":1942,"num_read_kb":3550,"num_write":265,"num_write_kb":5452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.058905+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:30:46.058905+0000","last_peered":"2024-09-14T05:30:46.058905+0000","last_clean":"2024-09-14T05:30:46.058905+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:30:46.058905+0000","last_undegraded":"2024-09-14T05:30:46.058905+0000","last_fullsized":"2024-09-14T05:30:46.058905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"96'865","reported_seq":3643,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.071611+0000","last_change":"2024-09-14T05:30:46.072251+0000","last_active":"2024-09-14T05:30:46.071611+0000","last_peered":"2024-09-14T05:30:46.071611+0000","last_clean":"2024-09-14T05:30:46.071611+0000","last_became_active":"2024-09-14T05:30:31.398064+0000","last_became_peered":"2024-09-14T05:30:31.398064+0000","last_unstale":"2024-09-14T05:30:46.071611+0000","last_undegraded":"2024-09-14T05:30:46.071611+0000","last_fullsized":"2024-09-14T05:30:46.071611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:09:33.638434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001881445,"stat_sum":{"num_bytes":30308876,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":27,"num_read":6380,"num_read_kb":8075,"num_write":1040,"num_write_kb":5150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":648894,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.5","version":"0'0","reported_seq":170,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061062+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:30:46.061062+0000","last_peered":"2024-09-14T05:30:46.061062+0000","last_clean":"2024-09-14T05:30:46.061062+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:30:46.061062+0000","last_undegraded":"2024-09-14T05:30:46.061062+0000","last_fullsized":"2024-09-14T05:30:46.061062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"94'537","reported_seq":2574,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.065100+0000","last_change":"2024-09-14T05:30:46.065871+0000","last_active":"2024-09-14T05:30:46.065100+0000","last_peered":"2024-09-14T05:30:46.065100+0000","last_clean":"2024-09-14T05:30:46.065100+0000","last_became_active":"2024-09-14T05:29:40.095441+0000","last_became_peered":"2024-09-14T05:29:40.095441+0000","last_unstale":"2024-09-14T05:30:46.065100+0000","last_undegraded":"2024-09-14T05:30:46.065100+0000","last_fullsized":"2024-09-14T05:30:46.065100+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:43.778442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010106340000000001,"stat_sum":{"num_bytes":7847118,"num_objects":53,"num_object_clones":13,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":34,"num_read":3487,"num_read_kb":12454,"num_write":608,"num_write_kb":11389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.4","version":"0'0","reported_seq":170,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.064808+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:30:46.064808+0000","last_peered":"2024-09-14T05:30:46.064808+0000","last_clean":"2024-09-14T05:30:46.064808+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:30:46.064808+0000","last_undegraded":"2024-09-14T05:30:46.064808+0000","last_fullsized":"2024-09-14T05:30:46.064808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"97'311","reported_seq":1755,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.073251+0000","last_change":"2024-09-14T05:30:46.073251+0000","last_active":"2024-09-14T05:30:46.073251+0000","last_peered":"2024-09-14T05:30:46.073251+0000","last_clean":"2024-09-14T05:30:46.073251+0000","last_became_active":"2024-09-14T05:29:24.016940+0000","last_became_peered":"2024-09-14T05:29:24.016940+0000","last_unstale":"2024-09-14T05:30:46.073251+0000","last_undegraded":"2024-09-14T05:30:46.073251+0000","last_fullsized":"2024-09-14T05:30:46.073251+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:03:23.026285+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0036575190000000001,"stat_sum":{"num_bytes":10220890,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":25,"num_read":1560,"num_read_kb":4382,"num_write":313,"num_write_kb":3996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.3","version":"93'512","reported_seq":2633,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T05:30:42.127680+0000","last_change":"2024-09-14T05:30:42.090823+0000","last_active":"2024-09-14T05:30:42.127680+0000","last_peered":"2024-09-14T05:30:42.127680+0000","last_clean":"2024-09-14T05:30:42.127680+0000","last_became_active":"2024-09-14T05:29:24.015905+0000","last_became_peered":"2024-09-14T05:29:24.015905+0000","last_unstale":"2024-09-14T05:30:42.127680+0000","last_undegraded":"2024-09-14T05:30:42.127680+0000","last_fullsized":"2024-09-14T05:30:42.127680+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:28:16.963101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262345,"stat_sum":{"num_bytes":12367677,"num_objects":68,"num_object_clones":20,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":41,"num_read":2406,"num_read_kb":9583,"num_write":547,"num_write_kb":7709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"2.2","version":"22'2","reported_seq":172,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.060834+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:30:46.060834+0000","last_peered":"2024-09-14T05:30:46.060834+0000","last_clean":"2024-09-14T05:30:46.060834+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:30:46.060834+0000","last_undegraded":"2024-09-14T05:30:46.060834+0000","last_fullsized":"2024-09-14T05:30:46.060834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"97'418","reported_seq":10658,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.215588+0000","last_change":"2024-09-14T05:30:46.067741+0000","last_active":"2024-09-14T05:30:46.215588+0000","last_peered":"2024-09-14T05:30:46.215588+0000","last_clean":"2024-09-14T05:30:46.215588+0000","last_became_active":"2024-09-14T05:29:24.011764+0000","last_became_peered":"2024-09-14T05:29:24.011764+0000","last_unstale":"2024-09-14T05:30:46.215588+0000","last_undegraded":"2024-09-14T05:30:46.215588+0000","last_fullsized":"2024-09-14T05:30:46.215588+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:52:30.014697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002615062,"stat_sum":{"num_bytes":15504609,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":29,"num_read":11558,"num_read_kb":38786,"num_write":374,"num_write_kb":4949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.1","version":"0'0","reported_seq":170,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.060986+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:30:46.060986+0000","last_peered":"2024-09-14T05:30:46.060986+0000","last_clean":"2024-09-14T05:30:46.060986+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:30:46.060986+0000","last_undegraded":"2024-09-14T05:30:46.060986+0000","last_fullsized":"2024-09-14T05:30:46.060986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"96'677","reported_seq":1849,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.075537+0000","last_change":"2024-09-14T05:30:46.075965+0000","last_active":"2024-09-14T05:30:46.075537+0000","last_peered":"2024-09-14T05:30:46.075537+0000","last_clean":"2024-09-14T05:30:46.075537+0000","last_became_active":"2024-09-14T05:29:40.095255+0000","last_became_peered":"2024-09-14T05:29:40.095255+0000","last_unstale":"2024-09-14T05:30:46.075537+0000","last_undegraded":"2024-09-14T05:30:46.075537+0000","last_fullsized":"2024-09-14T05:30:46.075537+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:48:24.243895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046994810000000001,"stat_sum":{"num_bytes":2523174,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":22,"num_read":3109,"num_read_kb":7242,"num_write":912,"num_write_kb":10028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.0","version":"0'0","reported_seq":170,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061943+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:30:46.061943+0000","last_peered":"2024-09-14T05:30:46.061943+0000","last_clean":"2024-09-14T05:30:46.061943+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:30:46.061943+0000","last_undegraded":"2024-09-14T05:30:46.061943+0000","last_fullsized":"2024-09-14T05:30:46.061943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"94'540","reported_seq":2566,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.068389+0000","last_change":"2024-09-14T05:30:46.068389+0000","last_active":"2024-09-14T05:30:46.068389+0000","last_peered":"2024-09-14T05:30:46.068389+0000","last_clean":"2024-09-14T05:30:46.068389+0000","last_became_active":"2024-09-14T05:29:24.015792+0000","last_became_peered":"2024-09-14T05:29:24.015792+0000","last_unstale":"2024-09-14T05:30:46.068389+0000","last_undegraded":"2024-09-14T05:30:46.068389+0000","last_fullsized":"2024-09-14T05:30:46.068389+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:51:18.518863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009539657,"stat_sum":{"num_bytes":5561684,"num_objects":59,"num_object_clones":14,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":39,"num_read":3491,"num_read_kb":9434,"num_write":601,"num_write_kb":5626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.3","version":"20'1","reported_seq":171,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.060656+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:30:46.060656+0000","last_peered":"2024-09-14T05:30:46.060656+0000","last_clean":"2024-09-14T05:30:46.060656+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:30:46.060656+0000","last_undegraded":"2024-09-14T05:30:46.060656+0000","last_fullsized":"2024-09-14T05:30:46.060656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":261,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061019+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:30:46.061019+0000","last_peered":"2024-09-14T05:30:46.061019+0000","last_clean":"2024-09-14T05:30:46.061019+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:30:46.061019+0000","last_undegraded":"2024-09-14T05:30:46.061019+0000","last_fullsized":"2024-09-14T05:30:46.061019+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"97'370","reported_seq":2110,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.064083+0000","last_change":"2024-09-14T05:30:46.064083+0000","last_active":"2024-09-14T05:30:46.064083+0000","last_peered":"2024-09-14T05:30:46.064083+0000","last_clean":"2024-09-14T05:30:46.064083+0000","last_became_active":"2024-09-14T05:29:24.012222+0000","last_became_peered":"2024-09-14T05:29:24.012222+0000","last_unstale":"2024-09-14T05:30:46.064083+0000","last_undegraded":"2024-09-14T05:30:46.064083+0000","last_fullsized":"2024-09-14T05:30:46.064083+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:00:18.099857+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0054397630000000002,"stat_sum":{"num_bytes":11525349,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":24,"num_read":2826,"num_read_kb":6026,"num_write":320,"num_write_kb":3878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.9","version":"93'583","reported_seq":2258,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.069093+0000","last_change":"2024-09-14T05:30:46.069473+0000","last_active":"2024-09-14T05:30:46.069093+0000","last_peered":"2024-09-14T05:30:46.069093+0000","last_clean":"2024-09-14T05:30:46.069093+0000","last_became_active":"2024-09-14T05:29:24.012041+0000","last_became_peered":"2024-09-14T05:29:24.012041+0000","last_unstale":"2024-09-14T05:30:46.069093+0000","last_undegraded":"2024-09-14T05:30:46.069093+0000","last_fullsized":"2024-09-14T05:30:46.069093+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:03:27.411006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073498620000000004,"stat_sum":{"num_bytes":9771921,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":21,"num_read":2883,"num_read_kb":7330,"num_write":734,"num_write_kb":6418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.a","version":"97'589","reported_seq":2461,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.063548+0000","last_change":"2024-09-14T05:30:46.063548+0000","last_active":"2024-09-14T05:30:46.063548+0000","last_peered":"2024-09-14T05:30:46.063548+0000","last_clean":"2024-09-14T05:30:46.063548+0000","last_became_active":"2024-09-14T05:29:24.011403+0000","last_became_peered":"2024-09-14T05:29:24.011403+0000","last_unstale":"2024-09-14T05:30:46.063548+0000","last_undegraded":"2024-09-14T05:30:46.063548+0000","last_fullsized":"2024-09-14T05:30:46.063548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:31:01.164650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030322140000000001,"stat_sum":{"num_bytes":13791126,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":26,"num_read":3872,"num_read_kb":7972,"num_write":669,"num_write_kb":4971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.b","version":"94'516","reported_seq":9691,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061328+0000","last_change":"2024-09-14T05:30:46.061500+0000","last_active":"2024-09-14T05:30:46.061328+0000","last_peered":"2024-09-14T05:30:46.061328+0000","last_clean":"2024-09-14T05:30:46.061328+0000","last_became_active":"2024-09-14T05:29:24.006836+0000","last_became_peered":"2024-09-14T05:29:24.006836+0000","last_unstale":"2024-09-14T05:30:46.061328+0000","last_undegraded":"2024-09-14T05:30:46.061328+0000","last_fullsized":"2024-09-14T05:30:46.061328+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:35:39.708618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037603300000000001,"stat_sum":{"num_bytes":30696360,"num_objects":54,"num_object_clones":7,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":32,"num_read":11264,"num_read_kb":29169,"num_write":600,"num_write_kb":13044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.c","version":"92'420","reported_seq":3343,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T05:30:42.089804+0000","last_change":"2024-09-14T05:30:42.090049+0000","last_active":"2024-09-14T05:30:42.089804+0000","last_peered":"2024-09-14T05:30:42.089804+0000","last_clean":"2024-09-14T05:30:42.089804+0000","last_became_active":"2024-09-14T05:29:24.012574+0000","last_became_peered":"2024-09-14T05:29:24.012574+0000","last_unstale":"2024-09-14T05:30:42.089804+0000","last_undegraded":"2024-09-14T05:30:42.089804+0000","last_fullsized":"2024-09-14T05:30:42.089804+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:59:10.407456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049961899999999997,"stat_sum":{"num_bytes":10692287,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":25,"num_read":2807,"num_read_kb":13312,"num_write":357,"num_write_kb":9799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.d","version":"97'1100","reported_seq":11209,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.069371+0000","last_change":"2024-09-14T05:30:46.069601+0000","last_active":"2024-09-14T05:30:46.069371+0000","last_peered":"2024-09-14T05:30:46.069371+0000","last_clean":"2024-09-14T05:30:46.069371+0000","last_became_active":"2024-09-14T05:29:24.012401+0000","last_became_peered":"2024-09-14T05:29:24.012401+0000","last_unstale":"2024-09-14T05:30:46.069371+0000","last_undegraded":"2024-09-14T05:30:46.069371+0000","last_fullsized":"2024-09-14T05:30:46.069371+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1100,"log_dups_size":0,"ondisk_log_size":1100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:33:10.493428+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0085316369999999999,"stat_sum":{"num_bytes":39708826,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":34,"num_read":14901,"num_read_kb":22340,"num_write":1329,"num_write_kb":6495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.e","version":"97'333","reported_seq":2683,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.075233+0000","last_change":"2024-09-14T05:30:46.075460+0000","last_active":"2024-09-14T05:30:46.075233+0000","last_peered":"2024-09-14T05:30:46.075233+0000","last_clean":"2024-09-14T05:30:46.075233+0000","last_became_active":"2024-09-14T05:29:24.015315+0000","last_became_peered":"2024-09-14T05:29:24.015315+0000","last_unstale":"2024-09-14T05:30:46.075233+0000","last_undegraded":"2024-09-14T05:30:46.075233+0000","last_fullsized":"2024-09-14T05:30:46.075233+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:01:11.642896+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0047092430000000001,"stat_sum":{"num_bytes":9005641,"num_objects":53,"num_object_clones":17,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":36,"num_read":2150,"num_read_kb":8871,"num_write":290,"num_write_kb":3939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.f","version":"87'458","reported_seq":11513,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T05:30:42.089771+0000","last_change":"2024-09-14T05:30:42.089969+0000","last_active":"2024-09-14T05:30:42.089771+0000","last_peered":"2024-09-14T05:30:42.089771+0000","last_clean":"2024-09-14T05:30:42.089771+0000","last_became_active":"2024-09-14T05:29:24.018854+0000","last_became_peered":"2024-09-14T05:29:24.018854+0000","last_unstale":"2024-09-14T05:30:42.089771+0000","last_undegraded":"2024-09-14T05:30:42.089771+0000","last_fullsized":"2024-09-14T05:30:42.089771+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:19:02.691236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043059000000000001,"stat_sum":{"num_bytes":24608807,"num_objects":75,"num_object_clones":20,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":43,"num_read":11854,"num_read_kb":31740,"num_write":408,"num_write_kb":11837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.10","version":"94'358","reported_seq":2863,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061135+0000","last_change":"2024-09-14T05:30:46.061628+0000","last_active":"2024-09-14T05:30:46.061135+0000","last_peered":"2024-09-14T05:30:46.061135+0000","last_clean":"2024-09-14T05:30:46.061135+0000","last_became_active":"2024-09-14T05:29:24.010897+0000","last_became_peered":"2024-09-14T05:29:24.010897+0000","last_unstale":"2024-09-14T05:30:46.061135+0000","last_undegraded":"2024-09-14T05:30:46.061135+0000","last_fullsized":"2024-09-14T05:30:46.061135+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:19:57.056458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078010799999999995,"stat_sum":{"num_bytes":30209040,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":26,"num_read":2374,"num_read_kb":22456,"num_write":331,"num_write_kb":8939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.11","version":"96'531","reported_seq":3760,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.196994+0000","last_change":"2024-09-14T05:30:46.062429+0000","last_active":"2024-09-14T05:30:46.196994+0000","last_peered":"2024-09-14T05:30:46.196994+0000","last_clean":"2024-09-14T05:30:46.196994+0000","last_became_active":"2024-09-14T05:29:24.013792+0000","last_became_peered":"2024-09-14T05:29:24.013792+0000","last_unstale":"2024-09-14T05:30:46.196994+0000","last_undegraded":"2024-09-14T05:30:46.196994+0000","last_fullsized":"2024-09-14T05:30:46.196994+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:21.993766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001953354,"stat_sum":{"num_bytes":11256390,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":26,"num_read":4987,"num_read_kb":24302,"num_write":504,"num_write_kb":3330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.12","version":"96'448","reported_seq":2349,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061386+0000","last_change":"2024-09-14T05:30:46.061694+0000","last_active":"2024-09-14T05:30:46.061386+0000","last_peered":"2024-09-14T05:30:46.061386+0000","last_clean":"2024-09-14T05:30:46.061386+0000","last_became_active":"2024-09-14T05:30:38.013627+0000","last_became_peered":"2024-09-14T05:30:38.013627+0000","last_unstale":"2024-09-14T05:30:46.061386+0000","last_undegraded":"2024-09-14T05:30:46.061386+0000","last_fullsized":"2024-09-14T05:30:46.061386+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:26:00.217325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055527400000000002,"stat_sum":{"num_bytes":12733139,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":20,"num_read":2778,"num_read_kb":16473,"num_write":481,"num_write_kb":10860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2769134,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.13","version":"97'637","reported_seq":2287,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.212957+0000","last_change":"2024-09-14T05:30:46.062888+0000","last_active":"2024-09-14T05:30:46.212957+0000","last_peered":"2024-09-14T05:30:46.212957+0000","last_clean":"2024-09-14T05:30:46.212957+0000","last_became_active":"2024-09-14T05:29:24.014371+0000","last_became_peered":"2024-09-14T05:29:24.014371+0000","last_unstale":"2024-09-14T05:30:46.212957+0000","last_undegraded":"2024-09-14T05:30:46.212957+0000","last_fullsized":"2024-09-14T05:30:46.212957+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:08:22.644295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023503259999999998,"stat_sum":{"num_bytes":33088997,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":25,"num_read":3378,"num_read_kb":19888,"num_write":893,"num_write_kb":5499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":542345866,"num_objects":1492,"num_object_clones":363,"num_object_copies":2984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1492,"num_whiteouts":909,"num_read":178198,"num_read_kb":561878,"num_write":19992,"num_write_kb":280592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200396800,"data_stored":205772395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17639,"ondisk_log_size":17639,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476759,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91108,"kb_used_data":41856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94280732,"statfs":{"total":96636764160,"available":96543469568,"internally_reserved":0,"allocated":42860544,"data_stored":43268435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52480,"kb_used_data":17124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94319360,"statfs":{"total":96636764160,"available":96583024640,"internally_reserved":0,"allocated":17534976,"data_stored":17583619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509464,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44360,"kb_used_data":17328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94327480,"statfs":{"total":96636764160,"available":96591339520,"internally_reserved":0,"allocated":17743872,"data_stored":18412012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66492,"kb_used_data":39464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94305348,"statfs":{"total":96636764160,"available":96568676352,"internally_reserved":0,"allocated":40411136,"data_stored":41394432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49728,"kb_used_data":22408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94322112,"statfs":{"total":96636764160,"available":96585842688,"internally_reserved":0,"allocated":22945792,"data_stored":22996796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85912,"kb_used_data":14456,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94285928,"statfs":{"total":96636764160,"available":96548790272,"internally_reserved":0,"allocated":14802944,"data_stored":14771019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47744,"kb_used_data":20424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94324096,"statfs":{"total":96636764160,"available":96587874304,"internally_reserved":0,"allocated":20914176,"data_stored":21144190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84016,"kb_used_data":34772,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94287824,"statfs":{"total":96636764160,"available":96550731776,"internally_reserved":0,"allocated":35606528,"data_stored":35663851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":41766912,"data_stored":42758913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33869824,"data_stored":34538733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19193856,"data_stored":19534777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13656064,"data_stored":14236250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":21286912,"data_stored":21499204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":38166528,"data_stored":39272128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16224256,"data_stored":17175056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16232448,"data_stored":16757334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:30:47.773 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T05:30:47.773 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T05:30:47.774 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:30:47.780 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:47.780 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:47.789 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:30:47.813 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:47.811+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:47.886 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:47.885+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:47.892 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:47.892 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:47.900 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:30:47.987 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:47.986+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:48.044 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:48.044 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:48.053 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:30:48.098 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:48.096+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:48.113 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:30:48.113 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:30:48.129 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-14T05:30:48.080473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":528400007,"num_objects":1472,"num_object_clones":354,"num_object_copies":2944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1472,"num_whiteouts":902,"num_read":178312,"num_read_kb":561972,"num_write":20123,"num_write_kb":282237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17760,"ondisk_log_size":17760,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":515544,"kb_used_data":201548,"kb_used_omap":12,"kb_used_meta":313203,"kb_avail":754459176,"statfs":{"total":773094113280,"available":772566196224,"internally_reserved":0,"allocated":206385152,"data_stored":208313270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":320720432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-304684246,"num_objects":-142,"num_object_clones":-40,"num_object_copies":-284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-142,"num_whiteouts":-18,"num_read":1625,"num_read_kb":1385,"num_write":259,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.077893"},"pg_stats":[{"pgid":"3.1f","version":"97'705","reported_seq":10691,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.181096+0000","last_change":"2024-09-14T05:30:46.070790+0000","last_active":"2024-09-14T05:30:46.181096+0000","last_peered":"2024-09-14T05:30:46.181096+0000","last_clean":"2024-09-14T05:30:46.181096+0000","last_became_active":"2024-09-14T05:29:24.018892+0000","last_became_peered":"2024-09-14T05:29:24.018892+0000","last_unstale":"2024-09-14T05:30:46.181096+0000","last_undegraded":"2024-09-14T05:30:46.181096+0000","last_fullsized":"2024-09-14T05:30:46.181096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:07:32.134157+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0054684850000000004,"stat_sum":{"num_bytes":34461912,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":30,"num_read":12069,"num_read_kb":37227,"num_write":795,"num_write_kb":31144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.1e","version":"97'375","reported_seq":3118,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.071407+0000","last_change":"2024-09-14T05:30:46.071407+0000","last_active":"2024-09-14T05:30:46.071407+0000","last_peered":"2024-09-14T05:30:46.071407+0000","last_clean":"2024-09-14T05:30:46.071407+0000","last_became_active":"2024-09-14T05:29:24.014468+0000","last_became_peered":"2024-09-14T05:29:24.014468+0000","last_unstale":"2024-09-14T05:30:46.071407+0000","last_undegraded":"2024-09-14T05:30:46.071407+0000","last_fullsized":"2024-09-14T05:30:46.071407+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:01:23.775510+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0063721719999999997,"stat_sum":{"num_bytes":22845168,"num_objects":45,"num_object_clones":13,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":26,"num_read":2771,"num_read_kb":32473,"num_write":390,"num_write_kb":21093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.1d","version":"97'1016","reported_seq":9700,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.068353+0000","last_change":"2024-09-14T05:30:46.069109+0000","last_active":"2024-09-14T05:30:46.068353+0000","last_peered":"2024-09-14T05:30:46.068353+0000","last_clean":"2024-09-14T05:30:46.068353+0000","last_became_active":"2024-09-14T05:29:24.014251+0000","last_became_peered":"2024-09-14T05:29:24.014251+0000","last_unstale":"2024-09-14T05:30:46.068353+0000","last_undegraded":"2024-09-14T05:30:46.068353+0000","last_fullsized":"2024-09-14T05:30:46.068353+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:01:01.591833+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0081406800000000008,"stat_sum":{"num_bytes":33229696,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":23,"num_read":12251,"num_read_kb":23813,"num_write":1546,"num_write_kb":7077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.1c","version":"97'534","reported_seq":10220,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.183951+0000","last_change":"2024-09-14T05:30:46.066151+0000","last_active":"2024-09-14T05:30:46.183951+0000","last_peered":"2024-09-14T05:30:46.183951+0000","last_clean":"2024-09-14T05:30:46.183951+0000","last_became_active":"2024-09-14T05:29:24.012427+0000","last_became_peered":"2024-09-14T05:29:24.012427+0000","last_unstale":"2024-09-14T05:30:46.183951+0000","last_undegraded":"2024-09-14T05:30:46.183951+0000","last_fullsized":"2024-09-14T05:30:46.183951+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:46:55.814565+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0047000719999999996,"stat_sum":{"num_bytes":33076136,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":33,"num_read":10673,"num_read_kb":34957,"num_write":513,"num_write_kb":4632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.1b","version":"97'521","reported_seq":3311,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.215051+0000","last_change":"2024-09-14T05:30:46.059132+0000","last_active":"2024-09-14T05:30:46.215051+0000","last_peered":"2024-09-14T05:30:46.215051+0000","last_clean":"2024-09-14T05:30:46.215051+0000","last_became_active":"2024-09-14T05:29:24.012079+0000","last_became_peered":"2024-09-14T05:29:24.012079+0000","last_unstale":"2024-09-14T05:30:46.215051+0000","last_undegraded":"2024-09-14T05:30:46.215051+0000","last_fullsized":"2024-09-14T05:30:46.215051+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:39:10.465754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0004929,"stat_sum":{"num_bytes":24215809,"num_objects":48,"num_object_clones":12,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":31,"num_read":4071,"num_read_kb":25920,"num_write":545,"num_write_kb":21860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.1a","version":"97'967","reported_seq":6186,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.196505+0000","last_change":"2024-09-14T05:30:46.069899+0000","last_active":"2024-09-14T05:30:46.196505+0000","last_peered":"2024-09-14T05:30:46.196505+0000","last_clean":"2024-09-14T05:30:46.196505+0000","last_became_active":"2024-09-14T05:29:24.012461+0000","last_became_peered":"2024-09-14T05:29:24.012461+0000","last_unstale":"2024-09-14T05:30:46.196505+0000","last_undegraded":"2024-09-14T05:30:46.196505+0000","last_fullsized":"2024-09-14T05:30:46.196505+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:55.157830+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0081011150000000007,"stat_sum":{"num_bytes":5611490,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":30,"num_read":10379,"num_read_kb":34350,"num_write":1259,"num_write_kb":8459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.19","version":"97'543","reported_seq":2046,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.169525+0000","last_change":"2024-09-14T05:30:46.068521+0000","last_active":"2024-09-14T05:30:46.169525+0000","last_peered":"2024-09-14T05:30:46.169525+0000","last_clean":"2024-09-14T05:30:46.169525+0000","last_became_active":"2024-09-14T05:29:24.018753+0000","last_became_peered":"2024-09-14T05:29:24.018753+0000","last_unstale":"2024-09-14T05:30:46.169525+0000","last_undegraded":"2024-09-14T05:30:46.169525+0000","last_fullsized":"2024-09-14T05:30:46.169525+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:47:08.137099+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0098771780000000003,"stat_sum":{"num_bytes":6018322,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":26,"num_read":2571,"num_read_kb":13570,"num_write":671,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.18","version":"97'643","reported_seq":2641,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.072986+0000","last_change":"2024-09-14T05:30:46.072986+0000","last_active":"2024-09-14T05:30:46.072986+0000","last_peered":"2024-09-14T05:30:46.072986+0000","last_clean":"2024-09-14T05:30:46.072986+0000","last_became_active":"2024-09-14T05:29:24.011176+0000","last_became_peered":"2024-09-14T05:29:24.011176+0000","last_unstale":"2024-09-14T05:30:46.072986+0000","last_undegraded":"2024-09-14T05:30:46.072986+0000","last_fullsized":"2024-09-14T05:30:46.072986+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:39:09.875678+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0075706009999999997,"stat_sum":{"num_bytes":3419100,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":22,"num_read":4441,"num_read_kb":7267,"num_write":666,"num_write_kb":3416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.17","version":"94'307","reported_seq":2905,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.071470+0000","last_change":"2024-09-14T05:30:46.072148+0000","last_active":"2024-09-14T05:30:46.071470+0000","last_peered":"2024-09-14T05:30:46.071470+0000","last_clean":"2024-09-14T05:30:46.071470+0000","last_became_active":"2024-09-14T05:29:24.014005+0000","last_became_peered":"2024-09-14T05:29:24.014005+0000","last_unstale":"2024-09-14T05:30:46.071470+0000","last_undegraded":"2024-09-14T05:30:46.071470+0000","last_fullsized":"2024-09-14T05:30:46.071470+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:03:38.962187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067189659999999998,"stat_sum":{"num_bytes":6954822,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":21,"num_read":2971,"num_read_kb":11191,"num_write":291,"num_write_kb":5639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.16","version":"97'308","reported_seq":9740,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.075750+0000","last_change":"2024-09-14T05:30:46.075750+0000","last_active":"2024-09-14T05:30:46.075750+0000","last_peered":"2024-09-14T05:30:46.075750+0000","last_clean":"2024-09-14T05:30:46.075750+0000","last_became_active":"2024-09-14T05:29:24.013434+0000","last_became_peered":"2024-09-14T05:29:24.013434+0000","last_unstale":"2024-09-14T05:30:46.075750+0000","last_undegraded":"2024-09-14T05:30:46.075750+0000","last_fullsized":"2024-09-14T05:30:46.075750+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:48:57.711176+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0052919129999999997,"stat_sum":{"num_bytes":18095362,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":35,"num_read":9226,"num_read_kb":22920,"num_write":292,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.15","version":"98'416","reported_seq":2582,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-14T05:30:47.203107+0000","last_change":"2024-09-14T05:30:47.086601+0000","last_active":"2024-09-14T05:30:47.203107+0000","last_peered":"2024-09-14T05:30:47.203107+0000","last_clean":"2024-09-14T05:30:47.203107+0000","last_became_active":"2024-09-14T05:29:24.014457+0000","last_became_peered":"2024-09-14T05:29:24.014457+0000","last_unstale":"2024-09-14T05:30:47.203107+0000","last_undegraded":"2024-09-14T05:30:47.203107+0000","last_fullsized":"2024-09-14T05:30:47.203107+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:10:54.235747+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0090467989999999995,"stat_sum":{"num_bytes":16981413,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":22,"num_read":2612,"num_read_kb":9349,"num_write":412,"num_write_kb":12875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"13","length":"9"}]},{"pgid":"3.14","version":"96'790","reported_seq":2309,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061838+0000","last_change":"2024-09-14T05:30:46.062287+0000","last_active":"2024-09-14T05:30:46.061838+0000","last_peered":"2024-09-14T05:30:46.061838+0000","last_clean":"2024-09-14T05:30:46.061838+0000","last_became_active":"2024-09-14T05:29:24.013266+0000","last_became_peered":"2024-09-14T05:29:24.013266+0000","last_unstale":"2024-09-14T05:30:46.061838+0000","last_undegraded":"2024-09-14T05:30:46.061838+0000","last_fullsized":"2024-09-14T05:30:46.061838+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:47:09.025200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018018629999999999,"stat_sum":{"num_bytes":12290463,"num_objects":46,"num_object_clones":7,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":32,"num_read":4166,"num_read_kb":5461,"num_write":1039,"num_write_kb":4015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.7","version":"0'0","reported_seq":170,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.060889+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:30:46.060889+0000","last_peered":"2024-09-14T05:30:46.060889+0000","last_clean":"2024-09-14T05:30:46.060889+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:30:46.060889+0000","last_undegraded":"2024-09-14T05:30:46.060889+0000","last_fullsized":"2024-09-14T05:30:46.060889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"97'318","reported_seq":2032,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.067950+0000","last_change":"2024-09-14T05:30:46.067950+0000","last_active":"2024-09-14T05:30:46.067950+0000","last_peered":"2024-09-14T05:30:46.067950+0000","last_clean":"2024-09-14T05:30:46.067950+0000","last_became_active":"2024-09-14T05:29:24.012099+0000","last_became_peered":"2024-09-14T05:29:24.012099+0000","last_unstale":"2024-09-14T05:30:46.067950+0000","last_undegraded":"2024-09-14T05:30:46.067950+0000","last_fullsized":"2024-09-14T05:30:46.067950+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:09:28.237972+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0092655359999999996,"stat_sum":{"num_bytes":2982656,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":22,"num_read":1942,"num_read_kb":3550,"num_write":265,"num_write_kb":5452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.058905+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:30:46.058905+0000","last_peered":"2024-09-14T05:30:46.058905+0000","last_clean":"2024-09-14T05:30:46.058905+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:30:46.058905+0000","last_undegraded":"2024-09-14T05:30:46.058905+0000","last_fullsized":"2024-09-14T05:30:46.058905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"96'865","reported_seq":3643,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.071611+0000","last_change":"2024-09-14T05:30:46.072251+0000","last_active":"2024-09-14T05:30:46.071611+0000","last_peered":"2024-09-14T05:30:46.071611+0000","last_clean":"2024-09-14T05:30:46.071611+0000","last_became_active":"2024-09-14T05:30:31.398064+0000","last_became_peered":"2024-09-14T05:30:31.398064+0000","last_unstale":"2024-09-14T05:30:46.071611+0000","last_undegraded":"2024-09-14T05:30:46.071611+0000","last_fullsized":"2024-09-14T05:30:46.071611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:09:33.638434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001881445,"stat_sum":{"num_bytes":30308876,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":27,"num_read":6380,"num_read_kb":8075,"num_write":1040,"num_write_kb":5150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":648894,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.5","version":"0'0","reported_seq":170,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061062+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:30:46.061062+0000","last_peered":"2024-09-14T05:30:46.061062+0000","last_clean":"2024-09-14T05:30:46.061062+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:30:46.061062+0000","last_undegraded":"2024-09-14T05:30:46.061062+0000","last_fullsized":"2024-09-14T05:30:46.061062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"94'537","reported_seq":2574,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.065100+0000","last_change":"2024-09-14T05:30:46.065871+0000","last_active":"2024-09-14T05:30:46.065100+0000","last_peered":"2024-09-14T05:30:46.065100+0000","last_clean":"2024-09-14T05:30:46.065100+0000","last_became_active":"2024-09-14T05:29:40.095441+0000","last_became_peered":"2024-09-14T05:29:40.095441+0000","last_unstale":"2024-09-14T05:30:46.065100+0000","last_undegraded":"2024-09-14T05:30:46.065100+0000","last_fullsized":"2024-09-14T05:30:46.065100+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:43.778442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010106340000000001,"stat_sum":{"num_bytes":7847118,"num_objects":53,"num_object_clones":13,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":34,"num_read":3487,"num_read_kb":12454,"num_write":608,"num_write_kb":11389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.4","version":"0'0","reported_seq":170,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.064808+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:30:46.064808+0000","last_peered":"2024-09-14T05:30:46.064808+0000","last_clean":"2024-09-14T05:30:46.064808+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:30:46.064808+0000","last_undegraded":"2024-09-14T05:30:46.064808+0000","last_fullsized":"2024-09-14T05:30:46.064808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"97'311","reported_seq":1755,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.073251+0000","last_change":"2024-09-14T05:30:46.073251+0000","last_active":"2024-09-14T05:30:46.073251+0000","last_peered":"2024-09-14T05:30:46.073251+0000","last_clean":"2024-09-14T05:30:46.073251+0000","last_became_active":"2024-09-14T05:29:24.016940+0000","last_became_peered":"2024-09-14T05:29:24.016940+0000","last_unstale":"2024-09-14T05:30:46.073251+0000","last_undegraded":"2024-09-14T05:30:46.073251+0000","last_fullsized":"2024-09-14T05:30:46.073251+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:03:23.026285+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0036575190000000001,"stat_sum":{"num_bytes":10220890,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":25,"num_read":1560,"num_read_kb":4382,"num_write":313,"num_write_kb":3996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.3","version":"98'536","reported_seq":2712,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-14T05:30:47.165364+0000","last_change":"2024-09-14T05:30:47.084336+0000","last_active":"2024-09-14T05:30:47.165364+0000","last_peered":"2024-09-14T05:30:47.165364+0000","last_clean":"2024-09-14T05:30:47.165364+0000","last_became_active":"2024-09-14T05:29:24.015905+0000","last_became_peered":"2024-09-14T05:29:24.015905+0000","last_unstale":"2024-09-14T05:30:47.165364+0000","last_undegraded":"2024-09-14T05:30:47.165364+0000","last_fullsized":"2024-09-14T05:30:47.165364+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:56:22.512385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068650279999999996,"stat_sum":{"num_bytes":8566589,"num_objects":65,"num_object_clones":19,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":41,"num_read":2436,"num_read_kb":9613,"num_write":569,"num_write_kb":7724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"13","length":"9"}]},{"pgid":"2.2","version":"22'2","reported_seq":172,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.060834+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:30:46.060834+0000","last_peered":"2024-09-14T05:30:46.060834+0000","last_clean":"2024-09-14T05:30:46.060834+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:30:46.060834+0000","last_undegraded":"2024-09-14T05:30:46.060834+0000","last_fullsized":"2024-09-14T05:30:46.060834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"97'418","reported_seq":10658,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.215588+0000","last_change":"2024-09-14T05:30:46.067741+0000","last_active":"2024-09-14T05:30:46.215588+0000","last_peered":"2024-09-14T05:30:46.215588+0000","last_clean":"2024-09-14T05:30:46.215588+0000","last_became_active":"2024-09-14T05:29:24.011764+0000","last_became_peered":"2024-09-14T05:29:24.011764+0000","last_unstale":"2024-09-14T05:30:46.215588+0000","last_undegraded":"2024-09-14T05:30:46.215588+0000","last_fullsized":"2024-09-14T05:30:46.215588+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:52:30.014697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002615062,"stat_sum":{"num_bytes":15504609,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":29,"num_read":11558,"num_read_kb":38786,"num_write":374,"num_write_kb":4949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.1","version":"0'0","reported_seq":170,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.060986+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:30:46.060986+0000","last_peered":"2024-09-14T05:30:46.060986+0000","last_clean":"2024-09-14T05:30:46.060986+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:30:46.060986+0000","last_undegraded":"2024-09-14T05:30:46.060986+0000","last_fullsized":"2024-09-14T05:30:46.060986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"96'677","reported_seq":1849,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.075537+0000","last_change":"2024-09-14T05:30:46.075965+0000","last_active":"2024-09-14T05:30:46.075537+0000","last_peered":"2024-09-14T05:30:46.075537+0000","last_clean":"2024-09-14T05:30:46.075537+0000","last_became_active":"2024-09-14T05:29:40.095255+0000","last_became_peered":"2024-09-14T05:29:40.095255+0000","last_unstale":"2024-09-14T05:30:46.075537+0000","last_undegraded":"2024-09-14T05:30:46.075537+0000","last_fullsized":"2024-09-14T05:30:46.075537+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:48:24.243895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046994810000000001,"stat_sum":{"num_bytes":2523174,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":22,"num_read":3109,"num_read_kb":7242,"num_write":912,"num_write_kb":10028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.0","version":"0'0","reported_seq":170,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061943+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:30:46.061943+0000","last_peered":"2024-09-14T05:30:46.061943+0000","last_clean":"2024-09-14T05:30:46.061943+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:30:46.061943+0000","last_undegraded":"2024-09-14T05:30:46.061943+0000","last_fullsized":"2024-09-14T05:30:46.061943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"94'540","reported_seq":2566,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.068389+0000","last_change":"2024-09-14T05:30:46.068389+0000","last_active":"2024-09-14T05:30:46.068389+0000","last_peered":"2024-09-14T05:30:46.068389+0000","last_clean":"2024-09-14T05:30:46.068389+0000","last_became_active":"2024-09-14T05:29:24.015792+0000","last_became_peered":"2024-09-14T05:29:24.015792+0000","last_unstale":"2024-09-14T05:30:46.068389+0000","last_undegraded":"2024-09-14T05:30:46.068389+0000","last_fullsized":"2024-09-14T05:30:46.068389+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:51:18.518863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009539657,"stat_sum":{"num_bytes":5561684,"num_objects":59,"num_object_clones":14,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":39,"num_read":3491,"num_read_kb":9434,"num_write":601,"num_write_kb":5626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"2.3","version":"20'1","reported_seq":171,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.060656+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:30:46.060656+0000","last_peered":"2024-09-14T05:30:46.060656+0000","last_clean":"2024-09-14T05:30:46.060656+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:30:46.060656+0000","last_undegraded":"2024-09-14T05:30:46.060656+0000","last_fullsized":"2024-09-14T05:30:46.060656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":261,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061019+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:30:46.061019+0000","last_peered":"2024-09-14T05:30:46.061019+0000","last_clean":"2024-09-14T05:30:46.061019+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:30:46.061019+0000","last_undegraded":"2024-09-14T05:30:46.061019+0000","last_fullsized":"2024-09-14T05:30:46.061019+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"97'370","reported_seq":2110,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.064083+0000","last_change":"2024-09-14T05:30:46.064083+0000","last_active":"2024-09-14T05:30:46.064083+0000","last_peered":"2024-09-14T05:30:46.064083+0000","last_clean":"2024-09-14T05:30:46.064083+0000","last_became_active":"2024-09-14T05:29:24.012222+0000","last_became_peered":"2024-09-14T05:29:24.012222+0000","last_unstale":"2024-09-14T05:30:46.064083+0000","last_undegraded":"2024-09-14T05:30:46.064083+0000","last_fullsized":"2024-09-14T05:30:46.064083+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:00:18.099857+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0054397630000000002,"stat_sum":{"num_bytes":11525349,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":24,"num_read":2826,"num_read_kb":6026,"num_write":320,"num_write_kb":3878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.9","version":"93'583","reported_seq":2258,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.069093+0000","last_change":"2024-09-14T05:30:46.069473+0000","last_active":"2024-09-14T05:30:46.069093+0000","last_peered":"2024-09-14T05:30:46.069093+0000","last_clean":"2024-09-14T05:30:46.069093+0000","last_became_active":"2024-09-14T05:29:24.012041+0000","last_became_peered":"2024-09-14T05:29:24.012041+0000","last_unstale":"2024-09-14T05:30:46.069093+0000","last_undegraded":"2024-09-14T05:30:46.069093+0000","last_fullsized":"2024-09-14T05:30:46.069093+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:03:27.411006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073498620000000004,"stat_sum":{"num_bytes":9771921,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":21,"num_read":2883,"num_read_kb":7330,"num_write":734,"num_write_kb":6418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.a","version":"97'589","reported_seq":2461,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.063548+0000","last_change":"2024-09-14T05:30:46.063548+0000","last_active":"2024-09-14T05:30:46.063548+0000","last_peered":"2024-09-14T05:30:46.063548+0000","last_clean":"2024-09-14T05:30:46.063548+0000","last_became_active":"2024-09-14T05:29:24.011403+0000","last_became_peered":"2024-09-14T05:29:24.011403+0000","last_unstale":"2024-09-14T05:30:46.063548+0000","last_undegraded":"2024-09-14T05:30:46.063548+0000","last_fullsized":"2024-09-14T05:30:46.063548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:31:01.164650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030322140000000001,"stat_sum":{"num_bytes":13791126,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":26,"num_read":3872,"num_read_kb":7972,"num_write":669,"num_write_kb":4971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.b","version":"94'516","reported_seq":9691,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061328+0000","last_change":"2024-09-14T05:30:46.061500+0000","last_active":"2024-09-14T05:30:46.061328+0000","last_peered":"2024-09-14T05:30:46.061328+0000","last_clean":"2024-09-14T05:30:46.061328+0000","last_became_active":"2024-09-14T05:29:24.006836+0000","last_became_peered":"2024-09-14T05:29:24.006836+0000","last_unstale":"2024-09-14T05:30:46.061328+0000","last_undegraded":"2024-09-14T05:30:46.061328+0000","last_fullsized":"2024-09-14T05:30:46.061328+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:35:39.708618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037603300000000001,"stat_sum":{"num_bytes":30696360,"num_objects":54,"num_object_clones":7,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":32,"num_read":11264,"num_read_kb":29169,"num_write":600,"num_write_kb":13044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.c","version":"98'424","reported_seq":3385,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-14T05:30:47.083834+0000","last_change":"2024-09-14T05:30:47.083834+0000","last_active":"2024-09-14T05:30:47.083834+0000","last_peered":"2024-09-14T05:30:47.083834+0000","last_clean":"2024-09-14T05:30:47.083834+0000","last_became_active":"2024-09-14T05:29:24.012574+0000","last_became_peered":"2024-09-14T05:29:24.012574+0000","last_unstale":"2024-09-14T05:30:47.083834+0000","last_undegraded":"2024-09-14T05:30:47.083834+0000","last_fullsized":"2024-09-14T05:30:47.083834+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:09.502178+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.006697817,"stat_sum":{"num_bytes":9950911,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":23,"num_read":2815,"num_read_kb":13316,"num_write":357,"num_write_kb":9799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"13","length":"9"}]},{"pgid":"3.d","version":"97'1100","reported_seq":11209,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.069371+0000","last_change":"2024-09-14T05:30:46.069601+0000","last_active":"2024-09-14T05:30:46.069371+0000","last_peered":"2024-09-14T05:30:46.069371+0000","last_clean":"2024-09-14T05:30:46.069371+0000","last_became_active":"2024-09-14T05:29:24.012401+0000","last_became_peered":"2024-09-14T05:29:24.012401+0000","last_unstale":"2024-09-14T05:30:46.069371+0000","last_undegraded":"2024-09-14T05:30:46.069371+0000","last_fullsized":"2024-09-14T05:30:46.069371+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1100,"log_dups_size":0,"ondisk_log_size":1100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:33:10.493428+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0085316369999999999,"stat_sum":{"num_bytes":39708826,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":34,"num_read":14901,"num_read_kb":22340,"num_write":1329,"num_write_kb":6495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.e","version":"97'333","reported_seq":2683,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.075233+0000","last_change":"2024-09-14T05:30:46.075460+0000","last_active":"2024-09-14T05:30:46.075233+0000","last_peered":"2024-09-14T05:30:46.075233+0000","last_clean":"2024-09-14T05:30:46.075233+0000","last_became_active":"2024-09-14T05:29:24.015315+0000","last_became_peered":"2024-09-14T05:29:24.015315+0000","last_unstale":"2024-09-14T05:30:46.075233+0000","last_undegraded":"2024-09-14T05:30:46.075233+0000","last_fullsized":"2024-09-14T05:30:46.075233+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:01:11.642896+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0047092430000000001,"stat_sum":{"num_bytes":9005641,"num_objects":53,"num_object_clones":17,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":36,"num_read":2150,"num_read_kb":8871,"num_write":290,"num_write_kb":3939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.f","version":"98'469","reported_seq":11557,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-14T05:30:47.083415+0000","last_change":"2024-09-14T05:30:47.083691+0000","last_active":"2024-09-14T05:30:47.083415+0000","last_peered":"2024-09-14T05:30:47.083415+0000","last_clean":"2024-09-14T05:30:47.083415+0000","last_became_active":"2024-09-14T05:29:24.018854+0000","last_became_peered":"2024-09-14T05:29:24.018854+0000","last_unstale":"2024-09-14T05:30:47.083415+0000","last_undegraded":"2024-09-14T05:30:47.083415+0000","last_fullsized":"2024-09-14T05:30:47.083415+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:59:09.376495+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.0065518160000000002,"stat_sum":{"num_bytes":15356631,"num_objects":64,"num_object_clones":16,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":39,"num_read":11854,"num_read_kb":31740,"num_write":411,"num_write_kb":11837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"13","length":"9"}]},{"pgid":"3.10","version":"94'358","reported_seq":2863,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061135+0000","last_change":"2024-09-14T05:30:46.061628+0000","last_active":"2024-09-14T05:30:46.061135+0000","last_peered":"2024-09-14T05:30:46.061135+0000","last_clean":"2024-09-14T05:30:46.061135+0000","last_became_active":"2024-09-14T05:29:24.010897+0000","last_became_peered":"2024-09-14T05:29:24.010897+0000","last_unstale":"2024-09-14T05:30:46.061135+0000","last_undegraded":"2024-09-14T05:30:46.061135+0000","last_fullsized":"2024-09-14T05:30:46.061135+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:19:57.056458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078010799999999995,"stat_sum":{"num_bytes":30209040,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":26,"num_read":2374,"num_read_kb":22456,"num_write":331,"num_write_kb":8939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.11","version":"96'531","reported_seq":3760,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.196994+0000","last_change":"2024-09-14T05:30:46.062429+0000","last_active":"2024-09-14T05:30:46.196994+0000","last_peered":"2024-09-14T05:30:46.196994+0000","last_clean":"2024-09-14T05:30:46.196994+0000","last_became_active":"2024-09-14T05:29:24.013792+0000","last_became_peered":"2024-09-14T05:29:24.013792+0000","last_unstale":"2024-09-14T05:30:46.196994+0000","last_undegraded":"2024-09-14T05:30:46.196994+0000","last_fullsized":"2024-09-14T05:30:46.196994+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:21.993766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001953354,"stat_sum":{"num_bytes":11256390,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":26,"num_read":4987,"num_read_kb":24302,"num_write":504,"num_write_kb":3330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.12","version":"96'448","reported_seq":2349,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.061386+0000","last_change":"2024-09-14T05:30:46.061694+0000","last_active":"2024-09-14T05:30:46.061386+0000","last_peered":"2024-09-14T05:30:46.061386+0000","last_clean":"2024-09-14T05:30:46.061386+0000","last_became_active":"2024-09-14T05:30:38.013627+0000","last_became_peered":"2024-09-14T05:30:38.013627+0000","last_unstale":"2024-09-14T05:30:46.061386+0000","last_undegraded":"2024-09-14T05:30:46.061386+0000","last_fullsized":"2024-09-14T05:30:46.061386+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:26:00.217325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055527400000000002,"stat_sum":{"num_bytes":12733139,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":20,"num_read":2778,"num_read_kb":16473,"num_write":481,"num_write_kb":10860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2769134,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"14","length":"8"}]},{"pgid":"3.13","version":"97'637","reported_seq":2287,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-14T05:30:46.212957+0000","last_change":"2024-09-14T05:30:46.062888+0000","last_active":"2024-09-14T05:30:46.212957+0000","last_peered":"2024-09-14T05:30:46.212957+0000","last_clean":"2024-09-14T05:30:46.212957+0000","last_became_active":"2024-09-14T05:29:24.014371+0000","last_became_peered":"2024-09-14T05:29:24.014371+0000","last_unstale":"2024-09-14T05:30:46.212957+0000","last_undegraded":"2024-09-14T05:30:46.212957+0000","last_fullsized":"2024-09-14T05:30:46.212957+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:08:22.644295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023503259999999998,"stat_sum":{"num_bytes":33088997,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":25,"num_read":3378,"num_read_kb":19888,"num_write":893,"num_write_kb":5499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"8"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":527809620,"num_objects":1468,"num_object_clones":354,"num_object_copies":2936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1468,"num_whiteouts":902,"num_read":178246,"num_read_kb":561917,"num_write":20020,"num_write_kb":280607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":193900544,"data_stored":198819790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17685,"ondisk_log_size":17685,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476760,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84812,"kb_used_data":35572,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94287028,"statfs":{"total":96636764160,"available":96549916672,"internally_reserved":0,"allocated":36425728,"data_stored":36347351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52480,"kb_used_data":17124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94319360,"statfs":{"total":96636764160,"available":96583024640,"internally_reserved":0,"allocated":17534976,"data_stored":17583619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509464,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44360,"kb_used_data":17328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94327480,"statfs":{"total":96636764160,"available":96591339520,"internally_reserved":0,"allocated":17743872,"data_stored":18412012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66492,"kb_used_data":39464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94305348,"statfs":{"total":96636764160,"available":96568676352,"internally_reserved":0,"allocated":40411136,"data_stored":41394432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49728,"kb_used_data":22408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94322112,"statfs":{"total":96636764160,"available":96585842688,"internally_reserved":0,"allocated":22945792,"data_stored":22996796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85912,"kb_used_data":14456,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94285928,"statfs":{"total":96636764160,"available":96548790272,"internally_reserved":0,"allocated":14802944,"data_stored":14771019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47744,"kb_used_data":20424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94324096,"statfs":{"total":96636764160,"available":96587874304,"internally_reserved":0,"allocated":20914176,"data_stored":21144190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84016,"kb_used_data":34772,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94287824,"statfs":{"total":96636764160,"available":96550731776,"internally_reserved":0,"allocated":35606528,"data_stored":35663851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":35270656,"data_stored":35806308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33869824,"data_stored":34538733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19193856,"data_stored":19534777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13656064,"data_stored":14236250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":21286912,"data_stored":21499204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":38166528,"data_stored":39272128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16224256,"data_stored":17175056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16232448,"data_stored":16757334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:30:48.180 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:48.181 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:48.189 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:48.187+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:48.190 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:30:48.289 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:48.288+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:48.294 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:30:48.337 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:48.337 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:48.346 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:30:48.390 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:48.390+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:48.416 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:48.417 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:48.435 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:48.435 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:48.451 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:48.451 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:48.468 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:48.468 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:48.485 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:48.485 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:48.491 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:48.489+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:48.496 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74187 3.f 3:f707b2d9:::rbd_data.112d46b1ca94.0000000000000061:11 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.926802334000001, 2024-09-14T05:30:48.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072367828999999995, 2024-09-14T05:30:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74187 2024-09-14T05:30:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0900000000000002e-07 2024-09-14T05:30:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568166+0000", 2024-09-14T05:30:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.683e-06 2024-09-14T05:30:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568172+0000", 2024-09-14T05:30:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0639999999999999e-06 2024-09-14T05:30:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568177+0000", 2024-09-14T05:30:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.967e-06 2024-09-14T05:30:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640193+0000", 2024-09-14T05:30:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072016284 2024-09-14T05:30:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640282+0000", 2024-09-14T05:30:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9011999999999994e-05 2024-09-14T05:30:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640532+0000", 2024-09-14T05:30:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024940999999999999 2024-09-14T05:30:48.500 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74188 3.f 3:f707b2d9:::rbd_data.112d46b1ca94.0000000000000061:11 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.926782961000001, 2024-09-14T05:30:48.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072600425999999996, 2024-09-14T05:30:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74188 2024-09-14T05:30:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568184+0000", 2024-09-14T05:30:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8700000000000001e-07 2024-09-14T05:30:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568200+0000", 2024-09-14T05:30:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6603e-05 2024-09-14T05:30:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568208+0000", 2024-09-14T05:30:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4800000000000004e-06 2024-09-14T05:30:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568213+0000", 2024-09-14T05:30:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6860000000000003e-06 2024-09-14T05:30:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640563+0000", 2024-09-14T05:30:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072349438000000002 2024-09-14T05:30:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640607+0000", 2024-09-14T05:30:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4014999999999998e-05 2024-09-14T05:30:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640784+0000", 2024-09-14T05:30:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017681700000000001 2024-09-14T05:30:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74194 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.506 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.926744829, 2024-09-14T05:30:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072795199000000005, 2024-09-14T05:30:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74194 2024-09-14T05:30:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:48.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568222+0000", 2024-09-14T05:30:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1300000000000001e-07 2024-09-14T05:30:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568224+0000", 2024-09-14T05:30:48.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0660000000000002e-06 2024-09-14T05:30:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568230+0000", 2024-09-14T05:30:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4470000000000003e-06 2024-09-14T05:30:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568235+0000", 2024-09-14T05:30:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1220000000000003e-06 2024-09-14T05:30:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640824+0000", 2024-09-14T05:30:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072588707000000002 2024-09-14T05:30:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640915+0000", 2024-09-14T05:30:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1008999999999995e-05 2024-09-14T05:30:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641017+0000", 2024-09-14T05:30:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010143499999999999 2024-09-14T05:30:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.510 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74208 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2693120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.926724253000003, 2024-09-14T05:30:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072905555999999996, 2024-09-14T05:30:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.511 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74208 2024-09-14T05:30:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9799999999999999e-07 2024-09-14T05:30:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568244+0000", 2024-09-14T05:30:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0339999999999999e-06 2024-09-14T05:30:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568251+0000", 2024-09-14T05:30:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4060000000000002e-06 2024-09-14T05:30:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568255+0000", 2024-09-14T05:30:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6589999999999998e-06 2024-09-14T05:30:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641048+0000", 2024-09-14T05:30:48.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072792647000000002 2024-09-14T05:30:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641086+0000", 2024-09-14T05:30:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8031999999999998e-05 2024-09-14T05:30:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.515 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641147+0000", 2024-09-14T05:30:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1379999999999998e-05 2024-09-14T05:30:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74236 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2662400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.926090590000001, 2024-09-14T05:30:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072517776000000006, 2024-09-14T05:30:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.516 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74236 2024-09-14T05:30:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568881+0000", 2024-09-14T05:30:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2340000000000002e-06 2024-09-14T05:30:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568886+0000", 2024-09-14T05:30:48.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9450000000000001e-06 2024-09-14T05:30:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568898+0000", 2024-09-14T05:30:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2150000000000001e-05 2024-09-14T05:30:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568907+0000", 2024-09-14T05:30:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9239999999999996e-06 2024-09-14T05:30:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.519 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641185+0000", 2024-09-14T05:30:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.07227807 2024-09-14T05:30:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641265+0000", 2024-09-14T05:30:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0575999999999996e-05 2024-09-14T05:30:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641393+0000", 2024-09-14T05:30:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012787700000000001 2024-09-14T05:30:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.520 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74239 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2695168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.926046196999998, 2024-09-14T05:30:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072593816000000005, 2024-09-14T05:30:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74239 2024-09-14T05:30:48.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4300000000000003e-07 2024-09-14T05:30:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568923+0000", 2024-09-14T05:30:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2019999999999998e-06 2024-09-14T05:30:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568933+0000", 2024-09-14T05:30:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0376e-05 2024-09-14T05:30:48.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568938+0000", 2024-09-14T05:30:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2850000000000002e-06 2024-09-14T05:30:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641424+0000", 2024-09-14T05:30:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072485230999999997 2024-09-14T05:30:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.524 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641464+0000", 2024-09-14T05:30:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0574000000000002e-05 2024-09-14T05:30:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641514+0000", 2024-09-14T05:30:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9604999999999997e-05 2024-09-14T05:30:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74252 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2664448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.525 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.925866282999998, 2024-09-14T05:30:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072631237000000001, 2024-09-14T05:30:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74252 2024-09-14T05:30:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569101+0000", 2024-09-14T05:30:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8700000000000005e-07 2024-09-14T05:30:48.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569104+0000", 2024-09-14T05:30:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.163e-06 2024-09-14T05:30:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569115+0000", 2024-09-14T05:30:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1335000000000001e-05 2024-09-14T05:30:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569121+0000", 2024-09-14T05:30:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0800000000000002e-06 2024-09-14T05:30:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641552+0000", 2024-09-14T05:30:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072430743000000006 2024-09-14T05:30:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641631+0000", 2024-09-14T05:30:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9388999999999995e-05 2024-09-14T05:30:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641731+0000", 2024-09-14T05:30:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9840000000000006e-05 2024-09-14T05:30:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74256 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2697216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.925835227, 2024-09-14T05:30:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072721090000000002, 2024-09-14T05:30:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.530 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74256 2024-09-14T05:30:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:48.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0299999999999999e-07 2024-09-14T05:30:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569133+0000", 2024-09-14T05:30:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.046e-06 2024-09-14T05:30:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569139+0000", 2024-09-14T05:30:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0279999999999997e-06 2024-09-14T05:30:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569144+0000", 2024-09-14T05:30:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6929999999999998e-06 2024-09-14T05:30:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641762+0000", 2024-09-14T05:30:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072617904999999996 2024-09-14T05:30:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641801+0000", 2024-09-14T05:30:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8902999999999997e-05 2024-09-14T05:30:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641852+0000", 2024-09-14T05:30:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1011999999999999e-05 2024-09-14T05:30:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74285 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2672640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.534 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.924989791000002, 2024-09-14T05:30:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072422069000000006, 2024-09-14T05:30:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74285 2024-09-14T05:30:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569977+0000", 2024-09-14T05:30:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3099999999999998e-07 2024-09-14T05:30:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.536 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569978+0000", 2024-09-14T05:30:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.598e-06 2024-09-14T05:30:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569984+0000", 2024-09-14T05:30:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4689999999999996e-06 2024-09-14T05:30:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569989+0000", 2024-09-14T05:30:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8409999999999999e-06 2024-09-14T05:30:48.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642229+0000", 2024-09-14T05:30:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072240046000000002 2024-09-14T05:30:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642306+0000", 2024-09-14T05:30:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7347000000000003e-05 2024-09-14T05:30:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642398+0000", 2024-09-14T05:30:48.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2336999999999994e-05 2024-09-14T05:30:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74292 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2705408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.924735083000002, 2024-09-14T05:30:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072361018999999999, 2024-09-14T05:30:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.539 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74292 2024-09-14T05:30:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570232+0000", 2024-09-14T05:30:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.06e-07 2024-09-14T05:30:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570235+0000", 2024-09-14T05:30:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3189999999999999e-06 2024-09-14T05:30:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570248+0000", 2024-09-14T05:30:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3006e-05 2024-09-14T05:30:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570254+0000", 2024-09-14T05:30:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.908e-06 2024-09-14T05:30:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642433+0000", 2024-09-14T05:30:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072179192000000003 2024-09-14T05:30:48.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642500+0000", 2024-09-14T05:30:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7080000000000001e-05 2024-09-14T05:30:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642592+0000", 2024-09-14T05:30:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1807999999999999e-05 2024-09-14T05:30:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.543 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74304 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2707456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.924378417, 2024-09-14T05:30:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072444960000000003, 2024-09-14T05:30:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74304 2024-09-14T05:30:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9800000000000004e-07 2024-09-14T05:30:48.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570590+0000", 2024-09-14T05:30:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0810000000000001e-06 2024-09-14T05:30:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570596+0000", 2024-09-14T05:30:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9789999999999998e-06 2024-09-14T05:30:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570601+0000", 2024-09-14T05:30:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6750000000000001e-06 2024-09-14T05:30:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642835+0000", 2024-09-14T05:30:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.07223359 2024-09-14T05:30:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642917+0000", 2024-09-14T05:30:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2013000000000002e-05 2024-09-14T05:30:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643033+0000", 2024-09-14T05:30:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116124 2024-09-14T05:30:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74321 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2709504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.923896427000003, 2024-09-14T05:30:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072350611999999995, 2024-09-14T05:30:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.548 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74321 2024-09-14T05:30:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3e-07 2024-09-14T05:30:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571073+0000", 2024-09-14T05:30:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6879999999999999e-06 2024-09-14T05:30:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571080+0000", 2024-09-14T05:30:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0319999999999998e-06 2024-09-14T05:30:48.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571085+0000", 2024-09-14T05:30:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2109999999999997e-06 2024-09-14T05:30:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643309+0000", 2024-09-14T05:30:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072223812999999998 2024-09-14T05:30:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643351+0000", 2024-09-14T05:30:48.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2239e-05 2024-09-14T05:30:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643420+0000", 2024-09-14T05:30:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8998999999999994e-05 2024-09-14T05:30:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74332 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2678784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.923573263999998, 2024-09-14T05:30:48.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072259062999999998, 2024-09-14T05:30:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74332 2024-09-14T05:30:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571394+0000", 2024-09-14T05:30:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:30:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571397+0000", 2024-09-14T05:30:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8469999999999998e-06 2024-09-14T05:30:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571411+0000", 2024-09-14T05:30:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.34e-05 2024-09-14T05:30:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571418+0000", 2024-09-14T05:30:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9149999999999999e-06 2024-09-14T05:30:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643458+0000", 2024-09-14T05:30:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072039881 2024-09-14T05:30:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643539+0000", 2024-09-14T05:30:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1650000000000006e-05 2024-09-14T05:30:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643652+0000", 2024-09-14T05:30:48.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000112593 2024-09-14T05:30:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74336 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2711552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.923536814000002, 2024-09-14T05:30:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072354038999999995, 2024-09-14T05:30:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.557 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74336 2024-09-14T05:30:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:48.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571430+0000", 2024-09-14T05:30:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8999999999999996e-07 2024-09-14T05:30:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571432+0000", 2024-09-14T05:30:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4159999999999998e-06 2024-09-14T05:30:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.559 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571439+0000", 2024-09-14T05:30:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.657e-06 2024-09-14T05:30:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571444+0000", 2024-09-14T05:30:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1329999999999998e-06 2024-09-14T05:30:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643690+0000", 2024-09-14T05:30:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072246088 2024-09-14T05:30:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643731+0000", 2024-09-14T05:30:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0311000000000001e-05 2024-09-14T05:30:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643783+0000", 2024-09-14T05:30:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2843999999999998e-05 2024-09-14T05:30:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.561 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74349 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.923515805000001, 2024-09-14T05:30:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072554297000000004, 2024-09-14T05:30:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74349 2024-09-14T05:30:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571451+0000", 2024-09-14T05:30:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.41e-07 2024-09-14T05:30:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571453+0000", 2024-09-14T05:30:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8360000000000001e-06 2024-09-14T05:30:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571475+0000", 2024-09-14T05:30:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2452000000000001e-05 2024-09-14T05:30:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571480+0000", 2024-09-14T05:30:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4430000000000002e-06 2024-09-14T05:30:48.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643821+0000", 2024-09-14T05:30:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072340347999999999 2024-09-14T05:30:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643904+0000", 2024-09-14T05:30:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3534000000000004e-05 2024-09-14T05:30:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644005+0000", 2024-09-14T05:30:48.565 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100343 2024-09-14T05:30:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74357 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2713600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.923471771999999, 2024-09-14T05:30:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072651144000000001, 2024-09-14T05:30:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.566 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74357 2024-09-14T05:30:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571495+0000", 2024-09-14T05:30:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8899999999999999e-07 2024-09-14T05:30:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571498+0000", 2024-09-14T05:30:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5840000000000002e-06 2024-09-14T05:30:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571504+0000", 2024-09-14T05:30:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6549999999999998e-06 2024-09-14T05:30:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571509+0000", 2024-09-14T05:30:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6500000000000004e-06 2024-09-14T05:30:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644043+0000", 2024-09-14T05:30:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072533832000000006 2024-09-14T05:30:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644090+0000", 2024-09-14T05:30:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7426000000000003e-05 2024-09-14T05:30:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644146+0000", 2024-09-14T05:30:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5408000000000002e-05 2024-09-14T05:30:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74364 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2682880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:48.570 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.923135000000002, 2024-09-14T05:30:48.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072531227000000004, 2024-09-14T05:30:48.571 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.571 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.571 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.571 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.571 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.571 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.571 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74364 2024-09-14T05:30:48.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.571 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571832+0000", 2024-09-14T05:30:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4e-07 2024-09-14T05:30:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571838+0000", 2024-09-14T05:30:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0519999999999998e-06 2024-09-14T05:30:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571865+0000", 2024-09-14T05:30:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7109000000000001e-05 2024-09-14T05:30:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571883+0000", 2024-09-14T05:30:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7428e-05 2024-09-14T05:30:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644182+0000", 2024-09-14T05:30:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072299551000000004 2024-09-14T05:30:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644260+0000", 2024-09-14T05:30:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7551000000000003e-05 2024-09-14T05:30:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644362+0000", 2024-09-14T05:30:48.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010259600000000001 2024-09-14T05:30:48.575 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.576 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.576 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74367 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2715648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.922771693000001, 2024-09-14T05:30:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072368259000000004, 2024-09-14T05:30:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74367 2024-09-14T05:30:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.577 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572195+0000", 2024-09-14T05:30:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.78e-07 2024-09-14T05:30:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572200+0000", 2024-09-14T05:30:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2520000000000001e-06 2024-09-14T05:30:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572213+0000", 2024-09-14T05:30:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3618000000000001e-05 2024-09-14T05:30:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572226+0000", 2024-09-14T05:30:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3040999999999999e-05 2024-09-14T05:30:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644409+0000", 2024-09-14T05:30:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072182982000000007 2024-09-14T05:30:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644489+0000", 2024-09-14T05:30:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9429000000000001e-05 2024-09-14T05:30:48.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.581 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644563+0000", 2024-09-14T05:30:48.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4059000000000004e-05 2024-09-14T05:30:48.581 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.581 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.581 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.581 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74380 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2684928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.581 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:48.581 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.922637915000003, 2024-09-14T05:30:48.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072442180999999994, 2024-09-14T05:30:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74380 2024-09-14T05:30:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572329+0000", 2024-09-14T05:30:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3300000000000001e-07 2024-09-14T05:30:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.584 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.584 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572333+0000", 2024-09-14T05:30:48.584 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8e-06 2024-09-14T05:30:48.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.584 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.584 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.584 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572346+0000", 2024-09-14T05:30:48.584 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.307e-05 2024-09-14T05:30:48.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.584 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.584 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572353+0000", 2024-09-14T05:30:48.585 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9e-06 2024-09-14T05:30:48.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.585 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644597+0000", 2024-09-14T05:30:48.585 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072244770999999999 2024-09-14T05:30:48.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.585 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644672+0000", 2024-09-14T05:30:48.585 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4945999999999997e-05 2024-09-14T05:30:48.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.586 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.586 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644770+0000", 2024-09-14T05:30:48.586 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7961000000000006e-05 2024-09-14T05:30:48.586 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.586 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.586 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.586 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74383 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2717696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:48.586 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:48.586 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 58.922381921000003, 2024-09-14T05:30:48.586 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072397258000000006, 2024-09-14T05:30:48.587 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.587 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.587 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.587 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.587 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.587 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.587 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74383 2024-09-14T05:30:48.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.587 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.587 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.587 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.587 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:48.587 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.588 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.588 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:48.588 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.588 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.588 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572585+0000", 2024-09-14T05:30:48.588 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:30:48.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.588 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.589 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572589+0000", 2024-09-14T05:30:48.589 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:30:48.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.589 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.589 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.589 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572602+0000", 2024-09-14T05:30:48.589 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2917e-05 2024-09-14T05:30:48.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.589 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.589 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.589 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572610+0000", 2024-09-14T05:30:48.589 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.943e-06 2024-09-14T05:30:48.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.590 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.590 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.590 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644791+0000", 2024-09-14T05:30:48.590 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072181113000000005 2024-09-14T05:30:48.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.590 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.590 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.590 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644861+0000", 2024-09-14T05:30:48.590 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0650999999999998e-05 2024-09-14T05:30:48.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.590 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.591 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.591 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644981+0000", 2024-09-14T05:30:48.591 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000120185 2024-09-14T05:30:48.591 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.591 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.591 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.591 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.591 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.591 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:48.592 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:48.592 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:48.593 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:30:48.594 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:48.594 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:48.594 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:48.595 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:48.596 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:48.592+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:48.693 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:48.693+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:48.736 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:48.736 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:48.736 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:48.736 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:48.745 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:30:48.794 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:48.794+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:48.846 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:48.846 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:48.846 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:48.846 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:48.846 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:48.854 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:30:48.898 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:48.897+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:48.958 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:30:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:30:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 101.18026523, 2024-09-14T05:30:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.24562984099999999, 2024-09-14T05:30:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:30:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:30:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1 2024-09-14T05:30:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:30:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:30:48.960 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776565+0000", 2024-09-14T05:30:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4780000000000001e-06 2024-09-14T05:30:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776571+0000", 2024-09-14T05:30:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7669999999999998e-06 2024-09-14T05:30:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776580+0000", 2024-09-14T05:30:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9830000000000002e-06 2024-09-14T05:30:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776589+0000", 2024-09-14T05:30:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7579999999999998e-06 2024-09-14T05:30:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.962 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776662+0000", 2024-09-14T05:30:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3332999999999997e-05 2024-09-14T05:30:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:30:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776670+0000", 2024-09-14T05:30:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9850000000000006e-06 2024-09-14T05:30:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.020867+0000", 2024-09-14T05:30:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.244196265 2024-09-14T05:30:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021022+0000", 2024-09-14T05:30:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000155699 2024-09-14T05:30:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021169+0000", 2024-09-14T05:30:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014661700000000001 2024-09-14T05:30:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.964 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:30:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021891+0000", 2024-09-14T05:30:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072146700000000005 2024-09-14T05:30:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022124+0000", 2024-09-14T05:30:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023354199999999999 2024-09-14T05:30:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:30:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022164+0000", 2024-09-14T05:30:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9897000000000001e-05 2024-09-14T05:30:48.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022193+0000", 2024-09-14T05:30:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9050000000000001e-05 2024-09-14T05:30:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:1989 3.1f e30/24 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head v 30'29, mlcod=30'27)", 2024-09-14T05:30:48.966 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:30:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.363494243000005, 2024-09-14T05:30:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.064242492999999998, 2024-09-14T05:30:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:30:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:30:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593336+0000", 2024-09-14T05:30:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3909999999999999e-06 2024-09-14T05:30:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.968 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616632+0000", 2024-09-14T05:30:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.023296598000000002 2024-09-14T05:30:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616657+0000", 2024-09-14T05:30:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5228e-05 2024-09-14T05:30:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616684+0000", 2024-09-14T05:30:48.969 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6441999999999999e-05 2024-09-14T05:30:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616755+0000", 2024-09-14T05:30:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1452000000000006e-05 2024-09-14T05:30:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616780+0000", 2024-09-14T05:30:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4436000000000001e-05 2024-09-14T05:30:48.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:30:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657527+0000", 2024-09-14T05:30:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.040747323000000002 2024-09-14T05:30:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657577+0000", 2024-09-14T05:30:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9623000000000002e-05 2024-09-14T05:30:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124262 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4481024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867845+0000", 2024-09-14T05:30:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.088983043999999, 2024-09-14T05:30:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048759513999999997, 2024-09-14T05:30:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124262 2024-09-14T05:30:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867845+0000", 2024-09-14T05:30:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867845+0000", 2024-09-14T05:30:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867846+0000", 2024-09-14T05:30:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2e-07 2024-09-14T05:30:48.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867848+0000", 2024-09-14T05:30:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0339999999999999e-06 2024-09-14T05:30:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867854+0000", 2024-09-14T05:30:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7350000000000001e-06 2024-09-14T05:30:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867859+0000", 2024-09-14T05:30:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9690000000000002e-06 2024-09-14T05:30:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916436+0000", 2024-09-14T05:30:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048577694999999997 2024-09-14T05:30:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916515+0000", 2024-09-14T05:30:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8982999999999999e-05 2024-09-14T05:30:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.975 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916605+0000", 2024-09-14T05:30:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9578000000000002e-05 2024-09-14T05:30:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124263 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4587520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:30:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.088963416999999, 2024-09-14T05:30:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048866746000000003, 2024-09-14T05:30:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.976 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124263 2024-09-14T05:30:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:30:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.977 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:30:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:30:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2199999999999999e-07 2024-09-14T05:30:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867867+0000", 2024-09-14T05:30:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.091e-06 2024-09-14T05:30:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867873+0000", 2024-09-14T05:30:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8869999999999997e-06 2024-09-14T05:30:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867884+0000", 2024-09-14T05:30:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0784999999999999e-05 2024-09-14T05:30:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916636+0000", 2024-09-14T05:30:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048752141999999998 2024-09-14T05:30:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916680+0000", 2024-09-14T05:30:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3621000000000001e-05 2024-09-14T05:30:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916732+0000", 2024-09-14T05:30:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1798000000000001e-05 2024-09-14T05:30:48.980 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124264 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4694016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867891+0000", 2024-09-14T05:30:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.088937061999999, 2024-09-14T05:30:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049046027999999998, 2024-09-14T05:30:48.981 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124264 2024-09-14T05:30:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867891+0000", 2024-09-14T05:30:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867891+0000", 2024-09-14T05:30:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867892+0000", 2024-09-14T05:30:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.39e-07 2024-09-14T05:30:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867894+0000", 2024-09-14T05:30:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.278e-06 2024-09-14T05:30:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867900+0000", 2024-09-14T05:30:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2469999999999997e-06 2024-09-14T05:30:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.986 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867905+0000", 2024-09-14T05:30:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0209999999999999e-06 2024-09-14T05:30:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916764+0000", 2024-09-14T05:30:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048859163999999997 2024-09-14T05:30:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916845+0000", 2024-09-14T05:30:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0309999999999995e-05 2024-09-14T05:30:48.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916937+0000", 2024-09-14T05:30:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2569000000000001e-05 2024-09-14T05:30:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124265 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:30:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.088916857000001, 2024-09-14T05:30:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049155040999999997, 2024-09-14T05:30:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124265 2024-09-14T05:30:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:30:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:30:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:30:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7099999999999997e-07 2024-09-14T05:30:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867914+0000", 2024-09-14T05:30:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9810000000000002e-06 2024-09-14T05:30:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867920+0000", 2024-09-14T05:30:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.643e-06 2024-09-14T05:30:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867924+0000", 2024-09-14T05:30:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.605e-06 2024-09-14T05:30:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916970+0000", 2024-09-14T05:30:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049045501999999998 2024-09-14T05:30:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917013+0000", 2024-09-14T05:30:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3857999999999999e-05 2024-09-14T05:30:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917067+0000", 2024-09-14T05:30:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3081000000000002e-05 2024-09-14T05:30:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.992 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124266 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4907008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867930+0000", 2024-09-14T05:30:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.088897998, 2024-09-14T05:30:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049337389000000002, 2024-09-14T05:30:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124266 2024-09-14T05:30:48.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867930+0000", 2024-09-14T05:30:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867930+0000", 2024-09-14T05:30:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867931+0000", 2024-09-14T05:30:48.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9999999999999998e-07 2024-09-14T05:30:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867933+0000", 2024-09-14T05:30:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.046e-06 2024-09-14T05:30:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867938+0000", 2024-09-14T05:30:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5589999999999996e-06 2024-09-14T05:30:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:48.995 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867941+0000", 2024-09-14T05:30:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.791e-06 2024-09-14T05:30:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917100+0000", 2024-09-14T05:30:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049158544999999998 2024-09-14T05:30:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917179+0000", 2024-09-14T05:30:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9350000000000004e-05 2024-09-14T05:30:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917268+0000", 2024-09-14T05:30:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8697999999999997e-05 2024-09-14T05:30:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124267 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5013504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:30:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.088883442, 2024-09-14T05:30:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049449223, 2024-09-14T05:30:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:48.997 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124267 2024-09-14T05:30:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:30:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:30:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:30:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.28e-07 2024-09-14T05:30:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867946+0000", 2024-09-14T05:30:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.116e-06 2024-09-14T05:30:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:48.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867950+0000", 2024-09-14T05:30:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2049999999999998e-06 2024-09-14T05:30:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867952+0000", 2024-09-14T05:30:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6800000000000002e-06 2024-09-14T05:30:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917300+0000", 2024-09-14T05:30:49.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049347633000000002 2024-09-14T05:30:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917343+0000", 2024-09-14T05:30:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3022000000000002e-05 2024-09-14T05:30:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917394+0000", 2024-09-14T05:30:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1338999999999998e-05 2024-09-14T05:30:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.001 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124268 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5120000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867955+0000", 2024-09-14T05:30:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.088873034999999, 2024-09-14T05:30:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049638350999999997, 2024-09-14T05:30:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124268 2024-09-14T05:30:49.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867955+0000", 2024-09-14T05:30:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867955+0000", 2024-09-14T05:30:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867956+0000", 2024-09-14T05:30:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1299999999999999e-07 2024-09-14T05:30:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867961+0000", 2024-09-14T05:30:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4269999999999999e-06 2024-09-14T05:30:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867965+0000", 2024-09-14T05:30:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.563e-06 2024-09-14T05:30:49.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867967+0000", 2024-09-14T05:30:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6469999999999998e-06 2024-09-14T05:30:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917427+0000", 2024-09-14T05:30:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049460102999999998 2024-09-14T05:30:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917506+0000", 2024-09-14T05:30:49.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8246999999999998e-05 2024-09-14T05:30:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917594+0000", 2024-09-14T05:30:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8151000000000004e-05 2024-09-14T05:30:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124269 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:30:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.088857418, 2024-09-14T05:30:49.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049749122999999999, 2024-09-14T05:30:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124269 2024-09-14T05:30:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:30:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:30:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:30:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2499999999999999e-07 2024-09-14T05:30:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867972+0000", 2024-09-14T05:30:49.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0440000000000001e-06 2024-09-14T05:30:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867975+0000", 2024-09-14T05:30:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9699999999999999e-06 2024-09-14T05:30:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867978+0000", 2024-09-14T05:30:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5909999999999998e-06 2024-09-14T05:30:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917625+0000", 2024-09-14T05:30:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049647450000000003 2024-09-14T05:30:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917668+0000", 2024-09-14T05:30:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3108000000000001e-05 2024-09-14T05:30:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917720+0000", 2024-09-14T05:30:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1734999999999998e-05 2024-09-14T05:30:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.011 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.011 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.011 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124270 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5292032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:49.011 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:30:49.011 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.088847478000002, 2024-09-14T05:30:49.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049946404, 2024-09-14T05:30:49.011 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.011 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.011 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.011 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.011 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.012 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.012 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124270 2024-09-14T05:30:49.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.012 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:30:49.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:30:49.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:30:49.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5600000000000002e-07 2024-09-14T05:30:49.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867982+0000", 2024-09-14T05:30:49.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.95e-07 2024-09-14T05:30:49.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.013 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.013 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867985+0000", 2024-09-14T05:30:49.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8890000000000002e-06 2024-09-14T05:30:49.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867988+0000", 2024-09-14T05:30:49.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5900000000000002e-06 2024-09-14T05:30:49.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917753+0000", 2024-09-14T05:30:49.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049765113 2024-09-14T05:30:49.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917833+0000", 2024-09-14T05:30:49.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0055000000000005e-05 2024-09-14T05:30:49.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917927+0000", 2024-09-14T05:30:49.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4506000000000006e-05 2024-09-14T05:30:49.015 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.015 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.015 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.015 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124271 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5398528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:49.015 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868010+0000", 2024-09-14T05:30:49.016 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.088818282999998, 2024-09-14T05:30:49.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050049336, 2024-09-14T05:30:49.016 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.016 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.016 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.016 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.016 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.016 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.016 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124271 2024-09-14T05:30:49.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.016 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868010+0000", 2024-09-14T05:30:49.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868010+0000", 2024-09-14T05:30:49.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868011+0000", 2024-09-14T05:30:49.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:49.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868012+0000", 2024-09-14T05:30:49.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5719999999999999e-06 2024-09-14T05:30:49.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868016+0000", 2024-09-14T05:30:49.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.861e-06 2024-09-14T05:30:49.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.018 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.018 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868019+0000", 2024-09-14T05:30:49.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7580000000000001e-06 2024-09-14T05:30:49.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917961+0000", 2024-09-14T05:30:49.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049941885999999998 2024-09-14T05:30:49.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918007+0000", 2024-09-14T05:30:49.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6187999999999999e-05 2024-09-14T05:30:49.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918059+0000", 2024-09-14T05:30:49.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2645000000000002e-05 2024-09-14T05:30:49.020 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.020 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.020 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.020 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124272 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:49.020 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:30:49.020 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.088806256000002, 2024-09-14T05:30:49.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050239936999999998, 2024-09-14T05:30:49.020 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.020 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.021 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.021 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.021 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.021 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.021 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124272 2024-09-14T05:30:49.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.021 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:30:49.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:30:49.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:30:49.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.16e-07 2024-09-14T05:30:49.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868023+0000", 2024-09-14T05:30:49.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7999999999999993e-07 2024-09-14T05:30:49.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868026+0000", 2024-09-14T05:30:49.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9859999999999999e-06 2024-09-14T05:30:49.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868029+0000", 2024-09-14T05:30:49.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.039e-06 2024-09-14T05:30:49.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.023 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.023 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918093+0000", 2024-09-14T05:30:49.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050063689000000001 2024-09-14T05:30:49.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918173+0000", 2024-09-14T05:30:49.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0131999999999998e-05 2024-09-14T05:30:49.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918262+0000", 2024-09-14T05:30:49.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8894999999999995e-05 2024-09-14T05:30:49.024 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.024 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.024 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.025 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124273 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:49.025 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868032+0000", 2024-09-14T05:30:49.025 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.088796107, 2024-09-14T05:30:49.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050357587000000002, 2024-09-14T05:30:49.025 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.025 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.025 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.025 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.025 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.025 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.025 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124273 2024-09-14T05:30:49.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.025 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868032+0000", 2024-09-14T05:30:49.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868032+0000", 2024-09-14T05:30:49.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868033+0000", 2024-09-14T05:30:49.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9799999999999999e-07 2024-09-14T05:30:49.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868034+0000", 2024-09-14T05:30:49.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1119999999999999e-06 2024-09-14T05:30:49.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868037+0000", 2024-09-14T05:30:49.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0189999999999998e-06 2024-09-14T05:30:49.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868039+0000", 2024-09-14T05:30:49.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.57e-06 2024-09-14T05:30:49.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918294+0000", 2024-09-14T05:30:49.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050254778999999999 2024-09-14T05:30:49.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.028 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.028 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918338+0000", 2024-09-14T05:30:49.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4288e-05 2024-09-14T05:30:49.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918390+0000", 2024-09-14T05:30:49.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1520999999999997e-05 2024-09-14T05:30:49.029 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.029 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.029 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.029 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124274 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:49.029 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:30:49.030 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.088786265, 2024-09-14T05:30:49.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050548334, 2024-09-14T05:30:49.030 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.030 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.030 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.030 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.030 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.030 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.030 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124274 2024-09-14T05:30:49.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.030 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:30:49.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:30:49.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:30:49.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1799999999999999e-07 2024-09-14T05:30:49.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868043+0000", 2024-09-14T05:30:49.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9999999999999995e-07 2024-09-14T05:30:49.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868046+0000", 2024-09-14T05:30:49.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8880000000000001e-06 2024-09-14T05:30:49.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868049+0000", 2024-09-14T05:30:49.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7e-06 2024-09-14T05:30:49.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918423+0000", 2024-09-14T05:30:49.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050374249000000003 2024-09-14T05:30:49.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918503+0000", 2024-09-14T05:30:49.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0049000000000006e-05 2024-09-14T05:30:49.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918590+0000", 2024-09-14T05:30:49.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7230000000000003e-05 2024-09-14T05:30:49.033 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.034 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.034 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.034 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124275 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5824512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:49.034 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:30:49.034 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.088776678999999, 2024-09-14T05:30:49.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050665241999999999, 2024-09-14T05:30:49.034 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.034 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.034 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.034 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.034 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.034 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.035 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124275 2024-09-14T05:30:49.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.035 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:30:49.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:30:49.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:30:49.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1e-07 2024-09-14T05:30:49.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868053+0000", 2024-09-14T05:30:49.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.049e-06 2024-09-14T05:30:49.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868056+0000", 2024-09-14T05:30:49.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9519999999999999e-06 2024-09-14T05:30:49.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868058+0000", 2024-09-14T05:30:49.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4789999999999999e-06 2024-09-14T05:30:49.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918622+0000", 2024-09-14T05:30:49.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050563679 2024-09-14T05:30:49.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918666+0000", 2024-09-14T05:30:49.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3600000000000003e-05 2024-09-14T05:30:49.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918717+0000", 2024-09-14T05:30:49.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1273000000000002e-05 2024-09-14T05:30:49.038 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.038 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.038 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.038 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124276 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:49.038 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:30:49.038 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.088767135000001, 2024-09-14T05:30:49.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050860996999999998, 2024-09-14T05:30:49.039 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.039 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.039 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.039 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.039 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.039 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.039 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124276 2024-09-14T05:30:49.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.039 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:30:49.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:30:49.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:30:49.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1e-07 2024-09-14T05:30:49.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868062+0000", 2024-09-14T05:30:49.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2099999999999995e-07 2024-09-14T05:30:49.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868065+0000", 2024-09-14T05:30:49.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9500000000000001e-06 2024-09-14T05:30:49.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868068+0000", 2024-09-14T05:30:49.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4760000000000001e-06 2024-09-14T05:30:49.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918750+0000", 2024-09-14T05:30:49.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050681926000000002 2024-09-14T05:30:49.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918829+0000", 2024-09-14T05:30:49.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9264000000000005e-05 2024-09-14T05:30:49.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918922+0000", 2024-09-14T05:30:49.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3250000000000003e-05 2024-09-14T05:30:49.043 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.043 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.043 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.043 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124277 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 6037504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:49.043 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:30:49.043 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.088757526999999, 2024-09-14T05:30:49.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050980939000000003, 2024-09-14T05:30:49.043 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.043 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.044 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.044 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.044 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.044 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.044 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124277 2024-09-14T05:30:49.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.044 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:30:49.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.045 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.045 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:30:49.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.045 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.045 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:30:49.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.11e-07 2024-09-14T05:30:49.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.045 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.045 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868072+0000", 2024-09-14T05:30:49.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9999999999999995e-07 2024-09-14T05:30:49.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868075+0000", 2024-09-14T05:30:49.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9969999999999999e-06 2024-09-14T05:30:49.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868078+0000", 2024-09-14T05:30:49.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5349999999999999e-06 2024-09-14T05:30:49.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918956+0000", 2024-09-14T05:30:49.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050878067999999999 2024-09-14T05:30:49.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919000+0000", 2024-09-14T05:30:49.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3837000000000001e-05 2024-09-14T05:30:49.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919052+0000", 2024-09-14T05:30:49.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2290999999999998e-05 2024-09-14T05:30:49.047 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.048 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.048 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.048 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124278 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 6144000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:49.048 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868080+0000", 2024-09-14T05:30:49.048 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.088748088999999, 2024-09-14T05:30:49.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.051170446000000001, 2024-09-14T05:30:49.048 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.048 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.048 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.048 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.048 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.049 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.049 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124278 2024-09-14T05:30:49.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.049 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868080+0000", 2024-09-14T05:30:49.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868080+0000", 2024-09-14T05:30:49.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868081+0000", 2024-09-14T05:30:49.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1500000000000001e-07 2024-09-14T05:30:49.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868081+0000", 2024-09-14T05:30:49.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9400000000000004e-07 2024-09-14T05:30:49.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868084+0000", 2024-09-14T05:30:49.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.943e-06 2024-09-14T05:30:49.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868087+0000", 2024-09-14T05:30:49.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6929999999999999e-06 2024-09-14T05:30:49.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919084+0000", 2024-09-14T05:30:49.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050996581999999999 2024-09-14T05:30:49.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919163+0000", 2024-09-14T05:30:49.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8931999999999996e-05 2024-09-14T05:30:49.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919251+0000", 2024-09-14T05:30:49.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8187000000000001e-05 2024-09-14T05:30:49.052 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.052 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.053 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.053 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124279 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 6250496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:30:49.053 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:30:49.053 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 31.088738479, 2024-09-14T05:30:49.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.051284430999999998, 2024-09-14T05:30:49.053 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.053 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.053 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.053 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.054 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.054 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.054 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124279 2024-09-14T05:30:49.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.054 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.054 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.054 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:30:49.054 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.054 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.054 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:30:49.054 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:30:49.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.11e-07 2024-09-14T05:30:49.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868091+0000", 2024-09-14T05:30:49.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8299999999999995e-07 2024-09-14T05:30:49.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868094+0000", 2024-09-14T05:30:49.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8870000000000001e-06 2024-09-14T05:30:49.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868097+0000", 2024-09-14T05:30:49.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5330000000000001e-06 2024-09-14T05:30:49.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919276+0000", 2024-09-14T05:30:49.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.051179740000000001 2024-09-14T05:30:49.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919322+0000", 2024-09-14T05:30:49.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5475000000000001e-05 2024-09-14T05:30:49.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919374+0000", 2024-09-14T05:30:49.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2602000000000002e-05 2024-09-14T05:30:49.057 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.057 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.057 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.057 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.057 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.058 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:49.059 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:30:49.060 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:49.058+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:49.095 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T05:30:49.160 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:49.159+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:49.210 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:49.210 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:49.210 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:49.210 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:49.221 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:30:49.264 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:49.263+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:49.324 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:49.325 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:49.325 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:49.325 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:49.325 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:49.333 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:30:49.375 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:49.373+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:49.462 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:49.461+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:49.478 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:49.478 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:49.478 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73760 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2848768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.917699564000003, 2024-09-14T05:30:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.04469911, 2024-09-14T05:30:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73760 2024-09-14T05:30:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.479 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559701+0000", 2024-09-14T05:30:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1909999999999999e-06 2024-09-14T05:30:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559708+0000", 2024-09-14T05:30:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7279999999999998e-06 2024-09-14T05:30:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.481 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559713+0000", 2024-09-14T05:30:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8389999999999997e-06 2024-09-14T05:30:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604310+0000", 2024-09-14T05:30:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044597664000000002 2024-09-14T05:30:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604352+0000", 2024-09-14T05:30:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1236000000000003e-05 2024-09-14T05:30:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.482 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604398+0000", 2024-09-14T05:30:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6091000000000003e-05 2024-09-14T05:30:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73773 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2801664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.917679395, 2024-09-14T05:30:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941026000000002, 2024-09-14T05:30:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.483 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73773 2024-09-14T05:30:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:49.484 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:49.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8599999999999999e-07 2024-09-14T05:30:49.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.485 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559721+0000", 2024-09-14T05:30:49.485 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0540000000000002e-06 2024-09-14T05:30:49.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.485 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.486 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559729+0000", 2024-09-14T05:30:49.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1359999999999997e-06 2024-09-14T05:30:49.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.486 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.486 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559734+0000", 2024-09-14T05:30:49.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2140000000000004e-06 2024-09-14T05:30:49.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.486 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.486 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604441+0000", 2024-09-14T05:30:49.486 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044706828999999997 2024-09-14T05:30:49.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604522+0000", 2024-09-14T05:30:49.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0603000000000001e-05 2024-09-14T05:30:49.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.487 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.487 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604660+0000", 2024-09-14T05:30:49.487 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000137804 2024-09-14T05:30:49.487 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.487 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.487 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.488 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73777 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2834432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.488 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:49.488 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.917656993999998, 2024-09-14T05:30:49.488 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045116270999999999, 2024-09-14T05:30:49.488 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.488 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.488 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.488 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.488 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.488 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.488 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73777 2024-09-14T05:30:49.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.488 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:49.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:49.489 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.489 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.489 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559742+0000", 2024-09-14T05:30:49.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4099999999999999e-07 2024-09-14T05:30:49.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559744+0000", 2024-09-14T05:30:49.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0040000000000002e-06 2024-09-14T05:30:49.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.490 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559749+0000", 2024-09-14T05:30:49.490 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9739999999999999e-06 2024-09-14T05:30:49.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.490 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559754+0000", 2024-09-14T05:30:49.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9540000000000003e-06 2024-09-14T05:30:49.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604695+0000", 2024-09-14T05:30:49.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941003 2024-09-14T05:30:49.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.491 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.491 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604766+0000", 2024-09-14T05:30:49.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0296999999999995e-05 2024-09-14T05:30:49.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.492 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.492 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604857+0000", 2024-09-14T05:30:49.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1598e-05 2024-09-14T05:30:49.492 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.492 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.492 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.492 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73790 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.492 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:49.492 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.917637059, 2024-09-14T05:30:49.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316437000000001, 2024-09-14T05:30:49.492 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.493 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.493 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.493 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.493 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.493 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.493 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73790 2024-09-14T05:30:49.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.493 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.493 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:49.493 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:49.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:49.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7599999999999998e-07 2024-09-14T05:30:49.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559763+0000", 2024-09-14T05:30:49.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0880000000000002e-06 2024-09-14T05:30:49.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559769+0000", 2024-09-14T05:30:49.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6740000000000002e-06 2024-09-14T05:30:49.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559774+0000", 2024-09-14T05:30:49.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6110000000000004e-06 2024-09-14T05:30:49.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604905+0000", 2024-09-14T05:30:49.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045130827999999998 2024-09-14T05:30:49.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.496 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.496 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604982+0000", 2024-09-14T05:30:49.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7410999999999995e-05 2024-09-14T05:30:49.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.496 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.496 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605077+0000", 2024-09-14T05:30:49.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5449e-05 2024-09-14T05:30:49.496 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.496 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.496 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.497 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73793 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2836480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.497 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:49.497 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.917617855000003, 2024-09-14T05:30:49.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045441212000000002, 2024-09-14T05:30:49.497 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.497 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.497 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.497 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.497 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.497 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.497 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73793 2024-09-14T05:30:49.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.498 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:49.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:49.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559781+0000", 2024-09-14T05:30:49.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2899999999999999e-07 2024-09-14T05:30:49.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559783+0000", 2024-09-14T05:30:49.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.075e-06 2024-09-14T05:30:49.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559788+0000", 2024-09-14T05:30:49.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5369999999999997e-06 2024-09-14T05:30:49.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559793+0000", 2024-09-14T05:30:49.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:30:49.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605109+0000", 2024-09-14T05:30:49.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316229999999999 2024-09-14T05:30:49.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605168+0000", 2024-09-14T05:30:49.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8949000000000001e-05 2024-09-14T05:30:49.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.501 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605221+0000", 2024-09-14T05:30:49.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3260000000000002e-05 2024-09-14T05:30:49.501 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.501 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.501 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.501 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73806 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.501 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:49.501 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.917599228999997, 2024-09-14T05:30:49.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045526189000000002, 2024-09-14T05:30:49.501 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.501 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.501 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.502 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.502 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.502 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.502 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73806 2024-09-14T05:30:49.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.502 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:49.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:49.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:49.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:30:49.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559801+0000", 2024-09-14T05:30:49.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.995e-06 2024-09-14T05:30:49.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559807+0000", 2024-09-14T05:30:49.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5450000000000003e-06 2024-09-14T05:30:49.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559811+0000", 2024-09-14T05:30:49.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5090000000000004e-06 2024-09-14T05:30:49.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605237+0000", 2024-09-14T05:30:49.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045426133 2024-09-14T05:30:49.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605278+0000", 2024-09-14T05:30:49.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1084999999999997e-05 2024-09-14T05:30:49.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605325+0000", 2024-09-14T05:30:49.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6582999999999998e-05 2024-09-14T05:30:49.505 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.505 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.505 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.505 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73809 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2838528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.505 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:49.505 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.917575724000002, 2024-09-14T05:30:49.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045596062999999999, 2024-09-14T05:30:49.506 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.506 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.506 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.506 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.506 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.506 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.506 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73809 2024-09-14T05:30:49.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.506 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.506 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.506 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:49.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:49.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559823+0000", 2024-09-14T05:30:49.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.08e-07 2024-09-14T05:30:49.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559825+0000", 2024-09-14T05:30:49.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.26e-06 2024-09-14T05:30:49.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559831+0000", 2024-09-14T05:30:49.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6690000000000003e-06 2024-09-14T05:30:49.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559835+0000", 2024-09-14T05:30:49.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5120000000000002e-06 2024-09-14T05:30:49.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605342+0000", 2024-09-14T05:30:49.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045506957000000001 2024-09-14T05:30:49.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605375+0000", 2024-09-14T05:30:49.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3250999999999997e-05 2024-09-14T05:30:49.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605418+0000", 2024-09-14T05:30:49.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3006000000000001e-05 2024-09-14T05:30:49.509 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.510 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.510 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.510 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73822 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2807808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.510 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:49.510 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.917548070999999, 2024-09-14T05:30:49.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045661405000000002, 2024-09-14T05:30:49.510 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.510 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.510 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.510 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.510 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.510 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.511 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73822 2024-09-14T05:30:49.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.511 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:49.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:49.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559851+0000", 2024-09-14T05:30:49.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4000000000000002e-07 2024-09-14T05:30:49.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559853+0000", 2024-09-14T05:30:49.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5380000000000001e-06 2024-09-14T05:30:49.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559859+0000", 2024-09-14T05:30:49.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0619999999999997e-06 2024-09-14T05:30:49.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559864+0000", 2024-09-14T05:30:49.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.977e-06 2024-09-14T05:30:49.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605435+0000", 2024-09-14T05:30:49.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045570672 2024-09-14T05:30:49.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605467+0000", 2024-09-14T05:30:49.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2486e-05 2024-09-14T05:30:49.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605511+0000", 2024-09-14T05:30:49.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4129999999999999e-05 2024-09-14T05:30:49.514 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.514 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.514 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.514 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73825 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2840576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.514 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:49.514 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.917524706999998, 2024-09-14T05:30:49.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045728694, 2024-09-14T05:30:49.515 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.515 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.515 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.515 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.515 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.515 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.515 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73825 2024-09-14T05:30:49.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.515 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.515 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.515 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:49.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:49.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559874+0000", 2024-09-14T05:30:49.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5400000000000002e-07 2024-09-14T05:30:49.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559876+0000", 2024-09-14T05:30:49.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.131e-06 2024-09-14T05:30:49.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559888+0000", 2024-09-14T05:30:49.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2216e-05 2024-09-14T05:30:49.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559893+0000", 2024-09-14T05:30:49.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6539999999999998e-06 2024-09-14T05:30:49.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605526+0000", 2024-09-14T05:30:49.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045633119 2024-09-14T05:30:49.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605560+0000", 2024-09-14T05:30:49.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3825999999999998e-05 2024-09-14T05:30:49.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605602+0000", 2024-09-14T05:30:49.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2293999999999999e-05 2024-09-14T05:30:49.518 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.519 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.519 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.519 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74023 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2863104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.519 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:49.519 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.916338732, 2024-09-14T05:30:49.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044635385999999999, 2024-09-14T05:30:49.519 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.519 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.519 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.519 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.519 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.519 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.520 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74023 2024-09-14T05:30:49.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.520 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.520 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.520 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:49.520 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.520 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.520 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:49.520 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561060+0000", 2024-09-14T05:30:49.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0699999999999998e-07 2024-09-14T05:30:49.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561072+0000", 2024-09-14T05:30:49.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2261e-05 2024-09-14T05:30:49.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561079+0000", 2024-09-14T05:30:49.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8399999999999997e-06 2024-09-14T05:30:49.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.522 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561084+0000", 2024-09-14T05:30:49.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7380000000000002e-06 2024-09-14T05:30:49.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.522 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605620+0000", 2024-09-14T05:30:49.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044536146999999998 2024-09-14T05:30:49.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.523 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605653+0000", 2024-09-14T05:30:49.523 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2969000000000002e-05 2024-09-14T05:30:49.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.523 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605695+0000", 2024-09-14T05:30:49.523 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2024000000000003e-05 2024-09-14T05:30:49.523 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.523 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.523 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.523 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74026 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2895872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.524 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:49.524 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.916307269999997, 2024-09-14T05:30:49.524 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044692717, 2024-09-14T05:30:49.524 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.524 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.524 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.524 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.524 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.524 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.524 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74026 2024-09-14T05:30:49.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.524 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:49.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:49.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:49.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:49.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561093+0000", 2024-09-14T05:30:49.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-14T05:30:49.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561099+0000", 2024-09-14T05:30:49.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0399999999999998e-06 2024-09-14T05:30:49.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561104+0000", 2024-09-14T05:30:49.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8180000000000003e-06 2024-09-14T05:30:49.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605710+0000", 2024-09-14T05:30:49.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044605657 2024-09-14T05:30:49.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605742+0000", 2024-09-14T05:30:49.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1949000000000002e-05 2024-09-14T05:30:49.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605784+0000", 2024-09-14T05:30:49.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1884000000000001e-05 2024-09-14T05:30:49.528 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.528 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.528 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.528 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74039 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.528 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:49.528 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.916180672000003, 2024-09-14T05:30:49.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044654905000000002, 2024-09-14T05:30:49.528 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.528 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.529 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.529 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.529 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.529 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.529 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74039 2024-09-14T05:30:49.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.529 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.529 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.529 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:49.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.529 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:49.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561218+0000", 2024-09-14T05:30:49.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4000000000000001e-07 2024-09-14T05:30:49.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561221+0000", 2024-09-14T05:30:49.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9730000000000002e-06 2024-09-14T05:30:49.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561229+0000", 2024-09-14T05:30:49.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1759999999999993e-06 2024-09-14T05:30:49.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.531 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561235+0000", 2024-09-14T05:30:49.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0109999999999997e-06 2024-09-14T05:30:49.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.531 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605798+0000", 2024-09-14T05:30:49.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044563285000000001 2024-09-14T05:30:49.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605831+0000", 2024-09-14T05:30:49.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2054999999999997e-05 2024-09-14T05:30:49.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605872+0000", 2024-09-14T05:30:49.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1764999999999997e-05 2024-09-14T05:30:49.532 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.532 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.532 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.532 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74042 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.533 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:49.533 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.916132218999998, 2024-09-14T05:30:49.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044773234000000002, 2024-09-14T05:30:49.533 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.533 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.533 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.533 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.533 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.533 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.533 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74042 2024-09-14T05:30:49.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.533 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.534 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:49.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.534 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:49.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.534 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.534 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:49.534 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4099999999999998e-07 2024-09-14T05:30:49.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561269+0000", 2024-09-14T05:30:49.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7930000000000002e-06 2024-09-14T05:30:49.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.535 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561278+0000", 2024-09-14T05:30:49.535 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5150000000000006e-06 2024-09-14T05:30:49.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.535 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561284+0000", 2024-09-14T05:30:49.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2029999999999997e-06 2024-09-14T05:30:49.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.536 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605934+0000", 2024-09-14T05:30:49.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044650449000000002 2024-09-14T05:30:49.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.536 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605983+0000", 2024-09-14T05:30:49.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8943999999999998e-05 2024-09-14T05:30:49.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.537 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.537 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606039+0000", 2024-09-14T05:30:49.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5689000000000002e-05 2024-09-14T05:30:49.537 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.537 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.537 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.537 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74052 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.537 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:49.537 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.916105344000002, 2024-09-14T05:30:49.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044830474000000002, 2024-09-14T05:30:49.538 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.538 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.538 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.538 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.538 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.538 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.538 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74052 2024-09-14T05:30:49.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.538 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:49.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:49.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:49.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1099999999999996e-07 2024-09-14T05:30:49.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561295+0000", 2024-09-14T05:30:49.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7770000000000001e-06 2024-09-14T05:30:49.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561303+0000", 2024-09-14T05:30:49.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6860000000000006e-06 2024-09-14T05:30:49.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561308+0000", 2024-09-14T05:30:49.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.023e-06 2024-09-14T05:30:49.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605893+0000", 2024-09-14T05:30:49.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044585458000000001 2024-09-14T05:30:49.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605933+0000", 2024-09-14T05:30:49.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9551999999999997e-05 2024-09-14T05:30:49.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606123+0000", 2024-09-14T05:30:49.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019046699999999999 2024-09-14T05:30:49.541 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.542 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.542 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.542 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74054 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.542 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:49.542 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.916083317000002, 2024-09-14T05:30:49.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093610999999999, 2024-09-14T05:30:49.542 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.542 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.542 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.542 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.542 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.542 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.543 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74054 2024-09-14T05:30:49.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.543 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.543 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.543 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:49.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.543 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.543 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:49.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:49.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9699999999999996e-07 2024-09-14T05:30:49.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561317+0000", 2024-09-14T05:30:49.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9740000000000001e-06 2024-09-14T05:30:49.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561324+0000", 2024-09-14T05:30:49.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4590000000000003e-06 2024-09-14T05:30:49.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561329+0000", 2024-09-14T05:30:49.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5969999999999999e-06 2024-09-14T05:30:49.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606142+0000", 2024-09-14T05:30:49.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044812347000000002 2024-09-14T05:30:49.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606184+0000", 2024-09-14T05:30:49.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2441999999999998e-05 2024-09-14T05:30:49.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606408+0000", 2024-09-14T05:30:49.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022429500000000001 2024-09-14T05:30:49.546 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.546 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.546 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.547 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74056 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.547 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:49.547 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.915888613999996, 2024-09-14T05:30:49.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045191685000000002, 2024-09-14T05:30:49.547 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.547 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.547 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.547 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.547 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.547 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.547 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74056 2024-09-14T05:30:49.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.547 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.548 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:49.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.548 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.548 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:49.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.548 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.548 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561510+0000", 2024-09-14T05:30:49.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2399999999999998e-07 2024-09-14T05:30:49.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561513+0000", 2024-09-14T05:30:49.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7219999999999999e-06 2024-09-14T05:30:49.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561524+0000", 2024-09-14T05:30:49.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1357e-05 2024-09-14T05:30:49.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561530+0000", 2024-09-14T05:30:49.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2890000000000003e-06 2024-09-14T05:30:49.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606438+0000", 2024-09-14T05:30:49.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044907326999999997 2024-09-14T05:30:49.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606520+0000", 2024-09-14T05:30:49.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1924999999999999e-05 2024-09-14T05:30:49.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606701+0000", 2024-09-14T05:30:49.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00018144100000000001 2024-09-14T05:30:49.551 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.551 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.551 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.551 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74058 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.551 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:49.551 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.915409941, 2024-09-14T05:30:49.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093368000000002, 2024-09-14T05:30:49.551 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.551 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.551 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.551 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.552 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.552 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.552 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74058 2024-09-14T05:30:49.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.552 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:49.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:49.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561989+0000", 2024-09-14T05:30:49.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3300000000000001e-07 2024-09-14T05:30:49.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561993+0000", 2024-09-14T05:30:49.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1790000000000001e-06 2024-09-14T05:30:49.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562007+0000", 2024-09-14T05:30:49.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3434e-05 2024-09-14T05:30:49.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562014+0000", 2024-09-14T05:30:49.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.537e-06 2024-09-14T05:30:49.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606736+0000", 2024-09-14T05:30:49.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044722117999999998 2024-09-14T05:30:49.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606829+0000", 2024-09-14T05:30:49.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2410999999999993e-05 2024-09-14T05:30:49.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607082+0000", 2024-09-14T05:30:49.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025285599999999998 2024-09-14T05:30:49.555 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.555 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.555 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.555 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74059 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.555 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:49.555 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.915351950000002, 2024-09-14T05:30:49.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045479928000000003, 2024-09-14T05:30:49.556 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.556 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.556 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.556 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.556 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.556 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.556 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74059 2024-09-14T05:30:49.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.556 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.556 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.556 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:49.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:49.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562047+0000", 2024-09-14T05:30:49.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6899999999999997e-07 2024-09-14T05:30:49.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562050+0000", 2024-09-14T05:30:49.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.943e-06 2024-09-14T05:30:49.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562058+0000", 2024-09-14T05:30:49.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0120000000000005e-06 2024-09-14T05:30:49.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562063+0000", 2024-09-14T05:30:49.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:30:49.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.559 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607186+0000", 2024-09-14T05:30:49.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045122348999999999 2024-09-14T05:30:49.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.559 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.559 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607265+0000", 2024-09-14T05:30:49.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9462999999999995e-05 2024-09-14T05:30:49.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.559 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.559 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607526+0000", 2024-09-14T05:30:49.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026090400000000001 2024-09-14T05:30:49.559 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.559 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.560 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.560 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74060 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2850816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:49.560 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:49.560 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 59.915326573000002, 2024-09-14T05:30:49.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045200092999999997, 2024-09-14T05:30:49.560 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.560 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.560 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.560 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.560 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.560 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.561 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74060 2024-09-14T05:30:49.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.561 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:49.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.561 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.561 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:49.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5200000000000002e-07 2024-09-14T05:30:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562075+0000", 2024-09-14T05:30:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5200000000000002e-06 2024-09-14T05:30:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562082+0000", 2024-09-14T05:30:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8929999999999999e-06 2024-09-14T05:30:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562088+0000", 2024-09-14T05:30:49.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5570000000000003e-06 2024-09-14T05:30:49.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607111+0000", 2024-09-14T05:30:49.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045023464999999999 2024-09-14T05:30:49.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607190+0000", 2024-09-14T05:30:49.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8721999999999996e-05 2024-09-14T05:30:49.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.564 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607272+0000", 2024-09-14T05:30:49.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1483999999999994e-05 2024-09-14T05:30:49.564 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.564 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.564 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.564 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:136816 3.7 3:e5cc52c3:::rbd_id.image_client.0-clone31:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-14T05:30:49.564 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:30.438431+0000", 2024-09-14T05:30:49.564 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 19.038966869999999, 2024-09-14T05:30:49.565 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.96261680299999997, 2024-09-14T05:30:49.565 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.565 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.565 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:49.565 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.565 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.565 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.565 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 136816 2024-09-14T05:30:49.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.565 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.565 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.565 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438431+0000", 2024-09-14T05:30:49.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.566 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438431+0000", 2024-09-14T05:30:49.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.566 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438432+0000", 2024-09-14T05:30:49.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0419999999999999e-06 2024-09-14T05:30:49.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.566 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438437+0000", 2024-09-14T05:30:49.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5000000000000001e-06 2024-09-14T05:30:49.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438446+0000", 2024-09-14T05:30:49.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5740000000000002e-06 2024-09-14T05:30:49.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438456+0000", 2024-09-14T05:30:49.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0056e-05 2024-09-14T05:30:49.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438500+0000", 2024-09-14T05:30:49.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3738e-05 2024-09-14T05:30:49.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:30:49.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438506+0000", 2024-09-14T05:30:49.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1659999999999999e-06 2024-09-14T05:30:49.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.399222+0000", 2024-09-14T05:30:49.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.96071578999999996 2024-09-14T05:30:49.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.399381+0000", 2024-09-14T05:30:49.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000158834 2024-09-14T05:30:49.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-14T05:30:49.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.399530+0000", 2024-09-14T05:30:49.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014862400000000001 2024-09-14T05:30:49.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:30:49.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.400507+0000", 2024-09-14T05:30:49.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0009779369999999999 2024-09-14T05:30:49.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:49.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.400969+0000", 2024-09-14T05:30:49.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046119800000000001 2024-09-14T05:30:49.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.570 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:30:49.570 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.401006+0000", 2024-09-14T05:30:49.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7594e-05 2024-09-14T05:30:49.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.570 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.570 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.401048+0000", 2024-09-14T05:30:49.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1749999999999998e-05 2024-09-14T05:30:49.570 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.570 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.570 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.571 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.571 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.571 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:49.572 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:30:49.573 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:49.571+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:49.672 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:49.672+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:49.716 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:49.716 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:49.716 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:49.716 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:49.725 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:30:49.773 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:49.772+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:49.827 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:49.827 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:49.827 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:49.827 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:49.827 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:49.836 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:30:49.877 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:49.876+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:49.975 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:49.975+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:49.980 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:49.981 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:49.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:49.981 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:49.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.981 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4235.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-14T05:30:49.981 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:30:49.981 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 96.13964781, 2024-09-14T05:30:49.981 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12887758399999999, 2024-09-14T05:30:49.981 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.981 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.981 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:30:49.981 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.981 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4235", 2024-09-14T05:30:49.981 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/3211559243", 2024-09-14T05:30:49.982 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 2 2024-09-14T05:30:49.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.982 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:30:49.982 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.982 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.982 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.982 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:30:49.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839833+0000", 2024-09-14T05:30:49.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:30:49.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839838+0000", 2024-09-14T05:30:49.983 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2440000000000001e-06 2024-09-14T05:30:49.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.983 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.983 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839848+0000", 2024-09-14T05:30:49.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0030000000000001e-05 2024-09-14T05:30:49.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839854+0000", 2024-09-14T05:30:49.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3750000000000002e-06 2024-09-14T05:30:49.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839908+0000", 2024-09-14T05:30:49.984 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3965999999999998e-05 2024-09-14T05:30:49.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.984 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:30:49.984 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839914+0000", 2024-09-14T05:30:49.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6139999999999997e-06 2024-09-14T05:30:49.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968558+0000", 2024-09-14T05:30:49.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12864361199999999 2024-09-14T05:30:49.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.985 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.985 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968710+0000", 2024-09-14T05:30:49.985 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015177300000000001 2024-09-14T05:30:49.986 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.986 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.986 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.986 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27764 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 188416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:49.986 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:30:49.986 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.768321428, 2024-09-14T05:30:49.986 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080826920999999996, 2024-09-14T05:30:49.986 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.986 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.986 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.987 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.987 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.987 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.987 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27764 2024-09-14T05:30:49.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.987 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:30:49.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.987 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.987 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:30:49.987 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211159+0000", 2024-09-14T05:30:49.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1800000000000001e-07 2024-09-14T05:30:49.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.988 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211161+0000", 2024-09-14T05:30:49.988 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.968e-06 2024-09-14T05:30:49.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.988 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211169+0000", 2024-09-14T05:30:49.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4800000000000001e-06 2024-09-14T05:30:49.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211174+0000", 2024-09-14T05:30:49.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1880000000000001e-06 2024-09-14T05:30:49.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.989 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.989 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291670+0000", 2024-09-14T05:30:49.989 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080495199000000003 2024-09-14T05:30:49.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291775+0000", 2024-09-14T05:30:49.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000105375 2024-09-14T05:30:49.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.990 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.990 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291985+0000", 2024-09-14T05:30:49.990 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021029299999999999 2024-09-14T05:30:49.990 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.990 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.990 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.990 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27765 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:49.990 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:30:49.991 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.768298678000001, 2024-09-14T05:30:49.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081115197999999999, 2024-09-14T05:30:49.991 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.991 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.991 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.991 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.991 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.991 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.991 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27765 2024-09-14T05:30:49.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.991 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.991 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.991 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:30:49.991 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:30:49.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211182+0000", 2024-09-14T05:30:49.992 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3599999999999999e-07 2024-09-14T05:30:49.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.992 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.992 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211184+0000", 2024-09-14T05:30:49.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.03e-06 2024-09-14T05:30:49.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211192+0000", 2024-09-14T05:30:49.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2309999999999996e-06 2024-09-14T05:30:49.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.993 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211197+0000", 2024-09-14T05:30:49.993 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4199999999999998e-06 2024-09-14T05:30:49.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.993 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292017+0000", 2024-09-14T05:30:49.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080819516999999994 2024-09-14T05:30:49.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292087+0000", 2024-09-14T05:30:49.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0056999999999997e-05 2024-09-14T05:30:49.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.994 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.994 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292296+0000", 2024-09-14T05:30:49.994 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000209507 2024-09-14T05:30:49.994 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.994 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.994 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.995 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27766 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:49.995 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:30:49.995 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.768275463999998, 2024-09-14T05:30:49.995 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081359077000000002, 2024-09-14T05:30:49.995 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.995 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:49.995 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:49.995 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:49.995 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:49.995 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:49.995 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27766 2024-09-14T05:30:49.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.996 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:49.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:49.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:30:49.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:49.996 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:30:49.996 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:49.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.996 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:49.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211205+0000", 2024-09-14T05:30:49.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7e-07 2024-09-14T05:30:49.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:49.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211207+0000", 2024-09-14T05:30:49.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4540000000000001e-06 2024-09-14T05:30:49.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.997 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:49.997 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211216+0000", 2024-09-14T05:30:49.997 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3680000000000002e-06 2024-09-14T05:30:49.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:49.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211221+0000", 2024-09-14T05:30:49.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3499999999999996e-06 2024-09-14T05:30:49.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:49.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292329+0000", 2024-09-14T05:30:49.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081108240999999998 2024-09-14T05:30:49.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.998 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:49.998 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292370+0000", 2024-09-14T05:30:49.998 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.121e-05 2024-09-14T05:30:49.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.999 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:49.999 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292563+0000", 2024-09-14T05:30:49.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019298399999999999 2024-09-14T05:30:49.999 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.999 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:49.999 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:49.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:49.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:49.999 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27767 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:49.999 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:49.999 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.76825195, 2024-09-14T05:30:49.999 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081623119999999993, 2024-09-14T05:30:49.999 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:49.999 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:50.000 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:50.000 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:50.000 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:50.000 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.000 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27767 2024-09-14T05:30:50.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.000 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:50.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:50.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:50.000 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.000 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:50.000 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:50.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:50.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:50.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0499999999999999e-07 2024-09-14T05:30:50.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:50.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211230+0000", 2024-09-14T05:30:50.001 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0700000000000001e-06 2024-09-14T05:30:50.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.001 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:50.001 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211236+0000", 2024-09-14T05:30:50.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8429999999999998e-06 2024-09-14T05:30:50.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211241+0000", 2024-09-14T05:30:50.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:30:50.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:50.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292594+0000", 2024-09-14T05:30:50.002 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081352868999999994 2024-09-14T05:30:50.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.002 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:50.002 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292631+0000", 2024-09-14T05:30:50.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7083999999999999e-05 2024-09-14T05:30:50.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.003 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:50.003 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292851+0000", 2024-09-14T05:30:50.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022011700000000001 2024-09-14T05:30:50.003 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.003 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:50.003 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.003 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27768 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 274432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:50.003 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:30:50.003 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.768233414000001, 2024-09-14T05:30:50.003 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081875032, 2024-09-14T05:30:50.003 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:50.004 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:50.004 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:50.004 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:50.004 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:50.004 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.004 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27768 2024-09-14T05:30:50.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.004 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:50.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:50.004 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:30:50.004 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.004 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:50.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:30:50.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:50.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211247+0000", 2024-09-14T05:30:50.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.46e-07 2024-09-14T05:30:50.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:50.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211248+0000", 2024-09-14T05:30:50.005 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7349999999999999e-06 2024-09-14T05:30:50.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.005 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:50.005 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211253+0000", 2024-09-14T05:30:50.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.899e-06 2024-09-14T05:30:50.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211258+0000", 2024-09-14T05:30:50.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2810000000000002e-06 2024-09-14T05:30:50.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:50.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292888+0000", 2024-09-14T05:30:50.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081629955000000004 2024-09-14T05:30:50.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.006 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:50.006 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292926+0000", 2024-09-14T05:30:50.006 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7947000000000001e-05 2024-09-14T05:30:50.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.007 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:50.007 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293121+0000", 2024-09-14T05:30:50.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019586899999999999 2024-09-14T05:30:50.007 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.007 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:50.007 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.007 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27769 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 294912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:50.007 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:30:50.007 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.768215325, 2024-09-14T05:30:50.007 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082181899000000003, 2024-09-14T05:30:50.008 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:50.008 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:50.008 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:50.008 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:50.008 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:50.008 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.008 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27769 2024-09-14T05:30:50.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.008 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:50.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.008 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:50.008 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:30:50.008 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:50.009 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:30:50.009 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.009 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:50.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211265+0000", 2024-09-14T05:30:50.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2199999999999999e-07 2024-09-14T05:30:50.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.010 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:50.010 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211267+0000", 2024-09-14T05:30:50.010 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2239999999999998e-06 2024-09-14T05:30:50.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:50.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211273+0000", 2024-09-14T05:30:50.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6540000000000004e-06 2024-09-14T05:30:50.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211277+0000", 2024-09-14T05:30:50.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6299999999999997e-06 2024-09-14T05:30:50.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.011 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:50.011 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293150+0000", 2024-09-14T05:30:50.011 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081872575000000003 2024-09-14T05:30:50.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:50.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293245+0000", 2024-09-14T05:30:50.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5327000000000004e-05 2024-09-14T05:30:50.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.012 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:50.012 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293446+0000", 2024-09-14T05:30:50.012 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000201067 2024-09-14T05:30:50.012 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.012 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:50.012 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.013 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.013 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27770 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:50.013 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:50.013 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.768195163000001, 2024-09-14T05:30:50.013 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082663707000000003, 2024-09-14T05:30:50.013 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:50.013 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:50.013 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:50.013 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:50.013 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:50.013 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.013 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27770 2024-09-14T05:30:50.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.014 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:50.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:50.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:50.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.014 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:50.014 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:50.014 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:50.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:50.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1199999999999998e-07 2024-09-14T05:30:50.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:50.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211292+0000", 2024-09-14T05:30:50.015 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1609999999999997e-06 2024-09-14T05:30:50.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.015 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:50.015 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211299+0000", 2024-09-14T05:30:50.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3300000000000004e-06 2024-09-14T05:30:50.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211303+0000", 2024-09-14T05:30:50.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7859999999999996e-06 2024-09-14T05:30:50.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.016 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:50.016 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293628+0000", 2024-09-14T05:30:50.016 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082324283999999998 2024-09-14T05:30:50.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:50.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293694+0000", 2024-09-14T05:30:50.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6277000000000003e-05 2024-09-14T05:30:50.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.017 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:50.017 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293948+0000", 2024-09-14T05:30:50.017 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025445699999999998 2024-09-14T05:30:50.017 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.018 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:50.018 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.018 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27790 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:50.018 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:30:50.018 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.768168758999998, 2024-09-14T05:30:50.018 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082461006000000003, 2024-09-14T05:30:50.018 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:50.018 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:50.018 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:50.018 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:50.018 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:50.019 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.019 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27790 2024-09-14T05:30:50.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.019 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:50.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:50.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:30:50.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.019 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:50.019 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:30:50.019 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:50.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211312+0000", 2024-09-14T05:30:50.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9299999999999998e-07 2024-09-14T05:30:50.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:50.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211314+0000", 2024-09-14T05:30:50.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000002e-06 2024-09-14T05:30:50.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.020 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:50.020 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211320+0000", 2024-09-14T05:30:50.020 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1589999999999998e-06 2024-09-14T05:30:50.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211325+0000", 2024-09-14T05:30:50.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9609999999999996e-06 2024-09-14T05:30:50.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:50.021 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293474+0000", 2024-09-14T05:30:50.021 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082149080999999999 2024-09-14T05:30:50.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.021 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293542+0000", 2024-09-14T05:30:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8497000000000005e-05 2024-09-14T05:30:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293772+0000", 2024-09-14T05:30:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022969900000000001 2024-09-14T05:30:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27791 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:50.023 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:30:50.023 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.768148445000001, 2024-09-14T05:30:50.023 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082700838999999998, 2024-09-14T05:30:50.023 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:50.023 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:50.023 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:50.023 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:50.023 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:50.023 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.023 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27791 2024-09-14T05:30:50.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.024 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:50.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:50.024 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:30:50.024 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.024 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.024 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:50.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:30:50.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:50.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211332+0000", 2024-09-14T05:30:50.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7399999999999999e-07 2024-09-14T05:30:50.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.025 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:50.025 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211334+0000", 2024-09-14T05:30:50.025 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8500000000000001e-06 2024-09-14T05:30:50.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:50.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211340+0000", 2024-09-14T05:30:50.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3010000000000003e-06 2024-09-14T05:30:50.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211345+0000", 2024-09-14T05:30:50.026 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.938e-06 2024-09-14T05:30:50.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.026 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:50.026 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293797+0000", 2024-09-14T05:30:50.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082452152000000001 2024-09-14T05:30:50.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:50.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293869+0000", 2024-09-14T05:30:50.027 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2236000000000004e-05 2024-09-14T05:30:50.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.027 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:50.027 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294032+0000", 2024-09-14T05:30:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016298800000000001 2024-09-14T05:30:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27792 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.768128914000002, 2024-09-14T05:30:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082897304000000005, 2024-09-14T05:30:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27792 2024-09-14T05:30:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:50.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:50.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:50.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:50.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2000000000000001e-07 2024-09-14T05:30:50.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:50.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211353+0000", 2024-09-14T05:30:50.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.761e-06 2024-09-14T05:30:50.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:50.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211359+0000", 2024-09-14T05:30:50.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2430000000000002e-06 2024-09-14T05:30:50.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211364+0000", 2024-09-14T05:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0239999999999997e-06 2024-09-14T05:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294059+0000", 2024-09-14T05:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082694834999999994 2024-09-14T05:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294114+0000", 2024-09-14T05:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4823e-05 2024-09-14T05:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294248+0000", 2024-09-14T05:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013429800000000001 2024-09-14T05:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27793 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.768109115000001, 2024-09-14T05:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083153362999999994, 2024-09-14T05:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27793 2024-09-14T05:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:50.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:50.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:50.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:50.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:50.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5100000000000001e-07 2024-09-14T05:30:50.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:50.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211373+0000", 2024-09-14T05:30:50.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.001e-06 2024-09-14T05:30:50.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:50.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211379+0000", 2024-09-14T05:30:50.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4559999999999997e-06 2024-09-14T05:30:50.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211384+0000", 2024-09-14T05:30:50.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9749999999999998e-06 2024-09-14T05:30:50.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:50.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294279+0000", 2024-09-14T05:30:50.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082894282999999999 2024-09-14T05:30:50.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:50.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294327+0000", 2024-09-14T05:30:50.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8208000000000003e-05 2024-09-14T05:30:50.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:50.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294524+0000", 2024-09-14T05:30:50.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019708900000000001 2024-09-14T05:30:50.038 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.038 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:50.039 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.039 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27794 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:50.039 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:50.039 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.768088790999997, 2024-09-14T05:30:50.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588590000000004, 2024-09-14T05:30:50.039 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:50.039 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:50.039 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:50.039 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:50.039 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:50.039 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.039 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27794 2024-09-14T05:30:50.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.040 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:50.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:50.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:50.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:50.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:50.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:50.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:50.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4499999999999997e-07 2024-09-14T05:30:50.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:50.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211393+0000", 2024-09-14T05:30:50.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.815e-06 2024-09-14T05:30:50.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.041 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:50.041 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211400+0000", 2024-09-14T05:30:50.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2899999999999999e-06 2024-09-14T05:30:50.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211404+0000", 2024-09-14T05:30:50.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:30:50.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:50.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294560+0000", 2024-09-14T05:30:50.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083155534000000003 2024-09-14T05:30:50.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:50.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294631+0000", 2024-09-14T05:30:50.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1378999999999995e-05 2024-09-14T05:30:50.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:50.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294980+0000", 2024-09-14T05:30:50.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00034844900000000002 2024-09-14T05:30:50.043 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.043 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:50.043 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.043 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27795 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:50.043 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:30:50.043 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.768069433000001, 2024-09-14T05:30:50.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083970367000000004, 2024-09-14T05:30:50.043 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:50.044 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:50.044 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:50.044 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:50.044 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:50.044 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.044 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27795 2024-09-14T05:30:50.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.044 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:50.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:50.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:30:50.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.045 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:50.045 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:30:50.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.045 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:50.045 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211411+0000", 2024-09-14T05:30:50.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5499999999999999e-07 2024-09-14T05:30:50.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:50.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211413+0000", 2024-09-14T05:30:50.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:30:50.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:50.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211419+0000", 2024-09-14T05:30:50.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1720000000000004e-06 2024-09-14T05:30:50.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211424+0000", 2024-09-14T05:30:50.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0690000000000001e-06 2024-09-14T05:30:50.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:50.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295013+0000", 2024-09-14T05:30:50.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588898999999994 2024-09-14T05:30:50.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:50.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295102+0000", 2024-09-14T05:30:50.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9462000000000005e-05 2024-09-14T05:30:50.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:50.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295381+0000", 2024-09-14T05:30:50.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000278566 2024-09-14T05:30:50.048 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.048 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:50.048 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.048 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27796 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:50.048 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:50.048 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.768049703999999, 2024-09-14T05:30:50.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084379107999999994, 2024-09-14T05:30:50.048 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:50.048 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:50.049 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:50.049 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:50.049 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:50.049 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.049 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27796 2024-09-14T05:30:50.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.049 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:50.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:50.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:50.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:50.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:50.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:50.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:50.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2800000000000002e-07 2024-09-14T05:30:50.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.050 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:50.050 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211432+0000", 2024-09-14T05:30:50.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:30:50.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:50.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211439+0000", 2024-09-14T05:30:50.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5239999999999997e-06 2024-09-14T05:30:50.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211444+0000", 2024-09-14T05:30:50.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.95e-06 2024-09-14T05:30:50.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:50.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295602+0000", 2024-09-14T05:30:50.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084158017000000002 2024-09-14T05:30:50.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:50.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295674+0000", 2024-09-14T05:30:50.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2385999999999994e-05 2024-09-14T05:30:50.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:50.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295809+0000", 2024-09-14T05:30:50.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013495900000000001 2024-09-14T05:30:50.052 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.052 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:50.052 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.053 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27797 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 266240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:50.053 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:50.053 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.768029722000001, 2024-09-14T05:30:50.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084898597000000006, 2024-09-14T05:30:50.053 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:50.053 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:50.053 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:50.053 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:50.053 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:50.053 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.053 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27797 2024-09-14T05:30:50.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.053 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:50.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.054 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:50.054 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:50.054 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.054 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:50.054 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:50.054 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.054 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:50.054 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:50.054 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:30:50.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:50.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211452+0000", 2024-09-14T05:30:50.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.708e-06 2024-09-14T05:30:50.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:50.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211458+0000", 2024-09-14T05:30:50.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3489999999999997e-06 2024-09-14T05:30:50.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211464+0000", 2024-09-14T05:30:50.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3820000000000003e-06 2024-09-14T05:30:50.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:50.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295839+0000", 2024-09-14T05:30:50.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084375055000000004 2024-09-14T05:30:50.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:50.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295918+0000", 2024-09-14T05:30:50.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9253999999999997e-05 2024-09-14T05:30:50.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:50.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296349+0000", 2024-09-14T05:30:50.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043051 2024-09-14T05:30:50.057 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.057 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:50.057 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.058 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27798 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:50.058 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:50.058 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.768010007000001, 2024-09-14T05:30:50.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085300275999999994, 2024-09-14T05:30:50.058 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:50.058 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:50.058 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:50.058 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:50.058 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:50.058 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.058 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27798 2024-09-14T05:30:50.058 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.058 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:50.058 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.058 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:50.059 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:50.059 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.059 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:50.059 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:50.059 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.059 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:50.060 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:50.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:50.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:50.060 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211472+0000", 2024-09-14T05:30:50.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7579999999999999e-06 2024-09-14T05:30:50.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:50.060 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211478+0000", 2024-09-14T05:30:50.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0310000000000004e-06 2024-09-14T05:30:50.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211483+0000", 2024-09-14T05:30:50.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8110000000000002e-06 2024-09-14T05:30:50.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:50.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296469+0000", 2024-09-14T05:30:50.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084986575999999994 2024-09-14T05:30:50.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:50.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296545+0000", 2024-09-14T05:30:50.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5749999999999998e-05 2024-09-14T05:30:50.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:50.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296770+0000", 2024-09-14T05:30:50.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022498900000000001 2024-09-14T05:30:50.062 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.062 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:50.062 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.062 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27799 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 307200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:50.062 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:50.062 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.767991094000003, 2024-09-14T05:30:50.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085718461999999995, 2024-09-14T05:30:50.062 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:50.062 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:50.062 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:50.063 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:50.063 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:50.063 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.063 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27799 2024-09-14T05:30:50.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.063 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:50.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.063 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:50.063 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:50.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.063 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:50.063 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:50.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:50.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:50.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8000000000000001e-07 2024-09-14T05:30:50.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:50.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211491+0000", 2024-09-14T05:30:50.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-14T05:30:50.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:50.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211497+0000", 2024-09-14T05:30:50.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.857e-06 2024-09-14T05:30:50.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211502+0000", 2024-09-14T05:30:50.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9790000000000001e-06 2024-09-14T05:30:50.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:50.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296800+0000", 2024-09-14T05:30:50.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085298099000000002 2024-09-14T05:30:50.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:50.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296871+0000", 2024-09-14T05:30:50.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0879999999999999e-05 2024-09-14T05:30:50.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:50.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297207+0000", 2024-09-14T05:30:50.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033651699999999999 2024-09-14T05:30:50.066 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.066 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:50.066 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.067 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27826 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:50.067 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:30:50.067 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.766275972999999, 2024-09-14T05:30:50.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084306729999999996, 2024-09-14T05:30:50.067 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:50.067 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:50.067 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:50.067 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:50.067 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:50.067 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.067 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27826 2024-09-14T05:30:50.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.067 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:50.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.067 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:50.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:30:50.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:50.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:30:50.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:50.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213205+0000", 2024-09-14T05:30:50.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.277e-06 2024-09-14T05:30:50.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:50.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213210+0000", 2024-09-14T05:30:50.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3120000000000001e-06 2024-09-14T05:30:50.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.069 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:50.069 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213226+0000", 2024-09-14T05:30:50.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5279000000000001e-05 2024-09-14T05:30:50.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.069 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.069 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213234+0000", 2024-09-14T05:30:50.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7020000000000007e-06 2024-09-14T05:30:50.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:50.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297239+0000", 2024-09-14T05:30:50.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084004876000000006 2024-09-14T05:30:50.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:50.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297311+0000", 2024-09-14T05:30:50.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1336999999999998e-05 2024-09-14T05:30:50.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:50.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297511+0000", 2024-09-14T05:30:50.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019994699999999999 2024-09-14T05:30:50.071 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.071 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:50.071 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.071 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27827 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:50.071 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:30:50.071 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.766230786000001, 2024-09-14T05:30:50.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084520537000000007, 2024-09-14T05:30:50.071 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:50.071 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:50.071 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:50.071 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:50.071 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:50.072 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.072 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27827 2024-09-14T05:30:50.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.072 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:50.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:50.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:30:50.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.073 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:50.073 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:30:50.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:50.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.073 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:50.073 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213250+0000", 2024-09-14T05:30:50.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1099999999999995e-07 2024-09-14T05:30:50.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.073 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:50.073 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213252+0000", 2024-09-14T05:30:50.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6699999999999998e-06 2024-09-14T05:30:50.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.074 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.074 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:50.074 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213259+0000", 2024-09-14T05:30:50.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8469999999999998e-06 2024-09-14T05:30:50.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.074 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.074 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.074 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213265+0000", 2024-09-14T05:30:50.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4940000000000003e-06 2024-09-14T05:30:50.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.075 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:50.075 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297541+0000", 2024-09-14T05:30:50.075 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084276731999999993 2024-09-14T05:30:50.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.075 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:50.075 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297579+0000", 2024-09-14T05:30:50.075 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8108999999999997e-05 2024-09-14T05:30:50.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:50.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:50.075 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:50.075 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297770+0000", 2024-09-14T05:30:50.076 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019007400000000001 2024-09-14T05:30:50.076 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.076 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:50.076 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.076 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:50.076 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:50.076 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:50.077 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:30:50.078 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:50.076+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:50.085 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-14T05:30:50.178 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:50.177+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:50.199 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:50.199 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:50.199 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:50.200 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:50.209 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:30:50.279 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:50.278+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:50.313 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:50.313 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:50.313 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:50.313 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:50.313 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:50.321 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:30:50.379 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:50.379+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:50.468 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:50.468 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:50.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:50.468 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:50.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.468 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3523 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6062080~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.468 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:50.468 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.951802861999994, 2024-09-14T05:30:50.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096911030999999995, 2024-09-14T05:30:50.468 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.469 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.469 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.469 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3523 2024-09-14T05:30:50.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.469 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:50.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:50.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514359+0000", 2024-09-14T05:30:50.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9299999999999999e-07 2024-09-14T05:30:50.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514361+0000", 2024-09-14T05:30:50.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:50.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514366+0000", 2024-09-14T05:30:50.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1660000000000002e-06 2024-09-14T05:30:50.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514371+0000", 2024-09-14T05:30:50.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.648e-06 2024-09-14T05:30:50.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.610954+0000", 2024-09-14T05:30:50.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096582170999999994 2024-09-14T05:30:50.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611002+0000", 2024-09-14T05:30:50.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7923999999999997e-05 2024-09-14T05:30:50.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611269+0000", 2024-09-14T05:30:50.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026789499999999998 2024-09-14T05:30:50.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.472 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.473 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3524 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6553600~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.473 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:50.473 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.951783669999998, 2024-09-14T05:30:50.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097263347999999999, 2024-09-14T05:30:50.473 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.473 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.473 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.473 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.473 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.473 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.474 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3524 2024-09-14T05:30:50.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.474 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:50.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:50.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:50.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:50.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514380+0000", 2024-09-14T05:30:50.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6500000000000001e-06 2024-09-14T05:30:50.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514386+0000", 2024-09-14T05:30:50.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7360000000000004e-06 2024-09-14T05:30:50.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514391+0000", 2024-09-14T05:30:50.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2830000000000003e-06 2024-09-14T05:30:50.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611295+0000", 2024-09-14T05:30:50.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096904814000000006 2024-09-14T05:30:50.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611343+0000", 2024-09-14T05:30:50.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7506999999999998e-05 2024-09-14T05:30:50.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611641+0000", 2024-09-14T05:30:50.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029798800000000001 2024-09-14T05:30:50.478 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.478 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.478 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.478 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3525 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7045120~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.478 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:50.478 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.951761782999995, 2024-09-14T05:30:50.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097644422999999994, 2024-09-14T05:30:50.479 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.479 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.479 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.479 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.479 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.479 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.479 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3525 2024-09-14T05:30:50.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.479 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:50.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:50.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514400+0000", 2024-09-14T05:30:50.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.77e-07 2024-09-14T05:30:50.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514402+0000", 2024-09-14T05:30:50.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6530000000000001e-06 2024-09-14T05:30:50.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514411+0000", 2024-09-14T05:30:50.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7429999999999997e-06 2024-09-14T05:30:50.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514417+0000", 2024-09-14T05:30:50.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2249999999999997e-06 2024-09-14T05:30:50.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611668+0000", 2024-09-14T05:30:50.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097250681000000005 2024-09-14T05:30:50.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611716+0000", 2024-09-14T05:30:50.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7336000000000002e-05 2024-09-14T05:30:50.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612044+0000", 2024-09-14T05:30:50.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032840799999999998 2024-09-14T05:30:50.482 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.483 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.483 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.483 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3526 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7536640~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.483 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:50.483 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.951737299000001, 2024-09-14T05:30:50.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097943884999999994, 2024-09-14T05:30:50.483 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.483 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.483 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.484 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3526 2024-09-14T05:30:50.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.484 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:50.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:50.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:50.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0400000000000002e-07 2024-09-14T05:30:50.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514426+0000", 2024-09-14T05:30:50.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:50.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514432+0000", 2024-09-14T05:30:50.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.105e-06 2024-09-14T05:30:50.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514438+0000", 2024-09-14T05:30:50.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5149999999999997e-06 2024-09-14T05:30:50.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612072+0000", 2024-09-14T05:30:50.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097633947999999998 2024-09-14T05:30:50.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612120+0000", 2024-09-14T05:30:50.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8458999999999998e-05 2024-09-14T05:30:50.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612368+0000", 2024-09-14T05:30:50.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024761999999999999 2024-09-14T05:30:50.487 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.487 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.487 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.487 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3527 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 8028160~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.488 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:50.488 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.951716586000003, 2024-09-14T05:30:50.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098244955999999994, 2024-09-14T05:30:50.488 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.488 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.488 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.489 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3527 2024-09-14T05:30:50.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.489 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:50.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:50.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519842+0000", 2024-09-14T05:30:50.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053975230000000004 2024-09-14T05:30:50.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519851+0000", 2024-09-14T05:30:50.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5599999999999994e-06 2024-09-14T05:30:50.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519858+0000", 2024-09-14T05:30:50.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2409999999999998e-06 2024-09-14T05:30:50.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519863+0000", 2024-09-14T05:30:50.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:30:50.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612395+0000", 2024-09-14T05:30:50.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092532552000000004 2024-09-14T05:30:50.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612442+0000", 2024-09-14T05:30:50.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6486000000000002e-05 2024-09-14T05:30:50.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612690+0000", 2024-09-14T05:30:50.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024791599999999999 2024-09-14T05:30:50.492 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.492 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.492 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.493 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3603 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5177344~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.493 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:50.493 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.946164140999997, 2024-09-14T05:30:50.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096796436, 2024-09-14T05:30:50.493 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.493 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.493 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.493 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3603 2024-09-14T05:30:50.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.493 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:50.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:50.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:50.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1899999999999999e-07 2024-09-14T05:30:50.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519998+0000", 2024-09-14T05:30:50.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5000000000000001e-07 2024-09-14T05:30:50.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520002+0000", 2024-09-14T05:30:50.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7299999999999999e-06 2024-09-14T05:30:50.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520004+0000", 2024-09-14T05:30:50.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.429e-06 2024-09-14T05:30:50.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616493+0000", 2024-09-14T05:30:50.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096488345000000003 2024-09-14T05:30:50.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616544+0000", 2024-09-14T05:30:50.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1724000000000001e-05 2024-09-14T05:30:50.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616794+0000", 2024-09-14T05:30:50.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000249139 2024-09-14T05:30:50.497 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.497 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.497 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.498 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3604 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5668864~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.498 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:50.498 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.946149503000001, 2024-09-14T05:30:50.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108222999999994, 2024-09-14T05:30:50.498 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.498 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.498 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.499 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.499 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3604 2024-09-14T05:30:50.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.499 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:50.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:50.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:50.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7000000000000001e-07 2024-09-14T05:30:50.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520013+0000", 2024-09-14T05:30:50.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.389e-06 2024-09-14T05:30:50.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520018+0000", 2024-09-14T05:30:50.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3019999999999996e-06 2024-09-14T05:30:50.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520026+0000", 2024-09-14T05:30:50.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6170000000000007e-06 2024-09-14T05:30:50.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616816+0000", 2024-09-14T05:30:50.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096789766999999999 2024-09-14T05:30:50.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616871+0000", 2024-09-14T05:30:50.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4811000000000001e-05 2024-09-14T05:30:50.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617120+0000", 2024-09-14T05:30:50.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024906700000000001 2024-09-14T05:30:50.502 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.502 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.503 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.503 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3605 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6160384~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.503 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:50.503 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.946131042999994, 2024-09-14T05:30:50.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097423945999999997, 2024-09-14T05:30:50.503 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.503 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.503 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.503 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.503 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.503 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.503 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3605 2024-09-14T05:30:50.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.504 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:50.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:50.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:50.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3999999999999998e-07 2024-09-14T05:30:50.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520032+0000", 2024-09-14T05:30:50.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.083e-06 2024-09-14T05:30:50.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520036+0000", 2024-09-14T05:30:50.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.019e-06 2024-09-14T05:30:50.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520038+0000", 2024-09-14T05:30:50.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4849999999999999e-06 2024-09-14T05:30:50.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617147+0000", 2024-09-14T05:30:50.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108665999999996 2024-09-14T05:30:50.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617199+0000", 2024-09-14T05:30:50.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2558000000000001e-05 2024-09-14T05:30:50.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617454+0000", 2024-09-14T05:30:50.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025489499999999999 2024-09-14T05:30:50.507 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.507 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.507 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.507 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3606 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6651904~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.508 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:50.508 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.946120320999995, 2024-09-14T05:30:50.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097745705000000002, 2024-09-14T05:30:50.508 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.508 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.508 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.508 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3606 2024-09-14T05:30:50.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.509 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:50.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:50.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:50.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-14T05:30:50.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520042+0000", 2024-09-14T05:30:50.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4799999999999997e-07 2024-09-14T05:30:50.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520046+0000", 2024-09-14T05:30:50.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8149999999999999e-06 2024-09-14T05:30:50.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520049+0000", 2024-09-14T05:30:50.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.576e-06 2024-09-14T05:30:50.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617476+0000", 2024-09-14T05:30:50.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097427558999999997 2024-09-14T05:30:50.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617522+0000", 2024-09-14T05:30:50.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5927999999999998e-05 2024-09-14T05:30:50.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617787+0000", 2024-09-14T05:30:50.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000264663 2024-09-14T05:30:50.513 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.513 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.513 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.513 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3607 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7143424~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.513 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:50.514 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.946109793999995, 2024-09-14T05:30:50.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097871125000000003, 2024-09-14T05:30:50.514 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.514 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.514 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.514 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.514 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.514 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.515 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3607 2024-09-14T05:30:50.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.515 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:50.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:50.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520052+0000", 2024-09-14T05:30:50.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.03e-07 2024-09-14T05:30:50.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520053+0000", 2024-09-14T05:30:50.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.85e-07 2024-09-14T05:30:50.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520056+0000", 2024-09-14T05:30:50.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9140000000000001e-06 2024-09-14T05:30:50.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520059+0000", 2024-09-14T05:30:50.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4600000000000002e-06 2024-09-14T05:30:50.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617809+0000", 2024-09-14T05:30:50.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097750216000000001 2024-09-14T05:30:50.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617864+0000", 2024-09-14T05:30:50.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5211999999999999e-05 2024-09-14T05:30:50.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617923+0000", 2024-09-14T05:30:50.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8235000000000001e-05 2024-09-14T05:30:50.518 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.518 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.518 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.519 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3608 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7634944~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.519 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:50.519 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.946099485999994, 2024-09-14T05:30:50.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098199212999999994, 2024-09-14T05:30:50.519 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.519 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.519 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.520 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3608 2024-09-14T05:30:50.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.520 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:50.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:50.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525953+0000", 2024-09-14T05:30:50.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058911850000000002 2024-09-14T05:30:50.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525957+0000", 2024-09-14T05:30:50.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2379999999999999e-06 2024-09-14T05:30:50.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525967+0000", 2024-09-14T05:30:50.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5389999999999996e-06 2024-09-14T05:30:50.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525972+0000", 2024-09-14T05:30:50.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.541e-06 2024-09-14T05:30:50.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617945+0000", 2024-09-14T05:30:50.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091972597000000003 2024-09-14T05:30:50.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617992+0000", 2024-09-14T05:30:50.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7397e-05 2024-09-14T05:30:50.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.618261+0000", 2024-09-14T05:30:50.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026871600000000001 2024-09-14T05:30:50.523 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.524 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.524 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.524 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3640 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6586368~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.524 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:50.524 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.939991644000003, 2024-09-14T05:30:50.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097068580000000002, 2024-09-14T05:30:50.524 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.525 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.525 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.525 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3640 2024-09-14T05:30:50.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.525 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:50.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:50.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:50.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.23e-07 2024-09-14T05:30:50.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526171+0000", 2024-09-14T05:30:50.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:30:50.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526175+0000", 2024-09-14T05:30:50.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8369999999999999e-06 2024-09-14T05:30:50.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526177+0000", 2024-09-14T05:30:50.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4959999999999999e-06 2024-09-14T05:30:50.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622847+0000", 2024-09-14T05:30:50.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096669790000000005 2024-09-14T05:30:50.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622902+0000", 2024-09-14T05:30:50.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5157000000000001e-05 2024-09-14T05:30:50.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623238+0000", 2024-09-14T05:30:50.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033603399999999997 2024-09-14T05:30:50.529 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.529 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.529 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.529 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3641 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7077888~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.529 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:50.529 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.939981134000007, 2024-09-14T05:30:50.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097371141999999994, 2024-09-14T05:30:50.529 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.530 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.530 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.530 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.530 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.530 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.530 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3641 2024-09-14T05:30:50.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.530 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:50.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:50.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531851+0000", 2024-09-14T05:30:50.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005671265 2024-09-14T05:30:50.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531855+0000", 2024-09-14T05:30:50.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0849999999999999e-06 2024-09-14T05:30:50.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531867+0000", 2024-09-14T05:30:50.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1028e-05 2024-09-14T05:30:50.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531872+0000", 2024-09-14T05:30:50.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000003e-06 2024-09-14T05:30:50.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623251+0000", 2024-09-14T05:30:50.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091378980999999998 2024-09-14T05:30:50.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623295+0000", 2024-09-14T05:30:50.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3881999999999998e-05 2024-09-14T05:30:50.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623551+0000", 2024-09-14T05:30:50.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025636099999999998 2024-09-14T05:30:50.534 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.534 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.534 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.534 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3721 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.535 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:50.535 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.934122858999999, 2024-09-14T05:30:50.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096806472000000005, 2024-09-14T05:30:50.535 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.535 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.535 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.535 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.535 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.535 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.535 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3721 2024-09-14T05:30:50.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.535 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:50.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:50.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532039+0000", 2024-09-14T05:30:50.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.05e-07 2024-09-14T05:30:50.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532040+0000", 2024-09-14T05:30:50.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9099999999999991e-07 2024-09-14T05:30:50.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532043+0000", 2024-09-14T05:30:50.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6610000000000002e-06 2024-09-14T05:30:50.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532046+0000", 2024-09-14T05:30:50.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4080000000000001e-06 2024-09-14T05:30:50.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628545+0000", 2024-09-14T05:30:50.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096499615999999996 2024-09-14T05:30:50.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628592+0000", 2024-09-14T05:30:50.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6907000000000003e-05 2024-09-14T05:30:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628845+0000", 2024-09-14T05:30:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025268399999999998 2024-09-14T05:30:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3722 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6193152~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.934112779000003, 2024-09-14T05:30:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097146587000000006, 2024-09-14T05:30:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3722 2024-09-14T05:30:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538191+0000", 2024-09-14T05:30:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061429969999999999 2024-09-14T05:30:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538200+0000", 2024-09-14T05:30:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7069999999999998e-06 2024-09-14T05:30:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538212+0000", 2024-09-14T05:30:50.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1504e-05 2024-09-14T05:30:50.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538219+0000", 2024-09-14T05:30:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7500000000000003e-06 2024-09-14T05:30:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628869+0000", 2024-09-14T05:30:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090649224 2024-09-14T05:30:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628919+0000", 2024-09-14T05:30:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0144e-05 2024-09-14T05:30:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.629195+0000", 2024-09-14T05:30:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027626099999999998 2024-09-14T05:30:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3758 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7110656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.927518563000007, 2024-09-14T05:30:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096977544999999998, 2024-09-14T05:30:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3758 2024-09-14T05:30:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538645+0000", 2024-09-14T05:30:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.843e-06 2024-09-14T05:30:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538651+0000", 2024-09-14T05:30:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6169999999999998e-06 2024-09-14T05:30:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538655+0000", 2024-09-14T05:30:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5399999999999997e-06 2024-09-14T05:30:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635225+0000", 2024-09-14T05:30:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096570149999999993 2024-09-14T05:30:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635272+0000", 2024-09-14T05:30:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7199e-05 2024-09-14T05:30:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635620+0000", 2024-09-14T05:30:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034782600000000003 2024-09-14T05:30:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3759 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.927500652000006, 2024-09-14T05:30:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097335331999999997, 2024-09-14T05:30:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3759 2024-09-14T05:30:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544013+0000", 2024-09-14T05:30:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053520829999999997 2024-09-14T05:30:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544016+0000", 2024-09-14T05:30:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6660000000000001e-06 2024-09-14T05:30:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544026+0000", 2024-09-14T05:30:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7089999999999997e-06 2024-09-14T05:30:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544043+0000", 2024-09-14T05:30:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833999999999998e-05 2024-09-14T05:30:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635646+0000", 2024-09-14T05:30:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091603544999999995 2024-09-14T05:30:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635696+0000", 2024-09-14T05:30:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9307000000000001e-05 2024-09-14T05:30:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635996+0000", 2024-09-14T05:30:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030018800000000001 2024-09-14T05:30:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3838 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6225920~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.921838972000003, 2024-09-14T05:30:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096894235999999995, 2024-09-14T05:30:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3838 2024-09-14T05:30:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544323+0000", 2024-09-14T05:30:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-14T05:30:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544325+0000", 2024-09-14T05:30:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.75e-06 2024-09-14T05:30:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544331+0000", 2024-09-14T05:30:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5019999999999998e-06 2024-09-14T05:30:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544335+0000", 2024-09-14T05:30:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2989999999999998e-06 2024-09-14T05:30:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640869+0000", 2024-09-14T05:30:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096533351000000003 2024-09-14T05:30:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640919+0000", 2024-09-14T05:30:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999999e-05 2024-09-14T05:30:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641217+0000", 2024-09-14T05:30:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029717600000000002 2024-09-14T05:30:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3839 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.921820034000007, 2024-09-14T05:30:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097240339999999995, 2024-09-14T05:30:50.563 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.563 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.563 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.563 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3839 2024-09-14T05:30:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549834+0000", 2024-09-14T05:30:50.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054927819999999999 2024-09-14T05:30:50.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549838+0000", 2024-09-14T05:30:50.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8689999999999997e-06 2024-09-14T05:30:50.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549848+0000", 2024-09-14T05:30:50.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0154e-05 2024-09-14T05:30:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549853+0000", 2024-09-14T05:30:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9740000000000001e-06 2024-09-14T05:30:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641245+0000", 2024-09-14T05:30:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091392245999999996 2024-09-14T05:30:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641295+0000", 2024-09-14T05:30:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0018999999999997e-05 2024-09-14T05:30:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641582+0000", 2024-09-14T05:30:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028629600000000001 2024-09-14T05:30:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3868 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.916157455999993, 2024-09-14T05:30:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096859555999999999, 2024-09-14T05:30:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3868 2024-09-14T05:30:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556173+0000", 2024-09-14T05:30:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006169038 2024-09-14T05:30:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556177+0000", 2024-09-14T05:30:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0010000000000003e-06 2024-09-14T05:30:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556188+0000", 2024-09-14T05:30:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0675e-05 2024-09-14T05:30:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556199+0000", 2024-09-14T05:30:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.155e-05 2024-09-14T05:30:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646511+0000", 2024-09-14T05:30:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090312187000000002 2024-09-14T05:30:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646563+0000", 2024-09-14T05:30:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2219000000000003e-05 2024-09-14T05:30:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646863+0000", 2024-09-14T05:30:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029988600000000002 2024-09-14T05:30:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:50.576 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:50.577 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:30:50.577 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:50.577+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:50.596 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:30:50.677 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:50.677+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:50.717 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:50.717 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:50.720 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:50.720 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:50.720 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:50.721 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:50.729 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:30:50.733 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:50.733 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:50.749 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:50.749 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:50.765 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:50.765 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:50.778 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:50.778+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:50.781 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:50.782 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:50.798 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:50.798 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:50.815 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:50.815 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:50.832 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:50.833 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:50.834 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:50.834 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:50.843 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:30:50.887 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:50.887+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:50.946 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:50.947 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:50.955 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:30:50.980 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:50.979+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:51.080 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:51.080+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:51.102 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:51.113 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:30:51.199 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:51.198+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:51.261 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:51.261 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:51.269 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:30:51.283 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:51.282+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:51.383 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:51.383+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:51.412 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:51.412 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:51.421 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:30:51.484 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:51.484+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:51.525 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:51.525 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:51.535 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:30:51.619 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:51.618+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:51.678 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:51.678 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:51.687 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:30:51.687 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:51.686+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:51.788 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:51.787+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:51.830 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:51.831 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:51.839 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:30:51.900 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:51.899+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:52.001 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:52.000+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:52.008 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:52.008 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:52.018 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:30:52.103 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T05:30:52.106 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:52.104+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:52.174 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:52.174 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:52.185 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:30:52.229 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:52.228+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:52.299 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:52.299 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:52.302 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:52.302+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:52.308 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:30:52.404 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:52.403+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:52.452 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:52.452 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:52.461 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:30:52.545 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:52.544+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:52.604 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:52.604 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:52.605 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:52.604+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:52.613 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:30:52.706 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:52.706+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:52.757 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:52.757 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:52.765 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:30:52.807 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:52.807+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:52.844 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:30:52.869 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:52.869 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:52.879 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:30:52.908 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:52.908+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:52.967 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:52.967 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:52.984 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:52.984 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:53.000 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:53.000 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:53.009 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:53.009+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:53.017 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:53.017 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:53.025 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:53.025 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:53.025 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:53.025 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:53.025 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:53.034 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:30:53.034 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:53.034 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:53.051 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:53.051 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:53.068 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:53.068 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:53.085 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:53.085 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:53.109 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-14T05:30:53.109 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:53.108+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:53.132 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T05:30:53.132 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T05:30:53.132 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T05:30:53.132 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 1.000000 2024-09-14T05:30:53.133 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-14T05:30:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:53.146 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:53.146 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:53.146 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:53.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.147 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73760 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2848768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.147 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:53.147 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.585355700000001, 2024-09-14T05:30:53.147 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.04469911, 2024-09-14T05:30:53.147 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.147 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.147 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.147 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.147 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.147 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.147 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73760 2024-09-14T05:30:53.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.148 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.148 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:53.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.148 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.148 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:53.148 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:30:53.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:53.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559701+0000", 2024-09-14T05:30:53.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1909999999999999e-06 2024-09-14T05:30:53.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.149 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.149 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559708+0000", 2024-09-14T05:30:53.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7279999999999998e-06 2024-09-14T05:30:53.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.150 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.150 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559713+0000", 2024-09-14T05:30:53.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8389999999999997e-06 2024-09-14T05:30:53.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.150 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.150 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604310+0000", 2024-09-14T05:30:53.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044597664000000002 2024-09-14T05:30:53.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.150 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604352+0000", 2024-09-14T05:30:53.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1236000000000003e-05 2024-09-14T05:30:53.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.151 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604398+0000", 2024-09-14T05:30:53.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6091000000000003e-05 2024-09-14T05:30:53.151 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.151 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.151 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.151 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73773 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2801664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.151 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:53.152 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.585335530999998, 2024-09-14T05:30:53.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941026000000002, 2024-09-14T05:30:53.152 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.152 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.152 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.152 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.152 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.152 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.152 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73773 2024-09-14T05:30:53.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.152 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:53.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.153 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:53.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.153 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:30:53.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8599999999999999e-07 2024-09-14T05:30:53.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.154 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559721+0000", 2024-09-14T05:30:53.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0540000000000002e-06 2024-09-14T05:30:53.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.154 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559729+0000", 2024-09-14T05:30:53.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1359999999999997e-06 2024-09-14T05:30:53.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.154 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559734+0000", 2024-09-14T05:30:53.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2140000000000004e-06 2024-09-14T05:30:53.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.155 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.155 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604441+0000", 2024-09-14T05:30:53.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044706828999999997 2024-09-14T05:30:53.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.155 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.155 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604522+0000", 2024-09-14T05:30:53.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0603000000000001e-05 2024-09-14T05:30:53.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604660+0000", 2024-09-14T05:30:53.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000137804 2024-09-14T05:30:53.156 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.156 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.156 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.156 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73777 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2834432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.156 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:53.156 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.585313130000003, 2024-09-14T05:30:53.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045116270999999999, 2024-09-14T05:30:53.156 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.157 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.157 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.157 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.157 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.157 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.157 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73777 2024-09-14T05:30:53.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.157 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:53.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:30:53.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559742+0000", 2024-09-14T05:30:53.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4099999999999999e-07 2024-09-14T05:30:53.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.158 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.158 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559744+0000", 2024-09-14T05:30:53.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0040000000000002e-06 2024-09-14T05:30:53.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559749+0000", 2024-09-14T05:30:53.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9739999999999999e-06 2024-09-14T05:30:53.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559754+0000", 2024-09-14T05:30:53.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9540000000000003e-06 2024-09-14T05:30:53.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604695+0000", 2024-09-14T05:30:53.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941003 2024-09-14T05:30:53.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604766+0000", 2024-09-14T05:30:53.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0296999999999995e-05 2024-09-14T05:30:53.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604857+0000", 2024-09-14T05:30:53.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1598e-05 2024-09-14T05:30:53.160 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.160 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.160 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.161 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73790 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.161 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:53.161 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.585293194999998, 2024-09-14T05:30:53.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316437000000001, 2024-09-14T05:30:53.161 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.161 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.161 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.161 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.161 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.161 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.161 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73790 2024-09-14T05:30:53.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.162 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:53.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:53.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:30:53.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7599999999999998e-07 2024-09-14T05:30:53.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559763+0000", 2024-09-14T05:30:53.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0880000000000002e-06 2024-09-14T05:30:53.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559769+0000", 2024-09-14T05:30:53.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6740000000000002e-06 2024-09-14T05:30:53.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559774+0000", 2024-09-14T05:30:53.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6110000000000004e-06 2024-09-14T05:30:53.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604905+0000", 2024-09-14T05:30:53.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045130827999999998 2024-09-14T05:30:53.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604982+0000", 2024-09-14T05:30:53.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7410999999999995e-05 2024-09-14T05:30:53.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605077+0000", 2024-09-14T05:30:53.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5449e-05 2024-09-14T05:30:53.165 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.165 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.165 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.165 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73793 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2836480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.165 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:53.166 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.585273991000001, 2024-09-14T05:30:53.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045441212000000002, 2024-09-14T05:30:53.166 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.166 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.166 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.166 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.166 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.166 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.166 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73793 2024-09-14T05:30:53.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.166 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.167 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:53.167 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.167 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.167 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:30:53.167 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.167 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.167 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559781+0000", 2024-09-14T05:30:53.167 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2899999999999999e-07 2024-09-14T05:30:53.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.168 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.168 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559783+0000", 2024-09-14T05:30:53.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.075e-06 2024-09-14T05:30:53.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.168 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.168 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559788+0000", 2024-09-14T05:30:53.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5369999999999997e-06 2024-09-14T05:30:53.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.168 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.168 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559793+0000", 2024-09-14T05:30:53.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:30:53.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605109+0000", 2024-09-14T05:30:53.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316229999999999 2024-09-14T05:30:53.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605168+0000", 2024-09-14T05:30:53.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8949000000000001e-05 2024-09-14T05:30:53.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605221+0000", 2024-09-14T05:30:53.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3260000000000002e-05 2024-09-14T05:30:53.170 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.170 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.170 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.170 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73806 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.170 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:53.170 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.585255365000002, 2024-09-14T05:30:53.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045526189000000002, 2024-09-14T05:30:53.170 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.170 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.170 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.171 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.171 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.171 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.171 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73806 2024-09-14T05:30:53.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.171 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.171 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.171 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:53.171 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.171 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.171 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:53.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.172 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.172 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:30:53.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:30:53.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.172 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.172 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559801+0000", 2024-09-14T05:30:53.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.995e-06 2024-09-14T05:30:53.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.172 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559807+0000", 2024-09-14T05:30:53.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5450000000000003e-06 2024-09-14T05:30:53.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559811+0000", 2024-09-14T05:30:53.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5090000000000004e-06 2024-09-14T05:30:53.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605237+0000", 2024-09-14T05:30:53.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045426133 2024-09-14T05:30:53.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605278+0000", 2024-09-14T05:30:53.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1084999999999997e-05 2024-09-14T05:30:53.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605325+0000", 2024-09-14T05:30:53.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6582999999999998e-05 2024-09-14T05:30:53.174 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.174 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.174 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.175 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73809 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2838528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.175 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:53.175 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.58523186, 2024-09-14T05:30:53.175 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045596062999999999, 2024-09-14T05:30:53.175 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.175 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.175 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.175 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.175 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.175 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.175 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73809 2024-09-14T05:30:53.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.175 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.176 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.176 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:53.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.176 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.176 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:30:53.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.176 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.176 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559823+0000", 2024-09-14T05:30:53.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.08e-07 2024-09-14T05:30:53.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.177 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559825+0000", 2024-09-14T05:30:53.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.26e-06 2024-09-14T05:30:53.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.177 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559831+0000", 2024-09-14T05:30:53.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6690000000000003e-06 2024-09-14T05:30:53.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.177 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559835+0000", 2024-09-14T05:30:53.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5120000000000002e-06 2024-09-14T05:30:53.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605342+0000", 2024-09-14T05:30:53.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045506957000000001 2024-09-14T05:30:53.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605375+0000", 2024-09-14T05:30:53.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3250999999999997e-05 2024-09-14T05:30:53.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605418+0000", 2024-09-14T05:30:53.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3006000000000001e-05 2024-09-14T05:30:53.179 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.179 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.179 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.179 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73822 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2807808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.179 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:53.179 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.585204206999997, 2024-09-14T05:30:53.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045661405000000002, 2024-09-14T05:30:53.179 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.179 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.179 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.179 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.180 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.180 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.180 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73822 2024-09-14T05:30:53.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.180 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.180 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:53.180 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.180 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:30:53.180 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.181 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559851+0000", 2024-09-14T05:30:53.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4000000000000002e-07 2024-09-14T05:30:53.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.181 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559853+0000", 2024-09-14T05:30:53.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5380000000000001e-06 2024-09-14T05:30:53.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.181 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559859+0000", 2024-09-14T05:30:53.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0619999999999997e-06 2024-09-14T05:30:53.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559864+0000", 2024-09-14T05:30:53.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.977e-06 2024-09-14T05:30:53.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605435+0000", 2024-09-14T05:30:53.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045570672 2024-09-14T05:30:53.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.183 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605467+0000", 2024-09-14T05:30:53.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2486e-05 2024-09-14T05:30:53.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.183 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.183 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605511+0000", 2024-09-14T05:30:53.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4129999999999999e-05 2024-09-14T05:30:53.183 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.183 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.183 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.183 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73825 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2840576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.183 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:53.184 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.585180843000003, 2024-09-14T05:30:53.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045728694, 2024-09-14T05:30:53.184 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.184 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.184 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.184 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.184 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.184 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.184 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73825 2024-09-14T05:30:53.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.184 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:53.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:30:53.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559874+0000", 2024-09-14T05:30:53.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5400000000000002e-07 2024-09-14T05:30:53.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559876+0000", 2024-09-14T05:30:53.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.131e-06 2024-09-14T05:30:53.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559888+0000", 2024-09-14T05:30:53.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2216e-05 2024-09-14T05:30:53.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559893+0000", 2024-09-14T05:30:53.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6539999999999998e-06 2024-09-14T05:30:53.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605526+0000", 2024-09-14T05:30:53.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045633119 2024-09-14T05:30:53.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605560+0000", 2024-09-14T05:30:53.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3825999999999998e-05 2024-09-14T05:30:53.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605602+0000", 2024-09-14T05:30:53.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2293999999999999e-05 2024-09-14T05:30:53.188 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.188 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.188 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.188 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74023 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2863104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.188 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:53.188 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.583994867999998, 2024-09-14T05:30:53.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044635385999999999, 2024-09-14T05:30:53.188 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.188 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.188 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.189 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.189 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.189 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.189 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74023 2024-09-14T05:30:53.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.189 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:53.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:30:53.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.190 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.190 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561060+0000", 2024-09-14T05:30:53.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0699999999999998e-07 2024-09-14T05:30:53.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.190 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.190 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561072+0000", 2024-09-14T05:30:53.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2261e-05 2024-09-14T05:30:53.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.190 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561079+0000", 2024-09-14T05:30:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8399999999999997e-06 2024-09-14T05:30:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561084+0000", 2024-09-14T05:30:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7380000000000002e-06 2024-09-14T05:30:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605620+0000", 2024-09-14T05:30:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044536146999999998 2024-09-14T05:30:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605653+0000", 2024-09-14T05:30:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2969000000000002e-05 2024-09-14T05:30:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605695+0000", 2024-09-14T05:30:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2024000000000003e-05 2024-09-14T05:30:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.193 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74026 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2895872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.193 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:53.193 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.583963406000002, 2024-09-14T05:30:53.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044692717, 2024-09-14T05:30:53.193 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.193 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.193 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.193 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.193 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.193 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.193 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74026 2024-09-14T05:30:53.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.193 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.194 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.194 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:53.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.194 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.194 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:53.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.194 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.194 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:30:53.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:30:53.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561093+0000", 2024-09-14T05:30:53.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-14T05:30:53.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561099+0000", 2024-09-14T05:30:53.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0399999999999998e-06 2024-09-14T05:30:53.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561104+0000", 2024-09-14T05:30:53.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8180000000000003e-06 2024-09-14T05:30:53.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.196 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605710+0000", 2024-09-14T05:30:53.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044605657 2024-09-14T05:30:53.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.196 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605742+0000", 2024-09-14T05:30:53.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1949000000000002e-05 2024-09-14T05:30:53.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.196 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605784+0000", 2024-09-14T05:30:53.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1884000000000001e-05 2024-09-14T05:30:53.196 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.197 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.197 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.197 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74039 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.197 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:53.197 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.583836808000001, 2024-09-14T05:30:53.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044654905000000002, 2024-09-14T05:30:53.197 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.197 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.197 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.197 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.197 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.197 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.198 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74039 2024-09-14T05:30:53.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.198 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:53.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:30:53.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561218+0000", 2024-09-14T05:30:53.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4000000000000001e-07 2024-09-14T05:30:53.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561221+0000", 2024-09-14T05:30:53.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9730000000000002e-06 2024-09-14T05:30:53.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561229+0000", 2024-09-14T05:30:53.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1759999999999993e-06 2024-09-14T05:30:53.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561235+0000", 2024-09-14T05:30:53.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0109999999999997e-06 2024-09-14T05:30:53.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605798+0000", 2024-09-14T05:30:53.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044563285000000001 2024-09-14T05:30:53.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605831+0000", 2024-09-14T05:30:53.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2054999999999997e-05 2024-09-14T05:30:53.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.201 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.201 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605872+0000", 2024-09-14T05:30:53.201 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1764999999999997e-05 2024-09-14T05:30:53.201 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.201 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.201 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.201 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74042 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.201 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:53.201 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.583788355000003, 2024-09-14T05:30:53.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044773234000000002, 2024-09-14T05:30:53.202 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.202 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.202 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.202 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.202 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.202 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.202 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74042 2024-09-14T05:30:53.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.202 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:53.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:53.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:30:53.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4099999999999998e-07 2024-09-14T05:30:53.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561269+0000", 2024-09-14T05:30:53.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7930000000000002e-06 2024-09-14T05:30:53.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561278+0000", 2024-09-14T05:30:53.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5150000000000006e-06 2024-09-14T05:30:53.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561284+0000", 2024-09-14T05:30:53.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2029999999999997e-06 2024-09-14T05:30:53.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605934+0000", 2024-09-14T05:30:53.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044650449000000002 2024-09-14T05:30:53.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605983+0000", 2024-09-14T05:30:53.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8943999999999998e-05 2024-09-14T05:30:53.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.205 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606039+0000", 2024-09-14T05:30:53.206 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5689000000000002e-05 2024-09-14T05:30:53.206 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.206 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.206 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.206 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74052 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.206 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:53.206 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.58376148, 2024-09-14T05:30:53.206 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044830474000000002, 2024-09-14T05:30:53.206 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.206 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.206 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.206 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.207 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.207 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.207 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74052 2024-09-14T05:30:53.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.207 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:53.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:53.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:30:53.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1099999999999996e-07 2024-09-14T05:30:53.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561295+0000", 2024-09-14T05:30:53.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7770000000000001e-06 2024-09-14T05:30:53.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561303+0000", 2024-09-14T05:30:53.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6860000000000006e-06 2024-09-14T05:30:53.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561308+0000", 2024-09-14T05:30:53.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.023e-06 2024-09-14T05:30:53.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605893+0000", 2024-09-14T05:30:53.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044585458000000001 2024-09-14T05:30:53.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.210 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605933+0000", 2024-09-14T05:30:53.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9551999999999997e-05 2024-09-14T05:30:53.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.210 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.210 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606123+0000", 2024-09-14T05:30:53.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019046699999999999 2024-09-14T05:30:53.210 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.210 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.210 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.210 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74054 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.210 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:53.211 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.583739453, 2024-09-14T05:30:53.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093610999999999, 2024-09-14T05:30:53.211 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.211 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.211 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.211 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.211 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.211 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.211 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74054 2024-09-14T05:30:53.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.211 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:53.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:53.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:30:53.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9699999999999996e-07 2024-09-14T05:30:53.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561317+0000", 2024-09-14T05:30:53.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9740000000000001e-06 2024-09-14T05:30:53.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561324+0000", 2024-09-14T05:30:53.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4590000000000003e-06 2024-09-14T05:30:53.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561329+0000", 2024-09-14T05:30:53.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5969999999999999e-06 2024-09-14T05:30:53.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606142+0000", 2024-09-14T05:30:53.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044812347000000002 2024-09-14T05:30:53.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606184+0000", 2024-09-14T05:30:53.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2441999999999998e-05 2024-09-14T05:30:53.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606408+0000", 2024-09-14T05:30:53.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022429500000000001 2024-09-14T05:30:53.215 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.215 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.215 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.215 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74056 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.215 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:53.215 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.583544750000001, 2024-09-14T05:30:53.215 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045191685000000002, 2024-09-14T05:30:53.215 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.215 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.215 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.215 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.215 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.215 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.216 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74056 2024-09-14T05:30:53.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.216 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:53.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:30:53.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561510+0000", 2024-09-14T05:30:53.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2399999999999998e-07 2024-09-14T05:30:53.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561513+0000", 2024-09-14T05:30:53.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7219999999999999e-06 2024-09-14T05:30:53.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561524+0000", 2024-09-14T05:30:53.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1357e-05 2024-09-14T05:30:53.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561530+0000", 2024-09-14T05:30:53.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2890000000000003e-06 2024-09-14T05:30:53.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606438+0000", 2024-09-14T05:30:53.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044907326999999997 2024-09-14T05:30:53.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606520+0000", 2024-09-14T05:30:53.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1924999999999999e-05 2024-09-14T05:30:53.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.219 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.219 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606701+0000", 2024-09-14T05:30:53.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00018144100000000001 2024-09-14T05:30:53.219 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.219 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.219 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.219 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74058 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.219 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:53.219 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.583066076999998, 2024-09-14T05:30:53.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093368000000002, 2024-09-14T05:30:53.220 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.220 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.220 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.220 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.220 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.220 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.220 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74058 2024-09-14T05:30:53.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.220 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:53.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:30:53.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561989+0000", 2024-09-14T05:30:53.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3300000000000001e-07 2024-09-14T05:30:53.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561993+0000", 2024-09-14T05:30:53.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1790000000000001e-06 2024-09-14T05:30:53.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562007+0000", 2024-09-14T05:30:53.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3434e-05 2024-09-14T05:30:53.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562014+0000", 2024-09-14T05:30:53.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.537e-06 2024-09-14T05:30:53.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606736+0000", 2024-09-14T05:30:53.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044722117999999998 2024-09-14T05:30:53.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606829+0000", 2024-09-14T05:30:53.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2410999999999993e-05 2024-09-14T05:30:53.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.223 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.223 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607082+0000", 2024-09-14T05:30:53.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025285599999999998 2024-09-14T05:30:53.223 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.223 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.223 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.224 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74059 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.224 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:53.224 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.583008086, 2024-09-14T05:30:53.224 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045479928000000003, 2024-09-14T05:30:53.224 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.224 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.224 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.224 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.224 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.224 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.224 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74059 2024-09-14T05:30:53.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.225 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:53.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:30:53.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562047+0000", 2024-09-14T05:30:53.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6899999999999997e-07 2024-09-14T05:30:53.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562050+0000", 2024-09-14T05:30:53.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.943e-06 2024-09-14T05:30:53.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562058+0000", 2024-09-14T05:30:53.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0120000000000005e-06 2024-09-14T05:30:53.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562063+0000", 2024-09-14T05:30:53.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:30:53.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607186+0000", 2024-09-14T05:30:53.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045122348999999999 2024-09-14T05:30:53.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607265+0000", 2024-09-14T05:30:53.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9462999999999995e-05 2024-09-14T05:30:53.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.228 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607526+0000", 2024-09-14T05:30:53.228 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026090400000000001 2024-09-14T05:30:53.228 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.228 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.228 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.228 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.228 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74060 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2850816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:53.228 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:53.228 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 63.582982708999999, 2024-09-14T05:30:53.228 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045200092999999997, 2024-09-14T05:30:53.228 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.228 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.229 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.229 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.229 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.229 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.229 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74060 2024-09-14T05:30:53.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.229 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:53.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:53.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:30:53.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5200000000000002e-07 2024-09-14T05:30:53.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562075+0000", 2024-09-14T05:30:53.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5200000000000002e-06 2024-09-14T05:30:53.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562082+0000", 2024-09-14T05:30:53.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8929999999999999e-06 2024-09-14T05:30:53.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562088+0000", 2024-09-14T05:30:53.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5570000000000003e-06 2024-09-14T05:30:53.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.232 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607111+0000", 2024-09-14T05:30:53.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045023464999999999 2024-09-14T05:30:53.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.232 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.232 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607190+0000", 2024-09-14T05:30:53.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8721999999999996e-05 2024-09-14T05:30:53.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.233 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.233 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607272+0000", 2024-09-14T05:30:53.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1483999999999994e-05 2024-09-14T05:30:53.233 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.233 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.233 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.233 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:136816 3.7 3:e5cc52c3:::rbd_id.image_client.0-clone31:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-14T05:30:53.233 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:30.438431+0000", 2024-09-14T05:30:53.233 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 22.706623006000001, 2024-09-14T05:30:53.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.96261680299999997, 2024-09-14T05:30:53.233 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.233 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.234 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:53.234 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.234 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.234 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.234 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 136816 2024-09-14T05:30:53.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.234 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438431+0000", 2024-09-14T05:30:53.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438431+0000", 2024-09-14T05:30:53.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438432+0000", 2024-09-14T05:30:53.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0419999999999999e-06 2024-09-14T05:30:53.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438437+0000", 2024-09-14T05:30:53.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5000000000000001e-06 2024-09-14T05:30:53.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438446+0000", 2024-09-14T05:30:53.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5740000000000002e-06 2024-09-14T05:30:53.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438456+0000", 2024-09-14T05:30:53.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0056e-05 2024-09-14T05:30:53.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.236 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.236 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438500+0000", 2024-09-14T05:30:53.236 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3738e-05 2024-09-14T05:30:53.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.237 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:30:53.237 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438506+0000", 2024-09-14T05:30:53.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1659999999999999e-06 2024-09-14T05:30:53.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.237 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.237 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.399222+0000", 2024-09-14T05:30:53.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.96071578999999996 2024-09-14T05:30:53.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.237 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.237 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.399381+0000", 2024-09-14T05:30:53.237 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000158834 2024-09-14T05:30:53.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.238 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-14T05:30:53.238 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.399530+0000", 2024-09-14T05:30:53.238 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014862400000000001 2024-09-14T05:30:53.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.238 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:30:53.238 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.400507+0000", 2024-09-14T05:30:53.238 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0009779369999999999 2024-09-14T05:30:53.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.238 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:53.238 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.400969+0000", 2024-09-14T05:30:53.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046119800000000001 2024-09-14T05:30:53.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:30:53.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.401006+0000", 2024-09-14T05:30:53.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7594e-05 2024-09-14T05:30:53.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.239 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.239 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.401048+0000", 2024-09-14T05:30:53.239 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1749999999999998e-05 2024-09-14T05:30:53.239 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.239 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.239 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.240 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.240 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.240 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:53.241 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:30:53.243 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:53.241+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:53.342 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:53.342+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:53.392 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:53.392 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:53.392 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:53.392 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:53.401 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:30:53.444 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:53.444+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:53.505 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:53.505 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:53.514 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:30:53.556 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:53.555+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:53.645 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:53.645+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:53.659 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:53.659 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:53.659 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:53.659 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:53.659 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.659 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4235.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-14T05:30:53.659 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:30:53.659 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 99.817840746000002, 2024-09-14T05:30:53.659 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12887758399999999, 2024-09-14T05:30:53.659 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.659 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.660 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:30:53.660 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.660 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4235", 2024-09-14T05:30:53.660 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/3211559243", 2024-09-14T05:30:53.660 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 2 2024-09-14T05:30:53.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.660 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.660 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.660 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.660 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:30:53.660 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.660 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.661 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.661 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:30:53.661 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.661 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.661 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839833+0000", 2024-09-14T05:30:53.661 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:30:53.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.661 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.661 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839838+0000", 2024-09-14T05:30:53.661 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2440000000000001e-06 2024-09-14T05:30:53.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.662 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.662 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839848+0000", 2024-09-14T05:30:53.662 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0030000000000001e-05 2024-09-14T05:30:53.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.662 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.662 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839854+0000", 2024-09-14T05:30:53.662 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3750000000000002e-06 2024-09-14T05:30:53.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.662 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.662 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839908+0000", 2024-09-14T05:30:53.662 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3965999999999998e-05 2024-09-14T05:30:53.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.663 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:30:53.663 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839914+0000", 2024-09-14T05:30:53.663 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6139999999999997e-06 2024-09-14T05:30:53.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.663 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.663 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968558+0000", 2024-09-14T05:30:53.663 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12864361199999999 2024-09-14T05:30:53.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.663 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.663 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968710+0000", 2024-09-14T05:30:53.663 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015177300000000001 2024-09-14T05:30:53.664 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.664 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.664 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.664 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27764 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 188416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.664 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:30:53.664 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.446514363999995, 2024-09-14T05:30:53.664 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080826920999999996, 2024-09-14T05:30:53.664 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.664 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.664 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.664 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.664 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.665 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.665 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27764 2024-09-14T05:30:53.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.665 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.665 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.665 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:30:53.665 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.665 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.665 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:30:53.665 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.666 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.666 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211159+0000", 2024-09-14T05:30:53.666 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1800000000000001e-07 2024-09-14T05:30:53.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.666 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.666 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211161+0000", 2024-09-14T05:30:53.666 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.968e-06 2024-09-14T05:30:53.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.666 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.666 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211169+0000", 2024-09-14T05:30:53.666 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4800000000000001e-06 2024-09-14T05:30:53.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.667 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.667 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211174+0000", 2024-09-14T05:30:53.667 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1880000000000001e-06 2024-09-14T05:30:53.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.667 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.667 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291670+0000", 2024-09-14T05:30:53.667 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080495199000000003 2024-09-14T05:30:53.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.667 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.667 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291775+0000", 2024-09-14T05:30:53.668 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000105375 2024-09-14T05:30:53.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.668 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.668 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.668 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291985+0000", 2024-09-14T05:30:53.668 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021029299999999999 2024-09-14T05:30:53.668 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.668 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.668 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.668 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.668 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27765 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.668 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:30:53.669 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.446491613999996, 2024-09-14T05:30:53.669 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081115197999999999, 2024-09-14T05:30:53.669 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.669 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.669 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.669 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.669 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.669 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.669 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27765 2024-09-14T05:30:53.669 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.669 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.669 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.669 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.669 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:30:53.669 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.670 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:30:53.670 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.670 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211182+0000", 2024-09-14T05:30:53.670 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3599999999999999e-07 2024-09-14T05:30:53.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.670 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211184+0000", 2024-09-14T05:30:53.671 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.03e-06 2024-09-14T05:30:53.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.671 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.671 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.671 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211192+0000", 2024-09-14T05:30:53.671 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2309999999999996e-06 2024-09-14T05:30:53.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.671 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.671 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.671 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211197+0000", 2024-09-14T05:30:53.671 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4199999999999998e-06 2024-09-14T05:30:53.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.671 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.671 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.672 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292017+0000", 2024-09-14T05:30:53.672 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080819516999999994 2024-09-14T05:30:53.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.672 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.672 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292087+0000", 2024-09-14T05:30:53.672 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0056999999999997e-05 2024-09-14T05:30:53.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.672 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.672 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292296+0000", 2024-09-14T05:30:53.672 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000209507 2024-09-14T05:30:53.672 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.672 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.673 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.673 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.673 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27766 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.673 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:30:53.673 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.446468400000001, 2024-09-14T05:30:53.673 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081359077000000002, 2024-09-14T05:30:53.673 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.673 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.673 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.673 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.673 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.673 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.673 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27766 2024-09-14T05:30:53.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.674 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.674 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.674 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:30:53.674 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.674 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.674 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:30:53.674 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.674 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.675 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211205+0000", 2024-09-14T05:30:53.675 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7e-07 2024-09-14T05:30:53.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.675 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.675 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211207+0000", 2024-09-14T05:30:53.675 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4540000000000001e-06 2024-09-14T05:30:53.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.675 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.675 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211216+0000", 2024-09-14T05:30:53.675 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3680000000000002e-06 2024-09-14T05:30:53.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.676 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.676 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211221+0000", 2024-09-14T05:30:53.676 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3499999999999996e-06 2024-09-14T05:30:53.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.676 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.676 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.676 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292329+0000", 2024-09-14T05:30:53.676 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081108240999999998 2024-09-14T05:30:53.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.676 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.676 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.676 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292370+0000", 2024-09-14T05:30:53.676 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.121e-05 2024-09-14T05:30:53.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.677 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.677 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292563+0000", 2024-09-14T05:30:53.677 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019298399999999999 2024-09-14T05:30:53.677 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.677 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.677 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.677 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27767 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.677 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:53.677 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.446444885999995, 2024-09-14T05:30:53.677 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081623119999999993, 2024-09-14T05:30:53.677 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.678 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.678 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.678 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.678 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.678 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.678 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27767 2024-09-14T05:30:53.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.678 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.678 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.678 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.678 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:53.678 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.678 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.678 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.679 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:53.679 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.679 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.679 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:30:53.679 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0499999999999999e-07 2024-09-14T05:30:53.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.679 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.679 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211230+0000", 2024-09-14T05:30:53.679 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0700000000000001e-06 2024-09-14T05:30:53.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.680 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.680 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211236+0000", 2024-09-14T05:30:53.680 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8429999999999998e-06 2024-09-14T05:30:53.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.680 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.680 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.680 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211241+0000", 2024-09-14T05:30:53.680 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:30:53.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.680 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.680 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.680 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292594+0000", 2024-09-14T05:30:53.680 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081352868999999994 2024-09-14T05:30:53.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.681 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.681 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292631+0000", 2024-09-14T05:30:53.681 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7083999999999999e-05 2024-09-14T05:30:53.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.681 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.681 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292851+0000", 2024-09-14T05:30:53.681 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022011700000000001 2024-09-14T05:30:53.681 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.681 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.681 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.682 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27768 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 274432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.682 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:30:53.682 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.446426349999996, 2024-09-14T05:30:53.682 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081875032, 2024-09-14T05:30:53.682 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.682 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.682 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.682 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.682 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.682 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.682 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27768 2024-09-14T05:30:53.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.682 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.682 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.683 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.683 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:30:53.683 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.683 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.683 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.683 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:30:53.683 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.683 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.683 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.683 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211247+0000", 2024-09-14T05:30:53.683 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.46e-07 2024-09-14T05:30:53.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.684 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.684 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.684 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211248+0000", 2024-09-14T05:30:53.684 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7349999999999999e-06 2024-09-14T05:30:53.684 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.684 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.684 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.684 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211253+0000", 2024-09-14T05:30:53.684 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.899e-06 2024-09-14T05:30:53.684 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.684 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.684 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.684 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211258+0000", 2024-09-14T05:30:53.684 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2810000000000002e-06 2024-09-14T05:30:53.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.685 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.685 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.685 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292888+0000", 2024-09-14T05:30:53.685 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081629955000000004 2024-09-14T05:30:53.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.685 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.685 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.685 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292926+0000", 2024-09-14T05:30:53.685 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7947000000000001e-05 2024-09-14T05:30:53.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.685 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.685 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.685 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293121+0000", 2024-09-14T05:30:53.686 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019586899999999999 2024-09-14T05:30:53.686 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.686 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.686 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.686 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.686 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27769 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 294912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.686 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:30:53.686 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.446408261000002, 2024-09-14T05:30:53.686 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082181899000000003, 2024-09-14T05:30:53.686 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.686 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.686 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.687 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.687 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.687 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.687 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27769 2024-09-14T05:30:53.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.687 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.687 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.687 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.687 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:30:53.687 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.687 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.687 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.687 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:30:53.688 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.688 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.688 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211265+0000", 2024-09-14T05:30:53.688 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2199999999999999e-07 2024-09-14T05:30:53.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.688 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.688 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211267+0000", 2024-09-14T05:30:53.688 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2239999999999998e-06 2024-09-14T05:30:53.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.688 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.689 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211273+0000", 2024-09-14T05:30:53.689 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6540000000000004e-06 2024-09-14T05:30:53.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.689 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.689 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211277+0000", 2024-09-14T05:30:53.689 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6299999999999997e-06 2024-09-14T05:30:53.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.689 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.690 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293150+0000", 2024-09-14T05:30:53.690 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081872575000000003 2024-09-14T05:30:53.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.692 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.692 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.693 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293245+0000", 2024-09-14T05:30:53.693 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5327000000000004e-05 2024-09-14T05:30:53.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.693 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.693 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.693 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293446+0000", 2024-09-14T05:30:53.693 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000201067 2024-09-14T05:30:53.693 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.693 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.693 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.693 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.693 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27770 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.693 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:53.694 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.446388099000004, 2024-09-14T05:30:53.694 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082663707000000003, 2024-09-14T05:30:53.694 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.694 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.694 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.694 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.694 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.694 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.694 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27770 2024-09-14T05:30:53.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.694 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.694 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.694 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.694 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:53.695 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.695 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.695 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.695 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:53.695 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.695 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.695 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.695 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:30:53.695 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1199999999999998e-07 2024-09-14T05:30:53.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.695 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.695 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.695 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211292+0000", 2024-09-14T05:30:53.696 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1609999999999997e-06 2024-09-14T05:30:53.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.696 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.696 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.696 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211299+0000", 2024-09-14T05:30:53.696 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3300000000000004e-06 2024-09-14T05:30:53.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.696 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.696 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.696 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211303+0000", 2024-09-14T05:30:53.696 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7859999999999996e-06 2024-09-14T05:30:53.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.696 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.697 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.697 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293628+0000", 2024-09-14T05:30:53.697 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082324283999999998 2024-09-14T05:30:53.697 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.697 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.697 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.697 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293694+0000", 2024-09-14T05:30:53.697 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6277000000000003e-05 2024-09-14T05:30:53.697 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.697 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.697 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.697 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293948+0000", 2024-09-14T05:30:53.697 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025445699999999998 2024-09-14T05:30:53.697 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.698 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.698 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.698 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.698 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27790 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.698 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:30:53.698 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.446361694999993, 2024-09-14T05:30:53.698 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082461006000000003, 2024-09-14T05:30:53.698 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.698 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.698 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.698 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.698 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.698 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.699 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27790 2024-09-14T05:30:53.699 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.699 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.699 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.699 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.699 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:30:53.699 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.699 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.699 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.699 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.699 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:30:53.699 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.699 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.699 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.699 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.700 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211312+0000", 2024-09-14T05:30:53.700 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9299999999999998e-07 2024-09-14T05:30:53.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.700 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.700 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211314+0000", 2024-09-14T05:30:53.700 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000002e-06 2024-09-14T05:30:53.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.700 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.700 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211320+0000", 2024-09-14T05:30:53.700 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1589999999999998e-06 2024-09-14T05:30:53.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.701 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.701 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211325+0000", 2024-09-14T05:30:53.701 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9609999999999996e-06 2024-09-14T05:30:53.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.701 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.701 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.701 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293474+0000", 2024-09-14T05:30:53.701 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082149080999999999 2024-09-14T05:30:53.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.701 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.701 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.701 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293542+0000", 2024-09-14T05:30:53.701 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8497000000000005e-05 2024-09-14T05:30:53.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.702 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.702 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293772+0000", 2024-09-14T05:30:53.702 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022969900000000001 2024-09-14T05:30:53.702 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.702 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.702 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.702 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27791 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.702 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:30:53.702 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.446341380999996, 2024-09-14T05:30:53.702 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082700838999999998, 2024-09-14T05:30:53.702 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.702 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.703 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.703 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.703 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.703 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.703 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27791 2024-09-14T05:30:53.703 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.703 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.703 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.703 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.703 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:30:53.703 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.703 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.703 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.704 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.704 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:30:53.704 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.704 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.704 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211332+0000", 2024-09-14T05:30:53.704 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7399999999999999e-07 2024-09-14T05:30:53.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.704 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.704 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211334+0000", 2024-09-14T05:30:53.704 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8500000000000001e-06 2024-09-14T05:30:53.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.705 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.705 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211340+0000", 2024-09-14T05:30:53.705 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3010000000000003e-06 2024-09-14T05:30:53.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.705 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.705 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211345+0000", 2024-09-14T05:30:53.705 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.938e-06 2024-09-14T05:30:53.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.705 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.706 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293797+0000", 2024-09-14T05:30:53.706 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082452152000000001 2024-09-14T05:30:53.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.706 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.706 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293869+0000", 2024-09-14T05:30:53.706 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2236000000000004e-05 2024-09-14T05:30:53.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.706 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.706 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294032+0000", 2024-09-14T05:30:53.706 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016298800000000001 2024-09-14T05:30:53.706 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.706 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.707 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.707 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27792 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.707 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:53.707 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.446321850000004, 2024-09-14T05:30:53.707 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082897304000000005, 2024-09-14T05:30:53.707 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.707 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.707 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.707 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.707 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.707 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.707 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27792 2024-09-14T05:30:53.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.708 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.708 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.708 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:53.708 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.708 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.708 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:53.708 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:30:53.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2000000000000001e-07 2024-09-14T05:30:53.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211353+0000", 2024-09-14T05:30:53.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.761e-06 2024-09-14T05:30:53.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.709 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.709 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211359+0000", 2024-09-14T05:30:53.709 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2430000000000002e-06 2024-09-14T05:30:53.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.710 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.710 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211364+0000", 2024-09-14T05:30:53.710 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0239999999999997e-06 2024-09-14T05:30:53.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.710 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.710 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294059+0000", 2024-09-14T05:30:53.710 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082694834999999994 2024-09-14T05:30:53.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.710 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.710 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294114+0000", 2024-09-14T05:30:53.710 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4823e-05 2024-09-14T05:30:53.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.711 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.711 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294248+0000", 2024-09-14T05:30:53.711 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013429800000000001 2024-09-14T05:30:53.711 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.711 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.711 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.711 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27793 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.711 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:53.711 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.446302051000004, 2024-09-14T05:30:53.711 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083153362999999994, 2024-09-14T05:30:53.712 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.712 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.712 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.712 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.712 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.712 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.712 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27793 2024-09-14T05:30:53.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.712 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.712 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.712 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:53.712 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.713 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.713 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:53.713 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.713 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.713 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:30:53.713 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5100000000000001e-07 2024-09-14T05:30:53.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.713 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.713 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211373+0000", 2024-09-14T05:30:53.713 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.001e-06 2024-09-14T05:30:53.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.714 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.714 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211379+0000", 2024-09-14T05:30:53.714 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4559999999999997e-06 2024-09-14T05:30:53.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.714 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.714 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211384+0000", 2024-09-14T05:30:53.714 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9749999999999998e-06 2024-09-14T05:30:53.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.714 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.714 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294279+0000", 2024-09-14T05:30:53.715 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082894282999999999 2024-09-14T05:30:53.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.715 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.715 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.715 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294327+0000", 2024-09-14T05:30:53.715 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8208000000000003e-05 2024-09-14T05:30:53.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.715 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.715 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.715 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294524+0000", 2024-09-14T05:30:53.715 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019708900000000001 2024-09-14T05:30:53.715 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.715 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.715 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.716 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.716 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27794 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.716 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:53.716 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.446281726999999, 2024-09-14T05:30:53.716 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588590000000004, 2024-09-14T05:30:53.716 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.716 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.716 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.716 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.716 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.716 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.716 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27794 2024-09-14T05:30:53.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.716 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.717 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.717 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:53.717 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.717 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.717 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:53.717 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.717 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.717 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:30:53.718 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4499999999999997e-07 2024-09-14T05:30:53.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.718 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.718 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211393+0000", 2024-09-14T05:30:53.718 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.815e-06 2024-09-14T05:30:53.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.718 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.718 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211400+0000", 2024-09-14T05:30:53.718 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2899999999999999e-06 2024-09-14T05:30:53.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.718 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.719 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211404+0000", 2024-09-14T05:30:53.719 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:30:53.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.719 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.719 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294560+0000", 2024-09-14T05:30:53.719 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083155534000000003 2024-09-14T05:30:53.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.719 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.719 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294631+0000", 2024-09-14T05:30:53.719 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1378999999999995e-05 2024-09-14T05:30:53.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.720 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.720 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294980+0000", 2024-09-14T05:30:53.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00034844900000000002 2024-09-14T05:30:53.720 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.720 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.720 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.720 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27795 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.720 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:30:53.720 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.446262368999996, 2024-09-14T05:30:53.720 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083970367000000004, 2024-09-14T05:30:53.720 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.720 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.721 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.721 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.721 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.721 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.721 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27795 2024-09-14T05:30:53.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.721 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.721 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.721 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:30:53.721 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.721 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.722 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:30:53.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.722 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.722 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211411+0000", 2024-09-14T05:30:53.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5499999999999999e-07 2024-09-14T05:30:53.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.722 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.722 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211413+0000", 2024-09-14T05:30:53.722 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:30:53.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.723 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.723 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211419+0000", 2024-09-14T05:30:53.723 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1720000000000004e-06 2024-09-14T05:30:53.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.723 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.723 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211424+0000", 2024-09-14T05:30:53.723 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0690000000000001e-06 2024-09-14T05:30:53.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.723 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.723 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295013+0000", 2024-09-14T05:30:53.723 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588898999999994 2024-09-14T05:30:53.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.724 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.724 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295102+0000", 2024-09-14T05:30:53.724 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9462000000000005e-05 2024-09-14T05:30:53.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.724 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.724 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295381+0000", 2024-09-14T05:30:53.724 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000278566 2024-09-14T05:30:53.724 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.724 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.724 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.724 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27796 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.725 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:53.725 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.446242639999994, 2024-09-14T05:30:53.725 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084379107999999994, 2024-09-14T05:30:53.725 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.725 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.725 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.725 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.725 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.725 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.725 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27796 2024-09-14T05:30:53.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.725 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.725 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.725 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.726 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:53.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.726 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.726 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:53.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.726 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.726 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:30:53.726 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2800000000000002e-07 2024-09-14T05:30:53.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211432+0000", 2024-09-14T05:30:53.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:30:53.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211439+0000", 2024-09-14T05:30:53.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5239999999999997e-06 2024-09-14T05:30:53.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.727 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.727 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211444+0000", 2024-09-14T05:30:53.727 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.95e-06 2024-09-14T05:30:53.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295602+0000", 2024-09-14T05:30:53.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084158017000000002 2024-09-14T05:30:53.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295674+0000", 2024-09-14T05:30:53.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2385999999999994e-05 2024-09-14T05:30:53.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.728 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.728 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295809+0000", 2024-09-14T05:30:53.728 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013495900000000001 2024-09-14T05:30:53.728 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.729 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.729 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.729 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27797 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 266240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.729 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:53.729 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.446222657999996, 2024-09-14T05:30:53.729 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084898597000000006, 2024-09-14T05:30:53.729 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.729 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.729 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.729 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.729 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.729 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.730 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27797 2024-09-14T05:30:53.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.730 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:53.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.730 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.730 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:53.730 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:30:53.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:30:53.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211452+0000", 2024-09-14T05:30:53.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.708e-06 2024-09-14T05:30:53.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.731 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.731 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211458+0000", 2024-09-14T05:30:53.731 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3489999999999997e-06 2024-09-14T05:30:53.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.732 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.732 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211464+0000", 2024-09-14T05:30:53.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3820000000000003e-06 2024-09-14T05:30:53.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.732 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.732 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295839+0000", 2024-09-14T05:30:53.732 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084375055000000004 2024-09-14T05:30:53.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.732 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295918+0000", 2024-09-14T05:30:53.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9253999999999997e-05 2024-09-14T05:30:53.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.733 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.733 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296349+0000", 2024-09-14T05:30:53.733 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043051 2024-09-14T05:30:53.733 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.733 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.733 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.733 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27798 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.733 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:53.734 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.446202943000003, 2024-09-14T05:30:53.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085300275999999994, 2024-09-14T05:30:53.734 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.734 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.734 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.734 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.734 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.734 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.734 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27798 2024-09-14T05:30:53.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.734 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.734 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.734 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:53.734 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:53.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:30:53.735 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:53.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.735 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.735 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211472+0000", 2024-09-14T05:30:53.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7579999999999999e-06 2024-09-14T05:30:53.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211478+0000", 2024-09-14T05:30:53.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0310000000000004e-06 2024-09-14T05:30:53.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.736 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211483+0000", 2024-09-14T05:30:53.736 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8110000000000002e-06 2024-09-14T05:30:53.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.736 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296469+0000", 2024-09-14T05:30:53.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084986575999999994 2024-09-14T05:30:53.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296545+0000", 2024-09-14T05:30:53.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5749999999999998e-05 2024-09-14T05:30:53.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.737 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.737 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296770+0000", 2024-09-14T05:30:53.737 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022498900000000001 2024-09-14T05:30:53.737 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.737 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.738 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.738 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27799 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 307200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.738 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:53.738 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.446184029999998, 2024-09-14T05:30:53.738 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085718461999999995, 2024-09-14T05:30:53.738 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.738 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.738 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.738 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.738 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.738 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.738 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27799 2024-09-14T05:30:53.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.739 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:53.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:53.739 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.739 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.739 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:30:53.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8000000000000001e-07 2024-09-14T05:30:53.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.740 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.740 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211491+0000", 2024-09-14T05:30:53.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-14T05:30:53.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.740 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.740 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211497+0000", 2024-09-14T05:30:53.740 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.857e-06 2024-09-14T05:30:53.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.740 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.741 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211502+0000", 2024-09-14T05:30:53.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9790000000000001e-06 2024-09-14T05:30:53.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.741 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.741 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296800+0000", 2024-09-14T05:30:53.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085298099000000002 2024-09-14T05:30:53.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.741 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.741 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296871+0000", 2024-09-14T05:30:53.741 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0879999999999999e-05 2024-09-14T05:30:53.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.742 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.742 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297207+0000", 2024-09-14T05:30:53.742 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033651699999999999 2024-09-14T05:30:53.742 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.742 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.742 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.742 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27826 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.742 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:30:53.742 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.444468908999994, 2024-09-14T05:30:53.742 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084306729999999996, 2024-09-14T05:30:53.742 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.742 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.742 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.743 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.743 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.743 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.743 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27826 2024-09-14T05:30:53.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.743 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:30:53.743 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.743 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.743 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:30:53.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.744 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213205+0000", 2024-09-14T05:30:53.744 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.277e-06 2024-09-14T05:30:53.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.744 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213210+0000", 2024-09-14T05:30:53.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3120000000000001e-06 2024-09-14T05:30:53.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.745 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213226+0000", 2024-09-14T05:30:53.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5279000000000001e-05 2024-09-14T05:30:53.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.745 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.745 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213234+0000", 2024-09-14T05:30:53.745 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7020000000000007e-06 2024-09-14T05:30:53.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.746 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.746 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297239+0000", 2024-09-14T05:30:53.746 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084004876000000006 2024-09-14T05:30:53.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.746 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.746 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297311+0000", 2024-09-14T05:30:53.746 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1336999999999998e-05 2024-09-14T05:30:53.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.746 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.746 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297511+0000", 2024-09-14T05:30:53.746 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019994699999999999 2024-09-14T05:30:53.746 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.747 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.747 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.747 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27827 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:30:53.747 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:30:53.747 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 67.444423721999996, 2024-09-14T05:30:53.747 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084520537000000007, 2024-09-14T05:30:53.747 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:53.747 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:53.747 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:53.747 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:53.747 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:53.747 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:53.747 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27827 2024-09-14T05:30:53.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.748 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:53.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.748 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:53.748 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:30:53.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.748 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:53.748 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:30:53.748 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:53.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.748 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:53.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213250+0000", 2024-09-14T05:30:53.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1099999999999995e-07 2024-09-14T05:30:53.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:53.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213252+0000", 2024-09-14T05:30:53.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6699999999999998e-06 2024-09-14T05:30:53.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:53.749 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213259+0000", 2024-09-14T05:30:53.749 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8469999999999998e-06 2024-09-14T05:30:53.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.749 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:53.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213265+0000", 2024-09-14T05:30:53.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4940000000000003e-06 2024-09-14T05:30:53.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:53.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297541+0000", 2024-09-14T05:30:53.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084276731999999993 2024-09-14T05:30:53.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.750 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:53.750 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297579+0000", 2024-09-14T05:30:53.750 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8108999999999997e-05 2024-09-14T05:30:53.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:53.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:53.751 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:53.751 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297770+0000", 2024-09-14T05:30:53.751 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019007400000000001 2024-09-14T05:30:53.751 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.751 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.751 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.751 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:53.751 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:53.751 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:53.753 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:30:53.753 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:53.751+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:53.853 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:53.853+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:53.861 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:53.862 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:53.862 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:53.862 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:53.870 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:30:53.955 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:53.954+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:53.974 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:53.974 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:53.974 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:53.974 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:53.975 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:53.983 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:30:54.001 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-14T05:30:54.067 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:54.067+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:54.138 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:54.138 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3523 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6062080~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.622659092999996, 2024-09-14T05:30:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096911030999999995, 2024-09-14T05:30:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3523 2024-09-14T05:30:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514359+0000", 2024-09-14T05:30:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9299999999999999e-07 2024-09-14T05:30:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514361+0000", 2024-09-14T05:30:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514366+0000", 2024-09-14T05:30:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1660000000000002e-06 2024-09-14T05:30:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514371+0000", 2024-09-14T05:30:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.648e-06 2024-09-14T05:30:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.610954+0000", 2024-09-14T05:30:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096582170999999994 2024-09-14T05:30:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611002+0000", 2024-09-14T05:30:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7923999999999997e-05 2024-09-14T05:30:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611269+0000", 2024-09-14T05:30:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026789499999999998 2024-09-14T05:30:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3524 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6553600~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.622639900999999, 2024-09-14T05:30:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097263347999999999, 2024-09-14T05:30:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3524 2024-09-14T05:30:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514380+0000", 2024-09-14T05:30:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6500000000000001e-06 2024-09-14T05:30:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514386+0000", 2024-09-14T05:30:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7360000000000004e-06 2024-09-14T05:30:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514391+0000", 2024-09-14T05:30:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2830000000000003e-06 2024-09-14T05:30:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611295+0000", 2024-09-14T05:30:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096904814000000006 2024-09-14T05:30:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611343+0000", 2024-09-14T05:30:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7506999999999998e-05 2024-09-14T05:30:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611641+0000", 2024-09-14T05:30:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029798800000000001 2024-09-14T05:30:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3525 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7045120~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.622618013999997, 2024-09-14T05:30:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097644422999999994, 2024-09-14T05:30:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3525 2024-09-14T05:30:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514400+0000", 2024-09-14T05:30:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.77e-07 2024-09-14T05:30:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514402+0000", 2024-09-14T05:30:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6530000000000001e-06 2024-09-14T05:30:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514411+0000", 2024-09-14T05:30:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7429999999999997e-06 2024-09-14T05:30:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514417+0000", 2024-09-14T05:30:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2249999999999997e-06 2024-09-14T05:30:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611668+0000", 2024-09-14T05:30:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097250681000000005 2024-09-14T05:30:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611716+0000", 2024-09-14T05:30:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7336000000000002e-05 2024-09-14T05:30:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612044+0000", 2024-09-14T05:30:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032840799999999998 2024-09-14T05:30:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3526 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7536640~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.622593530000003, 2024-09-14T05:30:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097943884999999994, 2024-09-14T05:30:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3526 2024-09-14T05:30:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0400000000000002e-07 2024-09-14T05:30:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514426+0000", 2024-09-14T05:30:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514432+0000", 2024-09-14T05:30:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.105e-06 2024-09-14T05:30:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514438+0000", 2024-09-14T05:30:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5149999999999997e-06 2024-09-14T05:30:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612072+0000", 2024-09-14T05:30:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097633947999999998 2024-09-14T05:30:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612120+0000", 2024-09-14T05:30:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8458999999999998e-05 2024-09-14T05:30:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612368+0000", 2024-09-14T05:30:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024761999999999999 2024-09-14T05:30:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3527 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 8028160~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.622572817000005, 2024-09-14T05:30:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098244955999999994, 2024-09-14T05:30:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3527 2024-09-14T05:30:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519842+0000", 2024-09-14T05:30:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053975230000000004 2024-09-14T05:30:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519851+0000", 2024-09-14T05:30:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5599999999999994e-06 2024-09-14T05:30:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519858+0000", 2024-09-14T05:30:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2409999999999998e-06 2024-09-14T05:30:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519863+0000", 2024-09-14T05:30:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:30:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612395+0000", 2024-09-14T05:30:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092532552000000004 2024-09-14T05:30:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612442+0000", 2024-09-14T05:30:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6486000000000002e-05 2024-09-14T05:30:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612690+0000", 2024-09-14T05:30:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024791599999999999 2024-09-14T05:30:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3603 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5177344~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.617020371999999, 2024-09-14T05:30:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096796436, 2024-09-14T05:30:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3603 2024-09-14T05:30:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1899999999999999e-07 2024-09-14T05:30:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519998+0000", 2024-09-14T05:30:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5000000000000001e-07 2024-09-14T05:30:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520002+0000", 2024-09-14T05:30:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7299999999999999e-06 2024-09-14T05:30:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520004+0000", 2024-09-14T05:30:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.429e-06 2024-09-14T05:30:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616493+0000", 2024-09-14T05:30:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096488345000000003 2024-09-14T05:30:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616544+0000", 2024-09-14T05:30:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1724000000000001e-05 2024-09-14T05:30:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616794+0000", 2024-09-14T05:30:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000249139 2024-09-14T05:30:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3604 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5668864~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.617005734000003, 2024-09-14T05:30:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108222999999994, 2024-09-14T05:30:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3604 2024-09-14T05:30:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7000000000000001e-07 2024-09-14T05:30:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520013+0000", 2024-09-14T05:30:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.389e-06 2024-09-14T05:30:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520018+0000", 2024-09-14T05:30:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3019999999999996e-06 2024-09-14T05:30:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520026+0000", 2024-09-14T05:30:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6170000000000007e-06 2024-09-14T05:30:54.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616816+0000", 2024-09-14T05:30:54.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096789766999999999 2024-09-14T05:30:54.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616871+0000", 2024-09-14T05:30:54.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4811000000000001e-05 2024-09-14T05:30:54.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617120+0000", 2024-09-14T05:30:54.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024906700000000001 2024-09-14T05:30:54.171 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.171 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.171 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.171 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3605 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6160384~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.171 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:54.171 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.616987273999996, 2024-09-14T05:30:54.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097423945999999997, 2024-09-14T05:30:54.171 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.171 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.171 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.171 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.172 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.172 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.172 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3605 2024-09-14T05:30:54.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.172 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:54.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:54.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:54.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3999999999999998e-07 2024-09-14T05:30:54.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520032+0000", 2024-09-14T05:30:54.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.083e-06 2024-09-14T05:30:54.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520036+0000", 2024-09-14T05:30:54.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.019e-06 2024-09-14T05:30:54.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520038+0000", 2024-09-14T05:30:54.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4849999999999999e-06 2024-09-14T05:30:54.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617147+0000", 2024-09-14T05:30:54.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108665999999996 2024-09-14T05:30:54.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617199+0000", 2024-09-14T05:30:54.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2558000000000001e-05 2024-09-14T05:30:54.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617454+0000", 2024-09-14T05:30:54.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025489499999999999 2024-09-14T05:30:54.175 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.175 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.175 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.175 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3606 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6651904~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.176 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:54.176 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.616976551999997, 2024-09-14T05:30:54.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097745705000000002, 2024-09-14T05:30:54.176 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.176 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.176 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.176 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.176 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.176 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.176 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3606 2024-09-14T05:30:54.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.176 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:54.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:54.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:54.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-14T05:30:54.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520042+0000", 2024-09-14T05:30:54.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4799999999999997e-07 2024-09-14T05:30:54.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520046+0000", 2024-09-14T05:30:54.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8149999999999999e-06 2024-09-14T05:30:54.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520049+0000", 2024-09-14T05:30:54.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.576e-06 2024-09-14T05:30:54.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617476+0000", 2024-09-14T05:30:54.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097427558999999997 2024-09-14T05:30:54.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617522+0000", 2024-09-14T05:30:54.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5927999999999998e-05 2024-09-14T05:30:54.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617787+0000", 2024-09-14T05:30:54.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000264663 2024-09-14T05:30:54.180 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.180 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.180 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.180 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3607 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7143424~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.180 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:54.180 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.616966024999996, 2024-09-14T05:30:54.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097871125000000003, 2024-09-14T05:30:54.180 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.180 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.180 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.181 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.181 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.181 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.181 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3607 2024-09-14T05:30:54.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.181 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:54.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:54.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520052+0000", 2024-09-14T05:30:54.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.03e-07 2024-09-14T05:30:54.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520053+0000", 2024-09-14T05:30:54.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.85e-07 2024-09-14T05:30:54.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520056+0000", 2024-09-14T05:30:54.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9140000000000001e-06 2024-09-14T05:30:54.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520059+0000", 2024-09-14T05:30:54.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4600000000000002e-06 2024-09-14T05:30:54.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617809+0000", 2024-09-14T05:30:54.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097750216000000001 2024-09-14T05:30:54.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617864+0000", 2024-09-14T05:30:54.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5211999999999999e-05 2024-09-14T05:30:54.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617923+0000", 2024-09-14T05:30:54.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8235000000000001e-05 2024-09-14T05:30:54.184 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.184 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.184 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.185 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3608 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7634944~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.185 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:54.185 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.616955716999996, 2024-09-14T05:30:54.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098199212999999994, 2024-09-14T05:30:54.185 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.185 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.185 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.185 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.185 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.185 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.185 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3608 2024-09-14T05:30:54.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.185 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:54.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:54.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525953+0000", 2024-09-14T05:30:54.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058911850000000002 2024-09-14T05:30:54.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525957+0000", 2024-09-14T05:30:54.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2379999999999999e-06 2024-09-14T05:30:54.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525967+0000", 2024-09-14T05:30:54.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5389999999999996e-06 2024-09-14T05:30:54.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525972+0000", 2024-09-14T05:30:54.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.541e-06 2024-09-14T05:30:54.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617945+0000", 2024-09-14T05:30:54.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091972597000000003 2024-09-14T05:30:54.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617992+0000", 2024-09-14T05:30:54.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7397e-05 2024-09-14T05:30:54.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.618261+0000", 2024-09-14T05:30:54.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026871600000000001 2024-09-14T05:30:54.189 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.189 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.189 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.189 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3640 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6586368~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.190 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:54.190 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.610847875000005, 2024-09-14T05:30:54.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097068580000000002, 2024-09-14T05:30:54.190 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.190 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.190 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.190 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.190 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.190 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.190 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3640 2024-09-14T05:30:54.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.190 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:54.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:54.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:54.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.23e-07 2024-09-14T05:30:54.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526171+0000", 2024-09-14T05:30:54.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:30:54.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526175+0000", 2024-09-14T05:30:54.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8369999999999999e-06 2024-09-14T05:30:54.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526177+0000", 2024-09-14T05:30:54.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4959999999999999e-06 2024-09-14T05:30:54.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622847+0000", 2024-09-14T05:30:54.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096669790000000005 2024-09-14T05:30:54.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622902+0000", 2024-09-14T05:30:54.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5157000000000001e-05 2024-09-14T05:30:54.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623238+0000", 2024-09-14T05:30:54.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033603399999999997 2024-09-14T05:30:54.194 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.194 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.194 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.194 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3641 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7077888~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.194 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:54.194 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.610837364999995, 2024-09-14T05:30:54.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097371141999999994, 2024-09-14T05:30:54.194 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.195 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.195 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.195 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.195 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.195 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.195 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3641 2024-09-14T05:30:54.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.195 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:54.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:54.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531851+0000", 2024-09-14T05:30:54.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005671265 2024-09-14T05:30:54.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531855+0000", 2024-09-14T05:30:54.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0849999999999999e-06 2024-09-14T05:30:54.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531867+0000", 2024-09-14T05:30:54.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1028e-05 2024-09-14T05:30:54.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531872+0000", 2024-09-14T05:30:54.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000003e-06 2024-09-14T05:30:54.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623251+0000", 2024-09-14T05:30:54.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091378980999999998 2024-09-14T05:30:54.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623295+0000", 2024-09-14T05:30:54.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3881999999999998e-05 2024-09-14T05:30:54.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623551+0000", 2024-09-14T05:30:54.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025636099999999998 2024-09-14T05:30:54.198 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.198 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.198 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.199 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3721 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.199 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:54.199 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.60497909, 2024-09-14T05:30:54.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096806472000000005, 2024-09-14T05:30:54.199 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.199 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.199 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.199 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.199 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.199 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.199 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3721 2024-09-14T05:30:54.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.199 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:54.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:54.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532039+0000", 2024-09-14T05:30:54.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.05e-07 2024-09-14T05:30:54.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532040+0000", 2024-09-14T05:30:54.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9099999999999991e-07 2024-09-14T05:30:54.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532043+0000", 2024-09-14T05:30:54.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6610000000000002e-06 2024-09-14T05:30:54.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532046+0000", 2024-09-14T05:30:54.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4080000000000001e-06 2024-09-14T05:30:54.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628545+0000", 2024-09-14T05:30:54.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096499615999999996 2024-09-14T05:30:54.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628592+0000", 2024-09-14T05:30:54.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6907000000000003e-05 2024-09-14T05:30:54.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628845+0000", 2024-09-14T05:30:54.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025268399999999998 2024-09-14T05:30:54.203 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.203 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.203 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.203 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3722 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6193152~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.203 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:54.203 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.604969010000005, 2024-09-14T05:30:54.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097146587000000006, 2024-09-14T05:30:54.203 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.203 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.203 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.204 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3722 2024-09-14T05:30:54.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.204 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:54.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:54.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538191+0000", 2024-09-14T05:30:54.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061429969999999999 2024-09-14T05:30:54.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538200+0000", 2024-09-14T05:30:54.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7069999999999998e-06 2024-09-14T05:30:54.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538212+0000", 2024-09-14T05:30:54.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1504e-05 2024-09-14T05:30:54.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538219+0000", 2024-09-14T05:30:54.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7500000000000003e-06 2024-09-14T05:30:54.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628869+0000", 2024-09-14T05:30:54.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090649224 2024-09-14T05:30:54.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628919+0000", 2024-09-14T05:30:54.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0144e-05 2024-09-14T05:30:54.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.629195+0000", 2024-09-14T05:30:54.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027626099999999998 2024-09-14T05:30:54.207 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.207 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.207 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.207 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3758 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7110656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.207 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:54.207 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.598374793999994, 2024-09-14T05:30:54.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096977544999999998, 2024-09-14T05:30:54.207 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.208 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.208 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.208 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3758 2024-09-14T05:30:54.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.208 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:54.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:54.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:54.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:54.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538645+0000", 2024-09-14T05:30:54.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.843e-06 2024-09-14T05:30:54.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538651+0000", 2024-09-14T05:30:54.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6169999999999998e-06 2024-09-14T05:30:54.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538655+0000", 2024-09-14T05:30:54.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5399999999999997e-06 2024-09-14T05:30:54.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635225+0000", 2024-09-14T05:30:54.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096570149999999993 2024-09-14T05:30:54.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635272+0000", 2024-09-14T05:30:54.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7199e-05 2024-09-14T05:30:54.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635620+0000", 2024-09-14T05:30:54.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034782600000000003 2024-09-14T05:30:54.211 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.212 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.212 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.212 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3759 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.212 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:54.212 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.598356882999994, 2024-09-14T05:30:54.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097335331999999997, 2024-09-14T05:30:54.212 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.212 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.212 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.212 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.212 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.212 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.213 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3759 2024-09-14T05:30:54.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.213 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:54.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:54.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544013+0000", 2024-09-14T05:30:54.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053520829999999997 2024-09-14T05:30:54.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544016+0000", 2024-09-14T05:30:54.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6660000000000001e-06 2024-09-14T05:30:54.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544026+0000", 2024-09-14T05:30:54.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7089999999999997e-06 2024-09-14T05:30:54.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544043+0000", 2024-09-14T05:30:54.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833999999999998e-05 2024-09-14T05:30:54.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635646+0000", 2024-09-14T05:30:54.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091603544999999995 2024-09-14T05:30:54.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635696+0000", 2024-09-14T05:30:54.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9307000000000001e-05 2024-09-14T05:30:54.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635996+0000", 2024-09-14T05:30:54.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030018800000000001 2024-09-14T05:30:54.216 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.216 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.216 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.216 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3838 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6225920~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.216 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:54.216 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.592695203000005, 2024-09-14T05:30:54.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096894235999999995, 2024-09-14T05:30:54.216 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.216 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.216 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.217 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.217 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.217 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.217 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3838 2024-09-14T05:30:54.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.217 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:54.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:54.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544323+0000", 2024-09-14T05:30:54.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-14T05:30:54.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544325+0000", 2024-09-14T05:30:54.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.75e-06 2024-09-14T05:30:54.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544331+0000", 2024-09-14T05:30:54.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5019999999999998e-06 2024-09-14T05:30:54.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544335+0000", 2024-09-14T05:30:54.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2989999999999998e-06 2024-09-14T05:30:54.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640869+0000", 2024-09-14T05:30:54.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096533351000000003 2024-09-14T05:30:54.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640919+0000", 2024-09-14T05:30:54.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999999e-05 2024-09-14T05:30:54.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641217+0000", 2024-09-14T05:30:54.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029717600000000002 2024-09-14T05:30:54.220 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.220 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.220 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.220 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3839 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.220 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:54.220 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.592676264999994, 2024-09-14T05:30:54.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097240339999999995, 2024-09-14T05:30:54.221 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.221 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.221 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.221 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.221 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.221 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.221 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3839 2024-09-14T05:30:54.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.221 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:54.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:54.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549834+0000", 2024-09-14T05:30:54.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054927819999999999 2024-09-14T05:30:54.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549838+0000", 2024-09-14T05:30:54.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8689999999999997e-06 2024-09-14T05:30:54.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549848+0000", 2024-09-14T05:30:54.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0154e-05 2024-09-14T05:30:54.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549853+0000", 2024-09-14T05:30:54.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9740000000000001e-06 2024-09-14T05:30:54.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641245+0000", 2024-09-14T05:30:54.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091392245999999996 2024-09-14T05:30:54.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641295+0000", 2024-09-14T05:30:54.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0018999999999997e-05 2024-09-14T05:30:54.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641582+0000", 2024-09-14T05:30:54.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028629600000000001 2024-09-14T05:30:54.224 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.224 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.224 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.225 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3868 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.225 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:54.225 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.587013686999995, 2024-09-14T05:30:54.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096859555999999999, 2024-09-14T05:30:54.225 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.225 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.225 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.225 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.225 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.225 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.225 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3868 2024-09-14T05:30:54.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.225 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:54.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:54.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556173+0000", 2024-09-14T05:30:54.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006169038 2024-09-14T05:30:54.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556177+0000", 2024-09-14T05:30:54.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0010000000000003e-06 2024-09-14T05:30:54.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556188+0000", 2024-09-14T05:30:54.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0675e-05 2024-09-14T05:30:54.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556199+0000", 2024-09-14T05:30:54.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.155e-05 2024-09-14T05:30:54.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646511+0000", 2024-09-14T05:30:54.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090312187000000002 2024-09-14T05:30:54.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646563+0000", 2024-09-14T05:30:54.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2219000000000003e-05 2024-09-14T05:30:54.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646863+0000", 2024-09-14T05:30:54.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029988600000000002 2024-09-14T05:30:54.229 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.229 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.229 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.229 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.229 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.229 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:54.230 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:30:54.231 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:54.230+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:54.331 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:54.331+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:54.374 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:54.374 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:54.374 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:54.374 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:54.383 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:30:54.432 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:54.432+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:54.486 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:54.486 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:54.486 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:54.486 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:54.486 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:54.495 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:30:54.533 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:54.532+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:54.634 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:54.634+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:54.639 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:54.639 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:54.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:54.639 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:54.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.639 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1989 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 16326080~54496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:54.639 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:54.640 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.076456613999994, 2024-09-14T05:30:54.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096664322999999996, 2024-09-14T05:30:54.640 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.640 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.640 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:54.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.640 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1989 2024-09-14T05:30:54.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.640 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:54.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:54.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561455+0000", 2024-09-14T05:30:54.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1699999999999997e-07 2024-09-14T05:30:54.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590703+0000", 2024-09-14T05:30:54.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029248225999999999 2024-09-14T05:30:54.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590732+0000", 2024-09-14T05:30:54.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8679000000000001e-05 2024-09-14T05:30:54.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590759+0000", 2024-09-14T05:30:54.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7319999999999999e-05 2024-09-14T05:30:54.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590835+0000", 2024-09-14T05:30:54.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5543999999999994e-05 2024-09-14T05:30:54.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590956+0000", 2024-09-14T05:30:54.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012168 2024-09-14T05:30:54.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 1", 2024-09-14T05:30:54.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.591261+0000", 2024-09-14T05:30:54.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030484799999999998 2024-09-14T05:30:54.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:54.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.637682+0000", 2024-09-14T05:30:54.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046420550999999997 2024-09-14T05:30:54.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:54.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.657912+0000", 2024-09-14T05:30:54.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020230011999999999 2024-09-14T05:30:54.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:54.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658007+0000", 2024-09-14T05:30:54.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5814000000000001e-05 2024-09-14T05:30:54.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658118+0000", 2024-09-14T05:30:54.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110832 2024-09-14T05:30:54.645 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.645 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.645 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.646 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2033 3.1e 3:7e47f6eb:::rbd_data.112d1f36d097.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30853661~63531] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:54.646 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:54.646 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.965181947999994, 2024-09-14T05:30:54.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065167630000000004, 2024-09-14T05:30:54.646 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.646 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.646 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:54.646 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.646 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.646 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.646 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2033 2024-09-14T05:30:54.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.646 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:54.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:54.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672730+0000", 2024-09-14T05:30:54.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.393e-06 2024-09-14T05:30:54.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691289+0000", 2024-09-14T05:30:54.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018558991 2024-09-14T05:30:54.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691316+0000", 2024-09-14T05:30:54.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7498999999999999e-05 2024-09-14T05:30:54.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691331+0000", 2024-09-14T05:30:54.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4652e-05 2024-09-14T05:30:54.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691397+0000", 2024-09-14T05:30:54.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6112000000000007e-05 2024-09-14T05:30:54.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691509+0000", 2024-09-14T05:30:54.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111599 2024-09-14T05:30:54.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:54.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.692253+0000", 2024-09-14T05:30:54.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074456200000000004 2024-09-14T05:30:54.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:54.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.719140+0000", 2024-09-14T05:30:54.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026886542999999999 2024-09-14T05:30:54.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:54.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737789+0000", 2024-09-14T05:30:54.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018649203999999999 2024-09-14T05:30:54.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:54.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737846+0000", 2024-09-14T05:30:54.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6501e-05 2024-09-14T05:30:54.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737896+0000", 2024-09-14T05:30:54.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0574000000000001e-05 2024-09-14T05:30:54.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.651 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.652 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3008 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13729792~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.652 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:54.652 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.334258078000005, 2024-09-14T05:30:54.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061007651000000003, 2024-09-14T05:30:54.652 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.652 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.652 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.652 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3008 2024-09-14T05:30:54.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.652 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:54.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:54.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303653+0000", 2024-09-14T05:30:54.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0699999999999999e-07 2024-09-14T05:30:54.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303654+0000", 2024-09-14T05:30:54.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-07 2024-09-14T05:30:54.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303657+0000", 2024-09-14T05:30:54.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2890000000000002e-06 2024-09-14T05:30:54.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303661+0000", 2024-09-14T05:30:54.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3290000000000004e-06 2024-09-14T05:30:54.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364233+0000", 2024-09-14T05:30:54.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060572134 2024-09-14T05:30:54.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364315+0000", 2024-09-14T05:30:54.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1549000000000001e-05 2024-09-14T05:30:54.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364660+0000", 2024-09-14T05:30:54.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034525300000000001 2024-09-14T05:30:54.656 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.656 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.656 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.656 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3009 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14221312~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.656 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:54.656 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.334243987999997, 2024-09-14T05:30:54.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061445063000000001, 2024-09-14T05:30:54.657 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.657 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.657 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.657 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.657 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.657 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.657 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3009 2024-09-14T05:30:54.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.657 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:54.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:54.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:54.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:54.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303669+0000", 2024-09-14T05:30:54.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7150000000000001e-06 2024-09-14T05:30:54.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303675+0000", 2024-09-14T05:30:54.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5339999999999996e-06 2024-09-14T05:30:54.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303681+0000", 2024-09-14T05:30:54.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3639999999999998e-06 2024-09-14T05:30:54.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364705+0000", 2024-09-14T05:30:54.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061024123 2024-09-14T05:30:54.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364786+0000", 2024-09-14T05:30:54.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1378999999999995e-05 2024-09-14T05:30:54.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365112+0000", 2024-09-14T05:30:54.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032558700000000001 2024-09-14T05:30:54.660 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.661 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.661 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.661 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3010 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14712832~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.661 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:54.661 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.334223124000005, 2024-09-14T05:30:54.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061834630000000002, 2024-09-14T05:30:54.661 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.661 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.661 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.661 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.661 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.662 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.662 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3010 2024-09-14T05:30:54.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.662 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:54.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:54.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303688+0000", 2024-09-14T05:30:54.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:30:54.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303690+0000", 2024-09-14T05:30:54.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9460000000000001e-06 2024-09-14T05:30:54.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303696+0000", 2024-09-14T05:30:54.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2230000000000004e-06 2024-09-14T05:30:54.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303700+0000", 2024-09-14T05:30:54.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7290000000000002e-06 2024-09-14T05:30:54.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365163+0000", 2024-09-14T05:30:54.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061463093000000003 2024-09-14T05:30:54.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365214+0000", 2024-09-14T05:30:54.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0977999999999999e-05 2024-09-14T05:30:54.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365522+0000", 2024-09-14T05:30:54.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000308227 2024-09-14T05:30:54.665 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.665 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.665 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.665 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3241 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.665 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:54.666 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.308548701999996, 2024-09-14T05:30:54.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061041892, 2024-09-14T05:30:54.666 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.666 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.666 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.666 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.666 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.666 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.666 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3241 2024-09-14T05:30:54.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.666 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:54.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:54.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329496+0000", 2024-09-14T05:30:54.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013421799999999999 2024-09-14T05:30:54.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329499+0000", 2024-09-14T05:30:54.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8459999999999999e-06 2024-09-14T05:30:54.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329505+0000", 2024-09-14T05:30:54.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4320000000000004e-06 2024-09-14T05:30:54.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329509+0000", 2024-09-14T05:30:54.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0339999999999998e-06 2024-09-14T05:30:54.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.389947+0000", 2024-09-14T05:30:54.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060438093999999998 2024-09-14T05:30:54.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390038+0000", 2024-09-14T05:30:54.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0253999999999993e-05 2024-09-14T05:30:54.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390404+0000", 2024-09-14T05:30:54.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036601400000000001 2024-09-14T05:30:54.670 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.670 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.670 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.670 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3242 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.670 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:54.670 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.308393793999997, 2024-09-14T05:30:54.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061282426000000001, 2024-09-14T05:30:54.670 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.670 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.670 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.671 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.671 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.671 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.671 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3242 2024-09-14T05:30:54.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.671 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:54.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:54.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:54.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9900000000000001e-07 2024-09-14T05:30:54.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329518+0000", 2024-09-14T05:30:54.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.328e-06 2024-09-14T05:30:54.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329522+0000", 2024-09-14T05:30:54.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.563e-06 2024-09-14T05:30:54.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329525+0000", 2024-09-14T05:30:54.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7379999999999999e-06 2024-09-14T05:30:54.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390437+0000", 2024-09-14T05:30:54.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060912463 2024-09-14T05:30:54.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390498+0000", 2024-09-14T05:30:54.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0615999999999997e-05 2024-09-14T05:30:54.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390799+0000", 2024-09-14T05:30:54.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030131899999999998 2024-09-14T05:30:54.674 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.674 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.674 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.674 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3243 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.675 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:54.675 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.308382007999995, 2024-09-14T05:30:54.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061728980000000003, 2024-09-14T05:30:54.675 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.675 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.675 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.675 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.675 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.675 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.675 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3243 2024-09-14T05:30:54.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.675 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:54.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:54.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329692+0000", 2024-09-14T05:30:54.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000163948 2024-09-14T05:30:54.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329696+0000", 2024-09-14T05:30:54.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5540000000000002e-06 2024-09-14T05:30:54.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329706+0000", 2024-09-14T05:30:54.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7960000000000007e-06 2024-09-14T05:30:54.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329712+0000", 2024-09-14T05:30:54.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6109999999999999e-06 2024-09-14T05:30:54.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390861+0000", 2024-09-14T05:30:54.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061148767 2024-09-14T05:30:54.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390942+0000", 2024-09-14T05:30:54.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1266000000000004e-05 2024-09-14T05:30:54.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391258+0000", 2024-09-14T05:30:54.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031503800000000002 2024-09-14T05:30:54.679 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.679 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.679 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.679 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3244 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14778368~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.679 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:54.679 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.30818696, 2024-09-14T05:30:54.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062050594000000001, 2024-09-14T05:30:54.679 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.679 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.679 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.679 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.680 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.680 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.680 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3244 2024-09-14T05:30:54.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.680 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:54.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:54.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329916+0000", 2024-09-14T05:30:54.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019290799999999999 2024-09-14T05:30:54.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329919+0000", 2024-09-14T05:30:54.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9019999999999999e-06 2024-09-14T05:30:54.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329926+0000", 2024-09-14T05:30:54.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9e-06 2024-09-14T05:30:54.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329932+0000", 2024-09-14T05:30:54.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.088e-06 2024-09-14T05:30:54.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391299+0000", 2024-09-14T05:30:54.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061366136000000002 2024-09-14T05:30:54.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391391+0000", 2024-09-14T05:30:54.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2204e-05 2024-09-14T05:30:54.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391774+0000", 2024-09-14T05:30:54.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038345600000000001 2024-09-14T05:30:54.683 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.683 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.683 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.684 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3921 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 5308416~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.684 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:54.684 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.236643150000006, 2024-09-14T05:30:54.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060842694000000003, 2024-09-14T05:30:54.684 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.684 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.684 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.684 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3921 2024-09-14T05:30:54.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.685 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:54.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:54.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401705+0000", 2024-09-14T05:30:54.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768499999999999 2024-09-14T05:30:54.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401709+0000", 2024-09-14T05:30:54.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0319999999999997e-06 2024-09-14T05:30:54.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401720+0000", 2024-09-14T05:30:54.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1005000000000001e-05 2024-09-14T05:30:54.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401727+0000", 2024-09-14T05:30:54.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1879999999999996e-06 2024-09-14T05:30:54.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461748+0000", 2024-09-14T05:30:54.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060021199999999997 2024-09-14T05:30:54.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461793+0000", 2024-09-14T05:30:54.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4530999999999998e-05 2024-09-14T05:30:54.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462110+0000", 2024-09-14T05:30:54.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031705299999999997 2024-09-14T05:30:54.688 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.688 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.688 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.688 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3923 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 6291456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.688 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:54.688 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.235997122000001, 2024-09-14T05:30:54.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060891273000000003, 2024-09-14T05:30:54.688 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.689 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.689 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.689 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.689 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.689 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.689 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3923 2024-09-14T05:30:54.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.689 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:54.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:54.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407054+0000", 2024-09-14T05:30:54.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005140787 2024-09-14T05:30:54.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407059+0000", 2024-09-14T05:30:54.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.04e-06 2024-09-14T05:30:54.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407069+0000", 2024-09-14T05:30:54.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4429999999999992e-06 2024-09-14T05:30:54.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407076+0000", 2024-09-14T05:30:54.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1679999999999998e-06 2024-09-14T05:30:54.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462514+0000", 2024-09-14T05:30:54.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055438551000000003 2024-09-14T05:30:54.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462562+0000", 2024-09-14T05:30:54.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7649999999999999e-05 2024-09-14T05:30:54.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462805+0000", 2024-09-14T05:30:54.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024263400000000001 2024-09-14T05:30:54.693 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.693 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.693 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.693 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4052 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 12550144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.693 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:54.693 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.220425113000005, 2024-09-14T05:30:54.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060872904999999998, 2024-09-14T05:30:54.693 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.693 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.694 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.694 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.694 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.694 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.694 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4052 2024-09-14T05:30:54.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.694 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:54.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:54.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417486+0000", 2024-09-14T05:30:54.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-14T05:30:54.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417487+0000", 2024-09-14T05:30:54.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4499999999999995e-07 2024-09-14T05:30:54.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417489+0000", 2024-09-14T05:30:54.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8899999999999999e-06 2024-09-14T05:30:54.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417492+0000", 2024-09-14T05:30:54.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4389999999999999e-06 2024-09-14T05:30:54.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478027+0000", 2024-09-14T05:30:54.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060534959999999999 2024-09-14T05:30:54.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478079+0000", 2024-09-14T05:30:54.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2104999999999997e-05 2024-09-14T05:30:54.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478358+0000", 2024-09-14T05:30:54.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000279351 2024-09-14T05:30:54.697 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.697 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.698 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.698 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4053 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13041664~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.698 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:54.698 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.220415728999996, 2024-09-14T05:30:54.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061258866000000002, 2024-09-14T05:30:54.698 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.698 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.698 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.698 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.698 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.699 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4053 2024-09-14T05:30:54.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.699 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:54.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:54.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:54.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.17e-07 2024-09-14T05:30:54.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417496+0000", 2024-09-14T05:30:54.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0890000000000001e-06 2024-09-14T05:30:54.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417499+0000", 2024-09-14T05:30:54.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8629999999999999e-06 2024-09-14T05:30:54.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417502+0000", 2024-09-14T05:30:54.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6290000000000002e-06 2024-09-14T05:30:54.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478387+0000", 2024-09-14T05:30:54.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060885433000000003 2024-09-14T05:30:54.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478435+0000", 2024-09-14T05:30:54.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8167999999999997e-05 2024-09-14T05:30:54.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478754+0000", 2024-09-14T05:30:54.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031846700000000001 2024-09-14T05:30:54.702 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.702 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.702 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.702 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4054 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13533184~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.702 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:54.702 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.220405845000002, 2024-09-14T05:30:54.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061604694000000002, 2024-09-14T05:30:54.703 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.703 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.703 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.703 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.703 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.703 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.703 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4054 2024-09-14T05:30:54.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.703 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:54.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:54.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:54.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.04e-07 2024-09-14T05:30:54.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417506+0000", 2024-09-14T05:30:54.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8299999999999995e-07 2024-09-14T05:30:54.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417509+0000", 2024-09-14T05:30:54.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.88e-06 2024-09-14T05:30:54.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417511+0000", 2024-09-14T05:30:54.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5950000000000001e-06 2024-09-14T05:30:54.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478784+0000", 2024-09-14T05:30:54.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061272971000000002 2024-09-14T05:30:54.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478841+0000", 2024-09-14T05:30:54.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-14T05:30:54.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479109+0000", 2024-09-14T05:30:54.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026794600000000003 2024-09-14T05:30:54.706 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.707 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.707 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.707 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4055 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14024704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.707 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:54.707 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.220396143000002, 2024-09-14T05:30:54.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061932299000000003, 2024-09-14T05:30:54.707 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.707 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.707 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.707 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.707 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.707 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.707 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4055 2024-09-14T05:30:54.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.708 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:54.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:54.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417515+0000", 2024-09-14T05:30:54.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-14T05:30:54.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417516+0000", 2024-09-14T05:30:54.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7199999999999997e-07 2024-09-14T05:30:54.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417519+0000", 2024-09-14T05:30:54.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.937e-06 2024-09-14T05:30:54.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417522+0000", 2024-09-14T05:30:54.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1420000000000002e-06 2024-09-14T05:30:54.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479140+0000", 2024-09-14T05:30:54.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061617933999999999 2024-09-14T05:30:54.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479188+0000", 2024-09-14T05:30:54.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8717000000000001e-05 2024-09-14T05:30:54.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479447+0000", 2024-09-14T05:30:54.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025838800000000002 2024-09-14T05:30:54.711 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.711 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.711 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.711 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4056 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14516224~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.711 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:54.711 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.220385969999995, 2024-09-14T05:30:54.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062263185999999998, 2024-09-14T05:30:54.712 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.712 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.712 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.712 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.712 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.712 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.712 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4056 2024-09-14T05:30:54.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.712 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:54.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:54.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:54.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8700000000000002e-07 2024-09-14T05:30:54.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417528+0000", 2024-09-14T05:30:54.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.579e-06 2024-09-14T05:30:54.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417533+0000", 2024-09-14T05:30:54.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2400000000000001e-06 2024-09-14T05:30:54.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417536+0000", 2024-09-14T05:30:54.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0309999999999999e-06 2024-09-14T05:30:54.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479477+0000", 2024-09-14T05:30:54.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061940967 2024-09-14T05:30:54.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479523+0000", 2024-09-14T05:30:54.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6335000000000003e-05 2024-09-14T05:30:54.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479788+0000", 2024-09-14T05:30:54.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026474700000000002 2024-09-14T05:30:54.715 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.716 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.716 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.716 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4057 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15007744~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.716 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:54.716 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.220371509000003, 2024-09-14T05:30:54.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062589928000000003, 2024-09-14T05:30:54.716 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.716 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.716 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.716 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.716 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.716 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.717 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4057 2024-09-14T05:30:54.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.717 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:54.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:54.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422838+0000", 2024-09-14T05:30:54.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052990349999999997 2024-09-14T05:30:54.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422854+0000", 2024-09-14T05:30:54.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5523999999999999e-05 2024-09-14T05:30:54.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422864+0000", 2024-09-14T05:30:54.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0084e-05 2024-09-14T05:30:54.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422872+0000", 2024-09-14T05:30:54.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-14T05:30:54.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479813+0000", 2024-09-14T05:30:54.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056941389000000002 2024-09-14T05:30:54.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479868+0000", 2024-09-14T05:30:54.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4973999999999999e-05 2024-09-14T05:30:54.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.480129+0000", 2024-09-14T05:30:54.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026084200000000001 2024-09-14T05:30:54.720 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.720 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.720 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.721 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4171 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14057472~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.721 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:54.721 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.208373949000006, 2024-09-14T05:30:54.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061184929999999998, 2024-09-14T05:30:54.721 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.721 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.721 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.721 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.721 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.721 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.721 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4171 2024-09-14T05:30:54.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.721 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:54.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:54.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:54.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.39e-07 2024-09-14T05:30:54.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429541+0000", 2024-09-14T05:30:54.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:30:54.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429547+0000", 2024-09-14T05:30:54.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3280000000000002e-06 2024-09-14T05:30:54.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429552+0000", 2024-09-14T05:30:54.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4859999999999997e-06 2024-09-14T05:30:54.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490379+0000", 2024-09-14T05:30:54.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060826681 2024-09-14T05:30:54.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490434+0000", 2024-09-14T05:30:54.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4577999999999998e-05 2024-09-14T05:30:54.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490722+0000", 2024-09-14T05:30:54.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028781400000000001 2024-09-14T05:30:54.725 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.725 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.725 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.725 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4172 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14548992~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.725 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:54.725 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.208351941000004, 2024-09-14T05:30:54.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061498256000000001, 2024-09-14T05:30:54.725 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.725 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.725 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.725 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.726 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.726 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.726 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4172 2024-09-14T05:30:54.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.726 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:54.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:54.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:54.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.72e-07 2024-09-14T05:30:54.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429561+0000", 2024-09-14T05:30:54.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.717e-06 2024-09-14T05:30:54.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429566+0000", 2024-09-14T05:30:54.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6529999999999999e-06 2024-09-14T05:30:54.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429571+0000", 2024-09-14T05:30:54.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5979999999999999e-06 2024-09-14T05:30:54.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490751+0000", 2024-09-14T05:30:54.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061180336000000002 2024-09-14T05:30:54.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490798+0000", 2024-09-14T05:30:54.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6875000000000001e-05 2024-09-14T05:30:54.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491057+0000", 2024-09-14T05:30:54.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025870500000000002 2024-09-14T05:30:54.729 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.729 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.729 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.730 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4173 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15040512~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:54.730 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:54.730 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.208334315000002, 2024-09-14T05:30:54.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061894820000000003, 2024-09-14T05:30:54.730 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:54.730 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:54.730 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:54.730 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:54.730 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:54.730 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:54.730 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4173 2024-09-14T05:30:54.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.730 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:54.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:54.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:54.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:54.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:54.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:54.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:54.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429577+0000", 2024-09-14T05:30:54.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-14T05:30:54.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:54.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429578+0000", 2024-09-14T05:30:54.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6980000000000001e-06 2024-09-14T05:30:54.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:54.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429585+0000", 2024-09-14T05:30:54.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.725e-06 2024-09-14T05:30:54.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:54.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429590+0000", 2024-09-14T05:30:54.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.938e-06 2024-09-14T05:30:54.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:54.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491091+0000", 2024-09-14T05:30:54.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061500773000000002 2024-09-14T05:30:54.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:54.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491136+0000", 2024-09-14T05:30:54.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5284000000000002e-05 2024-09-14T05:30:54.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:54.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:54.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:54.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491471+0000", 2024-09-14T05:30:54.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033485699999999998 2024-09-14T05:30:54.733 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.734 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.734 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.734 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:54.734 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:54.734 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:54.735 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:30:54.736 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:54.734+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:54.836 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:54.835+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:54.839 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:54.839 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:54.839 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:54.839 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:54.849 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:30:54.938 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:54.937+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:54.952 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:54.952 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:54.961 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:30:55.038 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:55.038+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:55.067 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:55.067 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:55.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1205 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.867966194000005, 2024-09-14T05:30:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0086364279999999998, 2024-09-14T05:30:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1205 2024-09-14T05:30:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197891+0000", 2024-09-14T05:30:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.364e-06 2024-09-14T05:30:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197896+0000", 2024-09-14T05:30:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3000000000000001e-06 2024-09-14T05:30:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197910+0000", 2024-09-14T05:30:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3339000000000001e-05 2024-09-14T05:30:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197919+0000", 2024-09-14T05:30:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8869999999999998e-06 2024-09-14T05:30:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197974+0000", 2024-09-14T05:30:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5720000000000002e-05 2024-09-14T05:30:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.198094+0000", 2024-09-14T05:30:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119739 2024-09-14T05:30:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206526+0000", 2024-09-14T05:30:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084320790000000003 2024-09-14T05:30:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1220 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.859209829999998, 2024-09-14T05:30:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012822337, 2024-09-14T05:30:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1220 2024-09-14T05:30:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206647+0000", 2024-09-14T05:30:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4600000000000003e-07 2024-09-14T05:30:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206652+0000", 2024-09-14T05:30:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6619999999999996e-06 2024-09-14T05:30:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206662+0000", 2024-09-14T05:30:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.995e-06 2024-09-14T05:30:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206667+0000", 2024-09-14T05:30:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-14T05:30:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206706+0000", 2024-09-14T05:30:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9523000000000001e-05 2024-09-14T05:30:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206755+0000", 2024-09-14T05:30:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000002e-05 2024-09-14T05:30:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206877+0000", 2024-09-14T05:30:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121938 2024-09-14T05:30:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.207310+0000", 2024-09-14T05:30:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000432452 2024-09-14T05:30:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219364+0000", 2024-09-14T05:30:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012054481000000001 2024-09-14T05:30:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219425+0000", 2024-09-14T05:30:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0618000000000001e-05 2024-09-14T05:30:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219468+0000", 2024-09-14T05:30:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3739000000000003e-05 2024-09-14T05:30:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1243 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.846244603000002, 2024-09-14T05:30:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078660580000000004, 2024-09-14T05:30:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1243 2024-09-14T05:30:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219612+0000", 2024-09-14T05:30:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.09e-07 2024-09-14T05:30:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219616+0000", 2024-09-14T05:30:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.216e-06 2024-09-14T05:30:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219630+0000", 2024-09-14T05:30:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3064999999999999e-05 2024-09-14T05:30:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219637+0000", 2024-09-14T05:30:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7409999999999992e-06 2024-09-14T05:30:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219697+0000", 2024-09-14T05:30:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.02e-05 2024-09-14T05:30:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219833+0000", 2024-09-14T05:30:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013595600000000001 2024-09-14T05:30:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227477+0000", 2024-09-14T05:30:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076439710000000003 2024-09-14T05:30:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1264 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.838233810000006, 2024-09-14T05:30:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080867990000000004, 2024-09-14T05:30:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1264 2024-09-14T05:30:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227623+0000", 2024-09-14T05:30:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:30:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227627+0000", 2024-09-14T05:30:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.194e-06 2024-09-14T05:30:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227639+0000", 2024-09-14T05:30:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1296e-05 2024-09-14T05:30:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227648+0000", 2024-09-14T05:30:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2280000000000004e-06 2024-09-14T05:30:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227689+0000", 2024-09-14T05:30:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1329999999999999e-05 2024-09-14T05:30:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227741+0000", 2024-09-14T05:30:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1641000000000002e-05 2024-09-14T05:30:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:30:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227914+0000", 2024-09-14T05:30:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000173215 2024-09-14T05:30:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228474+0000", 2024-09-14T05:30:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055980199999999998 2024-09-14T05:30:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235608+0000", 2024-09-14T05:30:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0071343830000000002 2024-09-14T05:30:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235669+0000", 2024-09-14T05:30:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1068999999999994e-05 2024-09-14T05:30:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235709+0000", 2024-09-14T05:30:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9671e-05 2024-09-14T05:30:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1526 3.4 3:2695cbe4:::rbd_data.112dd25fb4c.0000000000000119:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.739693247000005, 2024-09-14T05:30:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0079537359999999994, 2024-09-14T05:30:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1526 2024-09-14T05:30:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3600000000000003e-07 2024-09-14T05:30:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326168+0000", 2024-09-14T05:30:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3460000000000004e-06 2024-09-14T05:30:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326178+0000", 2024-09-14T05:30:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0291e-05 2024-09-14T05:30:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326183+0000", 2024-09-14T05:30:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-14T05:30:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326230+0000", 2024-09-14T05:30:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6650999999999998e-05 2024-09-14T05:30:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326295+0000", 2024-09-14T05:30:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5158000000000002e-05 2024-09-14T05:30:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326408+0000", 2024-09-14T05:30:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113039 2024-09-14T05:30:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.331648+0000", 2024-09-14T05:30:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052404929999999997 2024-09-14T05:30:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334011+0000", 2024-09-14T05:30:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002362669 2024-09-14T05:30:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334064+0000", 2024-09-14T05:30:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3198999999999997e-05 2024-09-14T05:30:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334116+0000", 2024-09-14T05:30:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2133999999999999e-05 2024-09-14T05:30:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1531 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.736875080999994, 2024-09-14T05:30:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082114309999999999, 2024-09-14T05:30:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1531 2024-09-14T05:30:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328982+0000", 2024-09-14T05:30:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:30:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328985+0000", 2024-09-14T05:30:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7900000000000001e-06 2024-09-14T05:30:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328996+0000", 2024-09-14T05:30:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.028e-05 2024-09-14T05:30:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329001+0000", 2024-09-14T05:30:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0610000000000003e-06 2024-09-14T05:30:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329035+0000", 2024-09-14T05:30:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4366999999999998e-05 2024-09-14T05:30:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329082+0000", 2024-09-14T05:30:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6646e-05 2024-09-14T05:30:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329195+0000", 2024-09-14T05:30:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113279 2024-09-14T05:30:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334293+0000", 2024-09-14T05:30:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050980399999999999 2024-09-14T05:30:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337101+0000", 2024-09-14T05:30:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0028076170000000001 2024-09-14T05:30:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337154+0000", 2024-09-14T05:30:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3641999999999999e-05 2024-09-14T05:30:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337192+0000", 2024-09-14T05:30:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7932000000000002e-05 2024-09-14T05:30:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:2131 3.b e30/24 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head v 30'36, mlcod=30'34)", 2024-09-14T05:30:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.273527725999998, 2024-09-14T05:30:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031871709999999998, 2024-09-14T05:30:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792331+0000", 2024-09-14T05:30:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6900000000000001e-06 2024-09-14T05:30:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797319+0000", 2024-09-14T05:30:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049876349999999998 2024-09-14T05:30:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797349+0000", 2024-09-14T05:30:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0046e-05 2024-09-14T05:30:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797359+0000", 2024-09-14T05:30:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0037999999999999e-05 2024-09-14T05:30:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797445+0000", 2024-09-14T05:30:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6132999999999997e-05 2024-09-14T05:30:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797472+0000", 2024-09-14T05:30:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7478999999999999e-05 2024-09-14T05:30:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824161+0000", 2024-09-14T05:30:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026688714999999998 2024-09-14T05:30:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824200+0000", 2024-09-14T05:30:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8974000000000003e-05 2024-09-14T05:30:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4259 3.4 3:25f21dc9:::rbd_data.112d3d611e54.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 294912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.674355112000001, 2024-09-14T05:30:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010043553, 2024-09-14T05:30:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4259 2024-09-14T05:30:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391503+0000", 2024-09-14T05:30:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9099999999999999e-06 2024-09-14T05:30:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391821+0000", 2024-09-14T05:30:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031871800000000001 2024-09-14T05:30:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391844+0000", 2024-09-14T05:30:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2045e-05 2024-09-14T05:30:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391853+0000", 2024-09-14T05:30:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7249999999999992e-06 2024-09-14T05:30:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391917+0000", 2024-09-14T05:30:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3458000000000001e-05 2024-09-14T05:30:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392037+0000", 2024-09-14T05:30:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120449 2024-09-14T05:30:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392305+0000", 2024-09-14T05:30:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026739299999999999 2024-09-14T05:30:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.393609+0000", 2024-09-14T05:30:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013042450000000001 2024-09-14T05:30:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401430+0000", 2024-09-14T05:30:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078211670000000004 2024-09-14T05:30:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401499+0000", 2024-09-14T05:30:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9374000000000004e-05 2024-09-14T05:30:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401544+0000", 2024-09-14T05:30:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5068999999999998e-05 2024-09-14T05:30:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4554 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.081454300999994, 2024-09-14T05:30:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069998999999999999, 2024-09-14T05:30:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4554 2024-09-14T05:30:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9299999999999999e-07 2024-09-14T05:30:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984404+0000", 2024-09-14T05:30:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.855e-06 2024-09-14T05:30:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984407+0000", 2024-09-14T05:30:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-14T05:30:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984410+0000", 2024-09-14T05:30:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8159999999999998e-06 2024-09-14T05:30:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985511+0000", 2024-09-14T05:30:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00110053 2024-09-14T05:30:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985561+0000", 2024-09-14T05:30:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0389000000000002e-05 2024-09-14T05:30:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985772+0000", 2024-09-14T05:30:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000210571 2024-09-14T05:30:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.987075+0000", 2024-09-14T05:30:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001303128 2024-09-14T05:30:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991303+0000", 2024-09-14T05:30:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042283310000000001 2024-09-14T05:30:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991359+0000", 2024-09-14T05:30:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6335999999999997e-05 2024-09-14T05:30:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991402+0000", 2024-09-14T05:30:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2074999999999999e-05 2024-09-14T05:30:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4711 3.4 3:226df663:::rbd_data.112d3d611e54.000000000000002d:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.898783144999996, 2024-09-14T05:30:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011304193000000001, 2024-09-14T05:30:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4711 2024-09-14T05:30:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167074+0000", 2024-09-14T05:30:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0999999999999997e-07 2024-09-14T05:30:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167078+0000", 2024-09-14T05:30:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.878e-06 2024-09-14T05:30:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167087+0000", 2024-09-14T05:30:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6940000000000007e-06 2024-09-14T05:30:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167093+0000", 2024-09-14T05:30:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7740000000000001e-06 2024-09-14T05:30:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167128+0000", 2024-09-14T05:30:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5231999999999997e-05 2024-09-14T05:30:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167207+0000", 2024-09-14T05:30:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9036999999999996e-05 2024-09-14T05:30:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.169140+0000", 2024-09-14T05:30:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001932897 2024-09-14T05:30:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178258+0000", 2024-09-14T05:30:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091180819999999996 2024-09-14T05:30:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178314+0000", 2024-09-14T05:30:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5571e-05 2024-09-14T05:30:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178348+0000", 2024-09-14T05:30:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3747000000000001e-05 2024-09-14T05:30:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178377+0000", 2024-09-14T05:30:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9471e-05 2024-09-14T05:30:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:72582 3.b e47/24 3:d5072567:::rbd_data.112d4a90444c.0000000000000119:head v 47'386, mlcod=46'383)", 2024-09-14T05:30:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.540829291999998, 2024-09-14T05:30:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0067720940000000002, 2024-09-14T05:30:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1100000000000002e-07 2024-09-14T05:30:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525031+0000", 2024-09-14T05:30:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0740000000000003e-06 2024-09-14T05:30:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525040+0000", 2024-09-14T05:30:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5329999999999994e-06 2024-09-14T05:30:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525045+0000", 2024-09-14T05:30:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:30:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525090+0000", 2024-09-14T05:30:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4743000000000002e-05 2024-09-14T05:30:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525102+0000", 2024-09-14T05:30:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2067e-05 2024-09-14T05:30:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531757+0000", 2024-09-14T05:30:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066550380000000003 2024-09-14T05:30:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531799+0000", 2024-09-14T05:30:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1640000000000001e-05 2024-09-14T05:30:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.7.0:509 3.b e100/24 3:d1b3949e:::rbd_data.112d46b1ca94.000000000000010c:11 v 100'523, mlcod=99'518)", 2024-09-14T05:30:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.009409+0000", 2024-09-14T05:30:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.0564471729999996, 2024-09-14T05:30:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075659999999999998, 2024-09-14T05:30:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009409+0000", 2024-09-14T05:30:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009409+0000", 2024-09-14T05:30:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009409+0000", 2024-09-14T05:30:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7599999999999997e-07 2024-09-14T05:30:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009421+0000", 2024-09-14T05:30:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1705999999999999e-05 2024-09-14T05:30:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009432+0000", 2024-09-14T05:30:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0769e-05 2024-09-14T05:30:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009439+0000", 2024-09-14T05:30:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5969999999999997e-06 2024-09-14T05:30:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009505+0000", 2024-09-14T05:30:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6310999999999996e-05 2024-09-14T05:30:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009521+0000", 2024-09-14T05:30:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.571e-05 2024-09-14T05:30:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.016931+0000", 2024-09-14T05:30:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0074101619999999997 2024-09-14T05:30:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.016975+0000", 2024-09-14T05:30:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4069000000000001e-05 2024-09-14T05:30:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.2.0:1434 3.5 e100/24 3:a529dc6d:::rbd_data.112d46b1ca94.00000000000000d4:11 v 100'327, mlcod=100'321)", 2024-09-14T05:30:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.013998+0000", 2024-09-14T05:30:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.0518577560000004, 2024-09-14T05:30:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0083544610000000005, 2024-09-14T05:30:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.013998+0000", 2024-09-14T05:30:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.013998+0000", 2024-09-14T05:30:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.013999+0000", 2024-09-14T05:30:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-14T05:30:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014003+0000", 2024-09-14T05:30:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.51e-06 2024-09-14T05:30:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014013+0000", 2024-09-14T05:30:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3379999999999994e-06 2024-09-14T05:30:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014018+0000", 2024-09-14T05:30:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5269999999999998e-06 2024-09-14T05:30:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014843+0000", 2024-09-14T05:30:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00082516000000000002 2024-09-14T05:30:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014871+0000", 2024-09-14T05:30:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7628e-05 2024-09-14T05:30:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.022313+0000", 2024-09-14T05:30:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.007441563 2024-09-14T05:30:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.022353+0000", 2024-09-14T05:30:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0055000000000002e-05 2024-09-14T05:30:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.0.0:1713 3.15 e100/24 3:a916a3d3:::rbd_data.112d46b1ca94.00000000000000d3:11 v 100'427, mlcod=99'422)", 2024-09-14T05:30:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.014451+0000", 2024-09-14T05:30:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.0514044719999998, 2024-09-14T05:30:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078576710000000001, 2024-09-14T05:30:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014451+0000", 2024-09-14T05:30:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014451+0000", 2024-09-14T05:30:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014452+0000", 2024-09-14T05:30:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9599999999999998e-07 2024-09-14T05:30:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014458+0000", 2024-09-14T05:30:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8699999999999997e-06 2024-09-14T05:30:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014469+0000", 2024-09-14T05:30:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0627e-05 2024-09-14T05:30:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014476+0000", 2024-09-14T05:30:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.046e-06 2024-09-14T05:30:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014995+0000", 2024-09-14T05:30:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051870599999999996 2024-09-14T05:30:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.015014+0000", 2024-09-14T05:30:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9768000000000001e-05 2024-09-14T05:30:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.022266+0000", 2024-09-14T05:30:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0072513430000000004 2024-09-14T05:30:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.022309+0000", 2024-09-14T05:30:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3514999999999999e-05 2024-09-14T05:30:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.7.0:510 3.b e100/24 3:d5c4519c:::rbd_data.112d46b1ca94.0000000000000024:11 v 100'525, mlcod=100'521)", 2024-09-14T05:30:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.017655+0000", 2024-09-14T05:30:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.0482010119999998, 2024-09-14T05:30:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0073483380000000003, 2024-09-14T05:30:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017655+0000", 2024-09-14T05:30:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017655+0000", 2024-09-14T05:30:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017656+0000", 2024-09-14T05:30:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8800000000000002e-07 2024-09-14T05:30:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017660+0000", 2024-09-14T05:30:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2370000000000003e-06 2024-09-14T05:30:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017669+0000", 2024-09-14T05:30:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7749999999999998e-06 2024-09-14T05:30:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017675+0000", 2024-09-14T05:30:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1990000000000003e-06 2024-09-14T05:30:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017727+0000", 2024-09-14T05:30:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1835000000000001e-05 2024-09-14T05:30:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017745+0000", 2024-09-14T05:30:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8233999999999999e-05 2024-09-14T05:30:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.024955+0000", 2024-09-14T05:30:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0072104279999999996 2024-09-14T05:30:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.025003+0000", 2024-09-14T05:30:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7942000000000003e-05 2024-09-14T05:30:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.5.0:2812 3.4 e100/39 3:2237b773:::rbd_data.112d46b1ca94.00000000000001f5:11 v 100'550, mlcod=100'546)", 2024-09-14T05:30:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.018029+0000", 2024-09-14T05:30:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.0478269869999997, 2024-09-14T05:30:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0071886470000000003, 2024-09-14T05:30:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018029+0000", 2024-09-14T05:30:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018029+0000", 2024-09-14T05:30:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018030+0000", 2024-09-14T05:30:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3900000000000004e-07 2024-09-14T05:30:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018036+0000", 2024-09-14T05:30:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9669999999999998e-06 2024-09-14T05:30:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018049+0000", 2024-09-14T05:30:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2897e-05 2024-09-14T05:30:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018055+0000", 2024-09-14T05:30:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8109999999999997e-06 2024-09-14T05:30:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018104+0000", 2024-09-14T05:30:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8408999999999997e-05 2024-09-14T05:30:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018125+0000", 2024-09-14T05:30:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0992000000000002e-05 2024-09-14T05:30:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.025174+0000", 2024-09-14T05:30:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0070491850000000003 2024-09-14T05:30:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.025218+0000", 2024-09-14T05:30:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3646999999999997e-05 2024-09-14T05:30:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.7.0:511 3.b e100/24 3:d76f30d4:::rbd_data.112d46b1ca94.0000000000000048:11 v 100'527, mlcod=100'521)", 2024-09-14T05:30:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.018634+0000", 2024-09-14T05:30:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.0472222110000002, 2024-09-14T05:30:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0090907680000000008, 2024-09-14T05:30:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018634+0000", 2024-09-14T05:30:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018634+0000", 2024-09-14T05:30:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018634+0000", 2024-09-14T05:30:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5899999999999996e-07 2024-09-14T05:30:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018639+0000", 2024-09-14T05:30:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5229999999999997e-06 2024-09-14T05:30:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018649+0000", 2024-09-14T05:30:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5999999999999996e-06 2024-09-14T05:30:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018656+0000", 2024-09-14T05:30:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5789999999999998e-06 2024-09-14T05:30:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.020500+0000", 2024-09-14T05:30:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0018434840000000001 2024-09-14T05:30:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.020522+0000", 2024-09-14T05:30:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2892999999999999e-05 2024-09-14T05:30:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.027685+0000", 2024-09-14T05:30:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.007162758 2024-09-14T05:30:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.027725+0000", 2024-09-14T05:30:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9271999999999999e-05 2024-09-14T05:30:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.2.0:1440 3.5 e100/24 3:a70fccf8:::rbd_data.112d46b1ca94.00000000000000ec:11 v 100'329, mlcod=100'325)", 2024-09-14T05:30:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.023047+0000", 2024-09-14T05:30:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.0428086739999998, 2024-09-14T05:30:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.007561229, 2024-09-14T05:30:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023047+0000", 2024-09-14T05:30:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023047+0000", 2024-09-14T05:30:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023048+0000", 2024-09-14T05:30:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0999999999999997e-07 2024-09-14T05:30:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023054+0000", 2024-09-14T05:30:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9510000000000003e-06 2024-09-14T05:30:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023067+0000", 2024-09-14T05:30:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2959000000000001e-05 2024-09-14T05:30:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023074+0000", 2024-09-14T05:30:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0450000000000003e-06 2024-09-14T05:30:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023155+0000", 2024-09-14T05:30:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1422999999999996e-05 2024-09-14T05:30:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023179+0000", 2024-09-14T05:30:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.376e-05 2024-09-14T05:30:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.030542+0000", 2024-09-14T05:30:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0073630859999999996 2024-09-14T05:30:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.030609+0000", 2024-09-14T05:30:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6194999999999999e-05 2024-09-14T05:30:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.0.0:1718 3.15 e100/24 3:add139e1:::rbd_data.112d46b1ca94.0000000000000007:11 v 100'429, mlcod=100'425)", 2024-09-14T05:30:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.023071+0000", 2024-09-14T05:30:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.0427848620000004, 2024-09-14T05:30:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076044830000000004, 2024-09-14T05:30:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023071+0000", 2024-09-14T05:30:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023071+0000", 2024-09-14T05:30:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023072+0000", 2024-09-14T05:30:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2099999999999996e-07 2024-09-14T05:30:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023077+0000", 2024-09-14T05:30:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3759999999999999e-06 2024-09-14T05:30:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023085+0000", 2024-09-14T05:30:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2009999999999999e-06 2024-09-14T05:30:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023091+0000", 2024-09-14T05:30:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3720000000000004e-06 2024-09-14T05:30:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023241+0000", 2024-09-14T05:30:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015081799999999999 2024-09-14T05:30:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023261+0000", 2024-09-14T05:30:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9076000000000002e-05 2024-09-14T05:30:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.030614+0000", 2024-09-14T05:30:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00735347 2024-09-14T05:30:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.030676+0000", 2024-09-14T05:30:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1549000000000003e-05 2024-09-14T05:30:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.0.0:1719 3.15 e100/24 3:adb89761:::rbd_data.112d46b1ca94.00000000000003a3:11 v 100'431, mlcod=100'425)", 2024-09-14T05:30:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.023954+0000", 2024-09-14T05:30:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.0419019250000003, 2024-09-14T05:30:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094961170000000001, 2024-09-14T05:30:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023954+0000", 2024-09-14T05:30:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023954+0000", 2024-09-14T05:30:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023955+0000", 2024-09-14T05:30:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7400000000000003e-07 2024-09-14T05:30:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023960+0000", 2024-09-14T05:30:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1460000000000004e-06 2024-09-14T05:30:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023970+0000", 2024-09-14T05:30:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0455000000000001e-05 2024-09-14T05:30:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023976+0000", 2024-09-14T05:30:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1659999999999999e-06 2024-09-14T05:30:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.025476+0000", 2024-09-14T05:30:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001499088 2024-09-14T05:30:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.025498+0000", 2024-09-14T05:30:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2041000000000001e-05 2024-09-14T05:30:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.033380+0000", 2024-09-14T05:30:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078826269999999997 2024-09-14T05:30:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.033450+0000", 2024-09-14T05:30:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9919999999999995e-05 2024-09-14T05:30:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:55.176 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:55.176 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:30:55.177 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T05:30:55.178 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:55.177+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:55.179 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:30:55.279 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:55.279+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:55.311 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:55.311 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:55.326 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:55.326 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:55.326 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:55.326 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:55.328 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:55.328 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:55.335 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:30:55.344 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:55.344 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:55.360 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:55.361 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:55.378 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:55.379 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:55.379 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:55.379+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:55.395 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:55.395 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:55.411 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:55.412 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:55.428 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:55.428 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:55.441 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:55.441 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:55.450 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:30:55.481 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:55.480+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:55.582 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:55.581+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:55.594 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:55.594 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:55.603 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:30:55.682 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:55.681+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:55.707 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:55.707 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:55.716 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:30:55.801 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:55.800+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:55.859 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:55.859 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:55.867 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:30:55.885 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:55.883+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:55.985 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:55.985+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:56.011 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:56.011 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:56.020 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:30:56.103 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:56.103+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:56.119 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-14T05:30:56.173 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:56.173 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:56.183 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:30:56.225 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:56.224+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:56.288 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:56.287+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:56.327 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:56.327 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:56.336 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:30:56.389 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:56.389+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:56.439 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:56.439 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:56.448 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:30:56.491 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:56.490+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:56.590 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:56.590 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:56.591 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:56.591+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:56.599 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:30:56.692 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:56.691+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:56.744 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:56.744 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:56.752 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:30:56.793 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:56.792+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:56.894 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:56.893+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:56.896 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:56.896 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:56.905 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:30:56.996 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:56.995+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:57.008 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:57.009 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:57.017 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:30:57.131 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:57.130+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:57.196 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:57.196+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:57.202 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:57.202 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:57.213 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:30:57.299 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:57.298+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:57.358 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:57.358 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:57.369 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:30:57.411 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:57.410+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:57.439 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:30:57.473 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:57.473 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:57.481 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:30:57.525 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:57.524+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:57.558 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:57.558 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:57.575 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:57.575 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:57.585 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:30:57.585 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:30:57.590 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:57.590 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:57.593 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:30:57.599 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:57.598+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:57.606 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:57.606 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:57.622 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:57.623 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:57.638 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:30:57.639 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:30:57.655 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:30:57.655 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:30:57.672 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:30:57.672 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:30:57.699 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:57.698+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:57.740 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:57.740 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:57.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:57.740 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:57.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3523 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6062080~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.224281891000004, 2024-09-14T05:30:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096911030999999995, 2024-09-14T05:30:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3523 2024-09-14T05:30:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:30:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514359+0000", 2024-09-14T05:30:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9299999999999999e-07 2024-09-14T05:30:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514361+0000", 2024-09-14T05:30:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514366+0000", 2024-09-14T05:30:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1660000000000002e-06 2024-09-14T05:30:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514371+0000", 2024-09-14T05:30:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.648e-06 2024-09-14T05:30:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.610954+0000", 2024-09-14T05:30:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096582170999999994 2024-09-14T05:30:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611002+0000", 2024-09-14T05:30:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7923999999999997e-05 2024-09-14T05:30:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611269+0000", 2024-09-14T05:30:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026789499999999998 2024-09-14T05:30:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3524 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6553600~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.224262698999993, 2024-09-14T05:30:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097263347999999999, 2024-09-14T05:30:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3524 2024-09-14T05:30:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:30:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514380+0000", 2024-09-14T05:30:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6500000000000001e-06 2024-09-14T05:30:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514386+0000", 2024-09-14T05:30:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7360000000000004e-06 2024-09-14T05:30:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514391+0000", 2024-09-14T05:30:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2830000000000003e-06 2024-09-14T05:30:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611295+0000", 2024-09-14T05:30:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096904814000000006 2024-09-14T05:30:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611343+0000", 2024-09-14T05:30:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7506999999999998e-05 2024-09-14T05:30:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611641+0000", 2024-09-14T05:30:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029798800000000001 2024-09-14T05:30:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3525 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7045120~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.224240812000005, 2024-09-14T05:30:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097644422999999994, 2024-09-14T05:30:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3525 2024-09-14T05:30:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:30:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514400+0000", 2024-09-14T05:30:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.77e-07 2024-09-14T05:30:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514402+0000", 2024-09-14T05:30:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6530000000000001e-06 2024-09-14T05:30:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514411+0000", 2024-09-14T05:30:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7429999999999997e-06 2024-09-14T05:30:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514417+0000", 2024-09-14T05:30:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2249999999999997e-06 2024-09-14T05:30:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611668+0000", 2024-09-14T05:30:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097250681000000005 2024-09-14T05:30:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611716+0000", 2024-09-14T05:30:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7336000000000002e-05 2024-09-14T05:30:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612044+0000", 2024-09-14T05:30:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032840799999999998 2024-09-14T05:30:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3526 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7536640~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.224216327999997, 2024-09-14T05:30:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097943884999999994, 2024-09-14T05:30:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3526 2024-09-14T05:30:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:30:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0400000000000002e-07 2024-09-14T05:30:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514426+0000", 2024-09-14T05:30:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:30:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514432+0000", 2024-09-14T05:30:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.105e-06 2024-09-14T05:30:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514438+0000", 2024-09-14T05:30:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5149999999999997e-06 2024-09-14T05:30:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612072+0000", 2024-09-14T05:30:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097633947999999998 2024-09-14T05:30:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612120+0000", 2024-09-14T05:30:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8458999999999998e-05 2024-09-14T05:30:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612368+0000", 2024-09-14T05:30:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024761999999999999 2024-09-14T05:30:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3527 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 8028160~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.224195614999999, 2024-09-14T05:30:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098244955999999994, 2024-09-14T05:30:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3527 2024-09-14T05:30:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:30:57.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519842+0000", 2024-09-14T05:30:57.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053975230000000004 2024-09-14T05:30:57.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519851+0000", 2024-09-14T05:30:57.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5599999999999994e-06 2024-09-14T05:30:57.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519858+0000", 2024-09-14T05:30:57.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2409999999999998e-06 2024-09-14T05:30:57.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519863+0000", 2024-09-14T05:30:57.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:30:57.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612395+0000", 2024-09-14T05:30:57.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092532552000000004 2024-09-14T05:30:57.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612442+0000", 2024-09-14T05:30:57.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6486000000000002e-05 2024-09-14T05:30:57.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612690+0000", 2024-09-14T05:30:57.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024791599999999999 2024-09-14T05:30:57.768 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.768 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.768 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.768 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3603 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5177344~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.768 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:57.768 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.218643170000007, 2024-09-14T05:30:57.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096796436, 2024-09-14T05:30:57.769 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.769 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.769 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.769 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.769 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.769 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.769 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3603 2024-09-14T05:30:57.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.769 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:57.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:57.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:30:57.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1899999999999999e-07 2024-09-14T05:30:57.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519998+0000", 2024-09-14T05:30:57.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5000000000000001e-07 2024-09-14T05:30:57.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520002+0000", 2024-09-14T05:30:57.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7299999999999999e-06 2024-09-14T05:30:57.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520004+0000", 2024-09-14T05:30:57.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.429e-06 2024-09-14T05:30:57.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616493+0000", 2024-09-14T05:30:57.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096488345000000003 2024-09-14T05:30:57.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616544+0000", 2024-09-14T05:30:57.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1724000000000001e-05 2024-09-14T05:30:57.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616794+0000", 2024-09-14T05:30:57.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000249139 2024-09-14T05:30:57.773 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.773 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.773 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.774 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3604 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5668864~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.774 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:57.774 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.218628531999997, 2024-09-14T05:30:57.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108222999999994, 2024-09-14T05:30:57.774 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.774 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.774 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.776 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.776 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.777 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.777 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3604 2024-09-14T05:30:57.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.777 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:57.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:57.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:30:57.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7000000000000001e-07 2024-09-14T05:30:57.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520013+0000", 2024-09-14T05:30:57.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.389e-06 2024-09-14T05:30:57.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520018+0000", 2024-09-14T05:30:57.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3019999999999996e-06 2024-09-14T05:30:57.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520026+0000", 2024-09-14T05:30:57.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6170000000000007e-06 2024-09-14T05:30:57.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616816+0000", 2024-09-14T05:30:57.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096789766999999999 2024-09-14T05:30:57.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616871+0000", 2024-09-14T05:30:57.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4811000000000001e-05 2024-09-14T05:30:57.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617120+0000", 2024-09-14T05:30:57.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024906700000000001 2024-09-14T05:30:57.781 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.781 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.781 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.781 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3605 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6160384~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.781 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:57.781 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.218610072000004, 2024-09-14T05:30:57.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097423945999999997, 2024-09-14T05:30:57.782 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.782 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.782 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.782 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3605 2024-09-14T05:30:57.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.782 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:57.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:57.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:30:57.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3999999999999998e-07 2024-09-14T05:30:57.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520032+0000", 2024-09-14T05:30:57.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.083e-06 2024-09-14T05:30:57.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520036+0000", 2024-09-14T05:30:57.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.019e-06 2024-09-14T05:30:57.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520038+0000", 2024-09-14T05:30:57.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4849999999999999e-06 2024-09-14T05:30:57.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617147+0000", 2024-09-14T05:30:57.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108665999999996 2024-09-14T05:30:57.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617199+0000", 2024-09-14T05:30:57.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2558000000000001e-05 2024-09-14T05:30:57.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617454+0000", 2024-09-14T05:30:57.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025489499999999999 2024-09-14T05:30:57.787 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.787 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.787 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.787 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3606 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6651904~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.788 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:57.788 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.218599350000005, 2024-09-14T05:30:57.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097745705000000002, 2024-09-14T05:30:57.788 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.788 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.788 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.788 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.788 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.788 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.788 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3606 2024-09-14T05:30:57.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.788 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:57.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:57.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:30:57.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-14T05:30:57.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520042+0000", 2024-09-14T05:30:57.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4799999999999997e-07 2024-09-14T05:30:57.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520046+0000", 2024-09-14T05:30:57.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8149999999999999e-06 2024-09-14T05:30:57.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520049+0000", 2024-09-14T05:30:57.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.576e-06 2024-09-14T05:30:57.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617476+0000", 2024-09-14T05:30:57.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097427558999999997 2024-09-14T05:30:57.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617522+0000", 2024-09-14T05:30:57.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5927999999999998e-05 2024-09-14T05:30:57.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617787+0000", 2024-09-14T05:30:57.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000264663 2024-09-14T05:30:57.792 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.792 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.792 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.793 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3607 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7143424~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.793 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:57.793 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.218588823000005, 2024-09-14T05:30:57.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097871125000000003, 2024-09-14T05:30:57.793 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.793 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.793 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.793 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.793 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.793 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.793 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3607 2024-09-14T05:30:57.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.793 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:57.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:30:57.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520052+0000", 2024-09-14T05:30:57.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.03e-07 2024-09-14T05:30:57.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520053+0000", 2024-09-14T05:30:57.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.85e-07 2024-09-14T05:30:57.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520056+0000", 2024-09-14T05:30:57.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9140000000000001e-06 2024-09-14T05:30:57.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520059+0000", 2024-09-14T05:30:57.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4600000000000002e-06 2024-09-14T05:30:57.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617809+0000", 2024-09-14T05:30:57.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097750216000000001 2024-09-14T05:30:57.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617864+0000", 2024-09-14T05:30:57.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5211999999999999e-05 2024-09-14T05:30:57.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617923+0000", 2024-09-14T05:30:57.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8235000000000001e-05 2024-09-14T05:30:57.797 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.797 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.797 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.797 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3608 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7634944~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.797 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:57.797 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.218578515000004, 2024-09-14T05:30:57.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098199212999999994, 2024-09-14T05:30:57.797 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.798 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.798 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.798 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.798 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.798 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.798 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3608 2024-09-14T05:30:57.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.798 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:57.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:30:57.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525953+0000", 2024-09-14T05:30:57.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058911850000000002 2024-09-14T05:30:57.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525957+0000", 2024-09-14T05:30:57.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2379999999999999e-06 2024-09-14T05:30:57.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525967+0000", 2024-09-14T05:30:57.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5389999999999996e-06 2024-09-14T05:30:57.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525972+0000", 2024-09-14T05:30:57.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.541e-06 2024-09-14T05:30:57.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617945+0000", 2024-09-14T05:30:57.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091972597000000003 2024-09-14T05:30:57.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617992+0000", 2024-09-14T05:30:57.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7397e-05 2024-09-14T05:30:57.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.618261+0000", 2024-09-14T05:30:57.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026871600000000001 2024-09-14T05:30:57.801 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.802 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.802 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.802 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3640 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6586368~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.802 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:57.802 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.212470672999999, 2024-09-14T05:30:57.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097068580000000002, 2024-09-14T05:30:57.802 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.802 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.802 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.802 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.802 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.802 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.803 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3640 2024-09-14T05:30:57.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.803 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:57.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:57.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:30:57.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.23e-07 2024-09-14T05:30:57.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526171+0000", 2024-09-14T05:30:57.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:30:57.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526175+0000", 2024-09-14T05:30:57.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8369999999999999e-06 2024-09-14T05:30:57.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526177+0000", 2024-09-14T05:30:57.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4959999999999999e-06 2024-09-14T05:30:57.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622847+0000", 2024-09-14T05:30:57.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096669790000000005 2024-09-14T05:30:57.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622902+0000", 2024-09-14T05:30:57.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5157000000000001e-05 2024-09-14T05:30:57.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623238+0000", 2024-09-14T05:30:57.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033603399999999997 2024-09-14T05:30:57.806 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.806 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.806 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.806 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3641 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7077888~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.806 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:57.806 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.212460163000003, 2024-09-14T05:30:57.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097371141999999994, 2024-09-14T05:30:57.807 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.807 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.807 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.807 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.807 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.807 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.807 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3641 2024-09-14T05:30:57.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.807 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:57.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:30:57.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531851+0000", 2024-09-14T05:30:57.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005671265 2024-09-14T05:30:57.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531855+0000", 2024-09-14T05:30:57.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0849999999999999e-06 2024-09-14T05:30:57.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531867+0000", 2024-09-14T05:30:57.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1028e-05 2024-09-14T05:30:57.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531872+0000", 2024-09-14T05:30:57.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000003e-06 2024-09-14T05:30:57.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623251+0000", 2024-09-14T05:30:57.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091378980999999998 2024-09-14T05:30:57.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623295+0000", 2024-09-14T05:30:57.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3881999999999998e-05 2024-09-14T05:30:57.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623551+0000", 2024-09-14T05:30:57.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025636099999999998 2024-09-14T05:30:57.811 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.811 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.811 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.811 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3721 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.811 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:57.811 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.206601887999994, 2024-09-14T05:30:57.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096806472000000005, 2024-09-14T05:30:57.811 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.811 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.811 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.811 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.812 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.812 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.812 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3721 2024-09-14T05:30:57.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.812 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:57.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:30:57.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532039+0000", 2024-09-14T05:30:57.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.05e-07 2024-09-14T05:30:57.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532040+0000", 2024-09-14T05:30:57.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9099999999999991e-07 2024-09-14T05:30:57.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532043+0000", 2024-09-14T05:30:57.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6610000000000002e-06 2024-09-14T05:30:57.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532046+0000", 2024-09-14T05:30:57.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4080000000000001e-06 2024-09-14T05:30:57.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628545+0000", 2024-09-14T05:30:57.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096499615999999996 2024-09-14T05:30:57.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628592+0000", 2024-09-14T05:30:57.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6907000000000003e-05 2024-09-14T05:30:57.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628845+0000", 2024-09-14T05:30:57.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025268399999999998 2024-09-14T05:30:57.815 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.815 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.815 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.815 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3722 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6193152~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.816 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:57.816 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.206591807999999, 2024-09-14T05:30:57.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097146587000000006, 2024-09-14T05:30:57.816 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.816 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.816 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.816 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.816 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.816 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.816 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3722 2024-09-14T05:30:57.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.816 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:57.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:30:57.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538191+0000", 2024-09-14T05:30:57.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061429969999999999 2024-09-14T05:30:57.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538200+0000", 2024-09-14T05:30:57.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7069999999999998e-06 2024-09-14T05:30:57.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538212+0000", 2024-09-14T05:30:57.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1504e-05 2024-09-14T05:30:57.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538219+0000", 2024-09-14T05:30:57.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7500000000000003e-06 2024-09-14T05:30:57.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628869+0000", 2024-09-14T05:30:57.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090649224 2024-09-14T05:30:57.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628919+0000", 2024-09-14T05:30:57.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0144e-05 2024-09-14T05:30:57.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.629195+0000", 2024-09-14T05:30:57.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027626099999999998 2024-09-14T05:30:57.820 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.820 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.820 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.820 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3758 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7110656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.820 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:57.820 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.199997592000003, 2024-09-14T05:30:57.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096977544999999998, 2024-09-14T05:30:57.820 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.820 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.820 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.820 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.820 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.821 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.821 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3758 2024-09-14T05:30:57.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.821 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:57.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:57.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:30:57.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:30:57.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538645+0000", 2024-09-14T05:30:57.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.843e-06 2024-09-14T05:30:57.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538651+0000", 2024-09-14T05:30:57.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6169999999999998e-06 2024-09-14T05:30:57.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538655+0000", 2024-09-14T05:30:57.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5399999999999997e-06 2024-09-14T05:30:57.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635225+0000", 2024-09-14T05:30:57.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096570149999999993 2024-09-14T05:30:57.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635272+0000", 2024-09-14T05:30:57.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7199e-05 2024-09-14T05:30:57.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635620+0000", 2024-09-14T05:30:57.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034782600000000003 2024-09-14T05:30:57.824 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.824 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.824 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.825 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3759 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.825 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:57.825 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.199979681000002, 2024-09-14T05:30:57.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097335331999999997, 2024-09-14T05:30:57.825 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.825 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.825 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.825 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3759 2024-09-14T05:30:57.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.826 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:57.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:30:57.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544013+0000", 2024-09-14T05:30:57.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053520829999999997 2024-09-14T05:30:57.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544016+0000", 2024-09-14T05:30:57.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6660000000000001e-06 2024-09-14T05:30:57.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544026+0000", 2024-09-14T05:30:57.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7089999999999997e-06 2024-09-14T05:30:57.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544043+0000", 2024-09-14T05:30:57.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833999999999998e-05 2024-09-14T05:30:57.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635646+0000", 2024-09-14T05:30:57.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091603544999999995 2024-09-14T05:30:57.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635696+0000", 2024-09-14T05:30:57.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9307000000000001e-05 2024-09-14T05:30:57.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635996+0000", 2024-09-14T05:30:57.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030018800000000001 2024-09-14T05:30:57.829 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.829 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.829 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.829 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3838 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6225920~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.829 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:57.829 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.194318000999999, 2024-09-14T05:30:57.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096894235999999995, 2024-09-14T05:30:57.830 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.830 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.830 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.830 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3838 2024-09-14T05:30:57.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.830 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:57.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:30:57.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544323+0000", 2024-09-14T05:30:57.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-14T05:30:57.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544325+0000", 2024-09-14T05:30:57.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.75e-06 2024-09-14T05:30:57.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544331+0000", 2024-09-14T05:30:57.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5019999999999998e-06 2024-09-14T05:30:57.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544335+0000", 2024-09-14T05:30:57.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2989999999999998e-06 2024-09-14T05:30:57.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640869+0000", 2024-09-14T05:30:57.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096533351000000003 2024-09-14T05:30:57.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640919+0000", 2024-09-14T05:30:57.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999999e-05 2024-09-14T05:30:57.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641217+0000", 2024-09-14T05:30:57.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029717600000000002 2024-09-14T05:30:57.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.833 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.834 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3839 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.834 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:57.834 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.194299063000003, 2024-09-14T05:30:57.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097240339999999995, 2024-09-14T05:30:57.834 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.834 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.834 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.834 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3839 2024-09-14T05:30:57.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.835 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:57.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:30:57.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549834+0000", 2024-09-14T05:30:57.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054927819999999999 2024-09-14T05:30:57.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549838+0000", 2024-09-14T05:30:57.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8689999999999997e-06 2024-09-14T05:30:57.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549848+0000", 2024-09-14T05:30:57.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0154e-05 2024-09-14T05:30:57.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549853+0000", 2024-09-14T05:30:57.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9740000000000001e-06 2024-09-14T05:30:57.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641245+0000", 2024-09-14T05:30:57.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091392245999999996 2024-09-14T05:30:57.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641295+0000", 2024-09-14T05:30:57.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0018999999999997e-05 2024-09-14T05:30:57.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641582+0000", 2024-09-14T05:30:57.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028629600000000001 2024-09-14T05:30:57.838 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.838 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.838 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.838 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3868 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:57.838 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:57.838 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.188636485000004, 2024-09-14T05:30:57.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096859555999999999, 2024-09-14T05:30:57.838 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:57.838 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:57.839 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:57.839 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:57.839 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:57.839 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:57.839 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3868 2024-09-14T05:30:57.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.839 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:57.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:57.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:57.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:57.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:30:57.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:57.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:57.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556173+0000", 2024-09-14T05:30:57.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006169038 2024-09-14T05:30:57.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:57.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556177+0000", 2024-09-14T05:30:57.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0010000000000003e-06 2024-09-14T05:30:57.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:57.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556188+0000", 2024-09-14T05:30:57.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0675e-05 2024-09-14T05:30:57.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:57.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556199+0000", 2024-09-14T05:30:57.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.155e-05 2024-09-14T05:30:57.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:57.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646511+0000", 2024-09-14T05:30:57.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090312187000000002 2024-09-14T05:30:57.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:57.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646563+0000", 2024-09-14T05:30:57.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2219000000000003e-05 2024-09-14T05:30:57.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:57.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:57.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:57.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646863+0000", 2024-09-14T05:30:57.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029988600000000002 2024-09-14T05:30:57.842 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.842 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.842 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.843 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:57.843 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:57.843 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:57.844 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:30:57.845 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:57.843+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:57.945 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:57.944+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:57.988 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:57.989 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:57.989 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:57.989 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:57.997 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:30:58.046 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:58.044+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:58.101 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:58.101 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:58.101 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:58.101 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:58.101 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:58.109 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:30:58.138 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T05:30:58.182 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:58.181+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:58.247 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:58.247+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:58.261 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:58.261 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:58.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:58.261 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:58.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.261 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1989 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 16326080~54496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:58.261 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:58.261 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.698215340000004, 2024-09-14T05:30:58.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096664322999999996, 2024-09-14T05:30:58.261 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.262 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.262 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.262 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.262 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.262 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.262 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1989 2024-09-14T05:30:58.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.262 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:58.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:30:58.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561455+0000", 2024-09-14T05:30:58.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1699999999999997e-07 2024-09-14T05:30:58.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590703+0000", 2024-09-14T05:30:58.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029248225999999999 2024-09-14T05:30:58.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590732+0000", 2024-09-14T05:30:58.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8679000000000001e-05 2024-09-14T05:30:58.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590759+0000", 2024-09-14T05:30:58.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7319999999999999e-05 2024-09-14T05:30:58.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590835+0000", 2024-09-14T05:30:58.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5543999999999994e-05 2024-09-14T05:30:58.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590956+0000", 2024-09-14T05:30:58.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012168 2024-09-14T05:30:58.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 1", 2024-09-14T05:30:58.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.591261+0000", 2024-09-14T05:30:58.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030484799999999998 2024-09-14T05:30:58.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:58.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.637682+0000", 2024-09-14T05:30:58.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046420550999999997 2024-09-14T05:30:58.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:58.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.657912+0000", 2024-09-14T05:30:58.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020230011999999999 2024-09-14T05:30:58.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658007+0000", 2024-09-14T05:30:58.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5814000000000001e-05 2024-09-14T05:30:58.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658118+0000", 2024-09-14T05:30:58.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110832 2024-09-14T05:30:58.267 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.267 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.267 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.267 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2033 3.1e 3:7e47f6eb:::rbd_data.112d1f36d097.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30853661~63531] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:58.267 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:58.267 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.586940674000004, 2024-09-14T05:30:58.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065167630000000004, 2024-09-14T05:30:58.267 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.267 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.268 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.268 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.268 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.268 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.268 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2033 2024-09-14T05:30:58.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.268 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:58.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:30:58.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672730+0000", 2024-09-14T05:30:58.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.393e-06 2024-09-14T05:30:58.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691289+0000", 2024-09-14T05:30:58.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018558991 2024-09-14T05:30:58.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691316+0000", 2024-09-14T05:30:58.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7498999999999999e-05 2024-09-14T05:30:58.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691331+0000", 2024-09-14T05:30:58.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4652e-05 2024-09-14T05:30:58.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691397+0000", 2024-09-14T05:30:58.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6112000000000007e-05 2024-09-14T05:30:58.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691509+0000", 2024-09-14T05:30:58.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111599 2024-09-14T05:30:58.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:58.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.692253+0000", 2024-09-14T05:30:58.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074456200000000004 2024-09-14T05:30:58.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:58.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.719140+0000", 2024-09-14T05:30:58.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026886542999999999 2024-09-14T05:30:58.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:58.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737789+0000", 2024-09-14T05:30:58.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018649203999999999 2024-09-14T05:30:58.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737846+0000", 2024-09-14T05:30:58.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6501e-05 2024-09-14T05:30:58.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737896+0000", 2024-09-14T05:30:58.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0574000000000001e-05 2024-09-14T05:30:58.273 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.273 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.273 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.273 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3008 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13729792~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.273 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:58.273 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.956016804000001, 2024-09-14T05:30:58.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061007651000000003, 2024-09-14T05:30:58.273 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.273 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.273 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.274 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.274 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.274 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.274 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3008 2024-09-14T05:30:58.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.274 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:58.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:30:58.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303653+0000", 2024-09-14T05:30:58.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0699999999999999e-07 2024-09-14T05:30:58.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303654+0000", 2024-09-14T05:30:58.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-07 2024-09-14T05:30:58.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303657+0000", 2024-09-14T05:30:58.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2890000000000002e-06 2024-09-14T05:30:58.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303661+0000", 2024-09-14T05:30:58.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3290000000000004e-06 2024-09-14T05:30:58.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364233+0000", 2024-09-14T05:30:58.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060572134 2024-09-14T05:30:58.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364315+0000", 2024-09-14T05:30:58.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1549000000000001e-05 2024-09-14T05:30:58.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364660+0000", 2024-09-14T05:30:58.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034525300000000001 2024-09-14T05:30:58.277 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.277 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.277 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.277 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3009 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14221312~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.277 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:58.278 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.956002713999993, 2024-09-14T05:30:58.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061445063000000001, 2024-09-14T05:30:58.278 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.278 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.278 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.278 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.278 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.278 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.278 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3009 2024-09-14T05:30:58.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.278 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:58.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:58.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:30:58.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:30:58.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303669+0000", 2024-09-14T05:30:58.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7150000000000001e-06 2024-09-14T05:30:58.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303675+0000", 2024-09-14T05:30:58.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5339999999999996e-06 2024-09-14T05:30:58.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303681+0000", 2024-09-14T05:30:58.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3639999999999998e-06 2024-09-14T05:30:58.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364705+0000", 2024-09-14T05:30:58.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061024123 2024-09-14T05:30:58.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364786+0000", 2024-09-14T05:30:58.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1378999999999995e-05 2024-09-14T05:30:58.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365112+0000", 2024-09-14T05:30:58.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032558700000000001 2024-09-14T05:30:58.282 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.282 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.282 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.283 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3010 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14712832~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.283 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:58.283 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.955981850000001, 2024-09-14T05:30:58.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061834630000000002, 2024-09-14T05:30:58.283 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.283 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.283 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.283 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.283 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.283 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.284 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3010 2024-09-14T05:30:58.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.284 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:58.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:30:58.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303688+0000", 2024-09-14T05:30:58.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:30:58.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303690+0000", 2024-09-14T05:30:58.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9460000000000001e-06 2024-09-14T05:30:58.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303696+0000", 2024-09-14T05:30:58.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2230000000000004e-06 2024-09-14T05:30:58.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303700+0000", 2024-09-14T05:30:58.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7290000000000002e-06 2024-09-14T05:30:58.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365163+0000", 2024-09-14T05:30:58.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061463093000000003 2024-09-14T05:30:58.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365214+0000", 2024-09-14T05:30:58.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0977999999999999e-05 2024-09-14T05:30:58.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365522+0000", 2024-09-14T05:30:58.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000308227 2024-09-14T05:30:58.287 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.287 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.287 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.287 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3241 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.287 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:58.288 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.930307428000006, 2024-09-14T05:30:58.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061041892, 2024-09-14T05:30:58.288 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.288 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.288 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.288 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.288 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.288 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.288 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3241 2024-09-14T05:30:58.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.288 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:58.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:30:58.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329496+0000", 2024-09-14T05:30:58.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013421799999999999 2024-09-14T05:30:58.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329499+0000", 2024-09-14T05:30:58.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8459999999999999e-06 2024-09-14T05:30:58.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329505+0000", 2024-09-14T05:30:58.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4320000000000004e-06 2024-09-14T05:30:58.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329509+0000", 2024-09-14T05:30:58.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0339999999999998e-06 2024-09-14T05:30:58.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.389947+0000", 2024-09-14T05:30:58.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060438093999999998 2024-09-14T05:30:58.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390038+0000", 2024-09-14T05:30:58.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0253999999999993e-05 2024-09-14T05:30:58.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390404+0000", 2024-09-14T05:30:58.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036601400000000001 2024-09-14T05:30:58.291 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.291 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.291 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.292 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3242 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.292 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:58.292 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.930152520000007, 2024-09-14T05:30:58.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061282426000000001, 2024-09-14T05:30:58.292 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.292 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.292 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.292 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3242 2024-09-14T05:30:58.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.292 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:58.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:58.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:30:58.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9900000000000001e-07 2024-09-14T05:30:58.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329518+0000", 2024-09-14T05:30:58.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.328e-06 2024-09-14T05:30:58.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329522+0000", 2024-09-14T05:30:58.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.563e-06 2024-09-14T05:30:58.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329525+0000", 2024-09-14T05:30:58.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7379999999999999e-06 2024-09-14T05:30:58.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390437+0000", 2024-09-14T05:30:58.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060912463 2024-09-14T05:30:58.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390498+0000", 2024-09-14T05:30:58.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0615999999999997e-05 2024-09-14T05:30:58.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390799+0000", 2024-09-14T05:30:58.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030131899999999998 2024-09-14T05:30:58.296 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.296 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.296 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.296 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3243 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.296 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:58.296 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.930140734000005, 2024-09-14T05:30:58.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061728980000000003, 2024-09-14T05:30:58.296 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.296 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.297 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.297 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.297 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.297 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.297 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3243 2024-09-14T05:30:58.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.297 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:58.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:30:58.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329692+0000", 2024-09-14T05:30:58.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000163948 2024-09-14T05:30:58.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329696+0000", 2024-09-14T05:30:58.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5540000000000002e-06 2024-09-14T05:30:58.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329706+0000", 2024-09-14T05:30:58.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7960000000000007e-06 2024-09-14T05:30:58.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329712+0000", 2024-09-14T05:30:58.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6109999999999999e-06 2024-09-14T05:30:58.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390861+0000", 2024-09-14T05:30:58.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061148767 2024-09-14T05:30:58.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390942+0000", 2024-09-14T05:30:58.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1266000000000004e-05 2024-09-14T05:30:58.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391258+0000", 2024-09-14T05:30:58.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031503800000000002 2024-09-14T05:30:58.300 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.301 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.301 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.301 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3244 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14778368~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.301 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:58.301 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.929945685999996, 2024-09-14T05:30:58.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062050594000000001, 2024-09-14T05:30:58.301 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.301 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.301 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.301 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.301 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.301 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.302 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3244 2024-09-14T05:30:58.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.302 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:58.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:30:58.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329916+0000", 2024-09-14T05:30:58.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019290799999999999 2024-09-14T05:30:58.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329919+0000", 2024-09-14T05:30:58.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9019999999999999e-06 2024-09-14T05:30:58.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329926+0000", 2024-09-14T05:30:58.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9e-06 2024-09-14T05:30:58.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329932+0000", 2024-09-14T05:30:58.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.088e-06 2024-09-14T05:30:58.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391299+0000", 2024-09-14T05:30:58.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061366136000000002 2024-09-14T05:30:58.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391391+0000", 2024-09-14T05:30:58.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2204e-05 2024-09-14T05:30:58.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391774+0000", 2024-09-14T05:30:58.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038345600000000001 2024-09-14T05:30:58.305 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.305 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.305 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.305 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3921 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 5308416~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.305 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:58.305 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.858401876000002, 2024-09-14T05:30:58.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060842694000000003, 2024-09-14T05:30:58.306 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.306 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.306 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.306 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.306 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.306 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.306 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3921 2024-09-14T05:30:58.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.306 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:58.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:30:58.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401705+0000", 2024-09-14T05:30:58.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768499999999999 2024-09-14T05:30:58.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401709+0000", 2024-09-14T05:30:58.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0319999999999997e-06 2024-09-14T05:30:58.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401720+0000", 2024-09-14T05:30:58.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1005000000000001e-05 2024-09-14T05:30:58.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401727+0000", 2024-09-14T05:30:58.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1879999999999996e-06 2024-09-14T05:30:58.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461748+0000", 2024-09-14T05:30:58.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060021199999999997 2024-09-14T05:30:58.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461793+0000", 2024-09-14T05:30:58.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4530999999999998e-05 2024-09-14T05:30:58.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462110+0000", 2024-09-14T05:30:58.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031705299999999997 2024-09-14T05:30:58.309 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.309 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.309 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3923 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 6291456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.857755847999996, 2024-09-14T05:30:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060891273000000003, 2024-09-14T05:30:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3923 2024-09-14T05:30:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:30:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407054+0000", 2024-09-14T05:30:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005140787 2024-09-14T05:30:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407059+0000", 2024-09-14T05:30:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.04e-06 2024-09-14T05:30:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407069+0000", 2024-09-14T05:30:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4429999999999992e-06 2024-09-14T05:30:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407076+0000", 2024-09-14T05:30:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1679999999999998e-06 2024-09-14T05:30:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462514+0000", 2024-09-14T05:30:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055438551000000003 2024-09-14T05:30:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462562+0000", 2024-09-14T05:30:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7649999999999999e-05 2024-09-14T05:30:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462805+0000", 2024-09-14T05:30:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024263400000000001 2024-09-14T05:30:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4052 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 12550144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.842183839, 2024-09-14T05:30:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060872904999999998, 2024-09-14T05:30:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4052 2024-09-14T05:30:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:30:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417486+0000", 2024-09-14T05:30:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-14T05:30:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417487+0000", 2024-09-14T05:30:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4499999999999995e-07 2024-09-14T05:30:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417489+0000", 2024-09-14T05:30:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8899999999999999e-06 2024-09-14T05:30:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417492+0000", 2024-09-14T05:30:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4389999999999999e-06 2024-09-14T05:30:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478027+0000", 2024-09-14T05:30:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060534959999999999 2024-09-14T05:30:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478079+0000", 2024-09-14T05:30:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2104999999999997e-05 2024-09-14T05:30:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478358+0000", 2024-09-14T05:30:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000279351 2024-09-14T05:30:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4053 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13041664~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.842174455000006, 2024-09-14T05:30:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061258866000000002, 2024-09-14T05:30:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4053 2024-09-14T05:30:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:30:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.17e-07 2024-09-14T05:30:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417496+0000", 2024-09-14T05:30:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0890000000000001e-06 2024-09-14T05:30:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417499+0000", 2024-09-14T05:30:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8629999999999999e-06 2024-09-14T05:30:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417502+0000", 2024-09-14T05:30:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6290000000000002e-06 2024-09-14T05:30:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478387+0000", 2024-09-14T05:30:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060885433000000003 2024-09-14T05:30:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478435+0000", 2024-09-14T05:30:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8167999999999997e-05 2024-09-14T05:30:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478754+0000", 2024-09-14T05:30:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031846700000000001 2024-09-14T05:30:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4054 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13533184~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.842164570999998, 2024-09-14T05:30:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061604694000000002, 2024-09-14T05:30:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4054 2024-09-14T05:30:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:30:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.04e-07 2024-09-14T05:30:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417506+0000", 2024-09-14T05:30:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8299999999999995e-07 2024-09-14T05:30:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417509+0000", 2024-09-14T05:30:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.88e-06 2024-09-14T05:30:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417511+0000", 2024-09-14T05:30:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5950000000000001e-06 2024-09-14T05:30:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478784+0000", 2024-09-14T05:30:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061272971000000002 2024-09-14T05:30:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478841+0000", 2024-09-14T05:30:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-14T05:30:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479109+0000", 2024-09-14T05:30:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026794600000000003 2024-09-14T05:30:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4055 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14024704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.842154868999998, 2024-09-14T05:30:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061932299000000003, 2024-09-14T05:30:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4055 2024-09-14T05:30:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:30:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417515+0000", 2024-09-14T05:30:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-14T05:30:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417516+0000", 2024-09-14T05:30:58.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7199999999999997e-07 2024-09-14T05:30:58.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417519+0000", 2024-09-14T05:30:58.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.937e-06 2024-09-14T05:30:58.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417522+0000", 2024-09-14T05:30:58.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1420000000000002e-06 2024-09-14T05:30:58.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479140+0000", 2024-09-14T05:30:58.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061617933999999999 2024-09-14T05:30:58.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479188+0000", 2024-09-14T05:30:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8717000000000001e-05 2024-09-14T05:30:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479447+0000", 2024-09-14T05:30:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025838800000000002 2024-09-14T05:30:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4056 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14516224~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.842144696000005, 2024-09-14T05:30:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062263185999999998, 2024-09-14T05:30:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4056 2024-09-14T05:30:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:30:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8700000000000002e-07 2024-09-14T05:30:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417528+0000", 2024-09-14T05:30:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.579e-06 2024-09-14T05:30:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417533+0000", 2024-09-14T05:30:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2400000000000001e-06 2024-09-14T05:30:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417536+0000", 2024-09-14T05:30:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0309999999999999e-06 2024-09-14T05:30:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479477+0000", 2024-09-14T05:30:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061940967 2024-09-14T05:30:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479523+0000", 2024-09-14T05:30:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6335000000000003e-05 2024-09-14T05:30:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479788+0000", 2024-09-14T05:30:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026474700000000002 2024-09-14T05:30:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4057 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15007744~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.842130234999999, 2024-09-14T05:30:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062589928000000003, 2024-09-14T05:30:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.337 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.337 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.337 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.337 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.337 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4057 2024-09-14T05:30:58.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.337 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:30:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422838+0000", 2024-09-14T05:30:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052990349999999997 2024-09-14T05:30:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422854+0000", 2024-09-14T05:30:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5523999999999999e-05 2024-09-14T05:30:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422864+0000", 2024-09-14T05:30:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0084e-05 2024-09-14T05:30:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422872+0000", 2024-09-14T05:30:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-14T05:30:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479813+0000", 2024-09-14T05:30:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056941389000000002 2024-09-14T05:30:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479868+0000", 2024-09-14T05:30:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4973999999999999e-05 2024-09-14T05:30:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.480129+0000", 2024-09-14T05:30:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026084200000000001 2024-09-14T05:30:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4171 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14057472~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.830132675000002, 2024-09-14T05:30:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061184929999999998, 2024-09-14T05:30:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4171 2024-09-14T05:30:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:30:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.39e-07 2024-09-14T05:30:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429541+0000", 2024-09-14T05:30:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:30:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429547+0000", 2024-09-14T05:30:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3280000000000002e-06 2024-09-14T05:30:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429552+0000", 2024-09-14T05:30:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4859999999999997e-06 2024-09-14T05:30:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490379+0000", 2024-09-14T05:30:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060826681 2024-09-14T05:30:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490434+0000", 2024-09-14T05:30:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4577999999999998e-05 2024-09-14T05:30:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490722+0000", 2024-09-14T05:30:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028781400000000001 2024-09-14T05:30:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4172 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14548992~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.830110667, 2024-09-14T05:30:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061498256000000001, 2024-09-14T05:30:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4172 2024-09-14T05:30:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:30:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.72e-07 2024-09-14T05:30:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429561+0000", 2024-09-14T05:30:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.717e-06 2024-09-14T05:30:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429566+0000", 2024-09-14T05:30:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6529999999999999e-06 2024-09-14T05:30:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429571+0000", 2024-09-14T05:30:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5979999999999999e-06 2024-09-14T05:30:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490751+0000", 2024-09-14T05:30:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061180336000000002 2024-09-14T05:30:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490798+0000", 2024-09-14T05:30:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6875000000000001e-05 2024-09-14T05:30:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491057+0000", 2024-09-14T05:30:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025870500000000002 2024-09-14T05:30:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4173 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15040512~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.830093040999998, 2024-09-14T05:30:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061894820000000003, 2024-09-14T05:30:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4173 2024-09-14T05:30:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:30:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429577+0000", 2024-09-14T05:30:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-14T05:30:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429578+0000", 2024-09-14T05:30:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6980000000000001e-06 2024-09-14T05:30:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429585+0000", 2024-09-14T05:30:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.725e-06 2024-09-14T05:30:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429590+0000", 2024-09-14T05:30:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.938e-06 2024-09-14T05:30:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491091+0000", 2024-09-14T05:30:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061500773000000002 2024-09-14T05:30:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491136+0000", 2024-09-14T05:30:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5284000000000002e-05 2024-09-14T05:30:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491471+0000", 2024-09-14T05:30:58.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033485699999999998 2024-09-14T05:30:58.354 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.354 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.354 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.354 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.354 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.354 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:58.356 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:30:58.356 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:58.354+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:58.457 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:58.456+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:58.500 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:58.500 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:58.501 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:58.501 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:58.509 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:30:58.558 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:58.556+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:58.611 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:58.612 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:58.612 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:58.612 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:58.612 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:58.621 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:30:58.662 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:58.660+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:58.761 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:58.760+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:30:58.764 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:58.764 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:58.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:58.765 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:58.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.765 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1205 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:58.765 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:58.765 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.565254510000003, 2024-09-14T05:30:58.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0086364279999999998, 2024-09-14T05:30:58.765 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.765 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.765 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.765 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1205 2024-09-14T05:30:58.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.766 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:58.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:30:58.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197891+0000", 2024-09-14T05:30:58.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.364e-06 2024-09-14T05:30:58.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197896+0000", 2024-09-14T05:30:58.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3000000000000001e-06 2024-09-14T05:30:58.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197910+0000", 2024-09-14T05:30:58.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3339000000000001e-05 2024-09-14T05:30:58.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197919+0000", 2024-09-14T05:30:58.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8869999999999998e-06 2024-09-14T05:30:58.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197974+0000", 2024-09-14T05:30:58.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5720000000000002e-05 2024-09-14T05:30:58.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.198094+0000", 2024-09-14T05:30:58.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119739 2024-09-14T05:30:58.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.769 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206526+0000", 2024-09-14T05:30:58.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084320790000000003 2024-09-14T05:30:58.769 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.769 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.769 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.770 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1220 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:58.770 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:58.770 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.556498145999996, 2024-09-14T05:30:58.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012822337, 2024-09-14T05:30:58.770 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.770 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.770 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.770 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1220 2024-09-14T05:30:58.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.771 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:58.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:30:58.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206647+0000", 2024-09-14T05:30:58.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4600000000000003e-07 2024-09-14T05:30:58.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206652+0000", 2024-09-14T05:30:58.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6619999999999996e-06 2024-09-14T05:30:58.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206662+0000", 2024-09-14T05:30:58.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.995e-06 2024-09-14T05:30:58.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206667+0000", 2024-09-14T05:30:58.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-14T05:30:58.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206706+0000", 2024-09-14T05:30:58.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9523000000000001e-05 2024-09-14T05:30:58.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206755+0000", 2024-09-14T05:30:58.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000002e-05 2024-09-14T05:30:58.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:58.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206877+0000", 2024-09-14T05:30:58.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121938 2024-09-14T05:30:58.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:58.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.207310+0000", 2024-09-14T05:30:58.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000432452 2024-09-14T05:30:58.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:58.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219364+0000", 2024-09-14T05:30:58.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012054481000000001 2024-09-14T05:30:58.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219425+0000", 2024-09-14T05:30:58.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0618000000000001e-05 2024-09-14T05:30:58.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219468+0000", 2024-09-14T05:30:58.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3739000000000003e-05 2024-09-14T05:30:58.776 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.776 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.776 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.776 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1243 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:58.776 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:58.776 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.543532919, 2024-09-14T05:30:58.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078660580000000004, 2024-09-14T05:30:58.776 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.776 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.776 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:58.777 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.777 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.777 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.777 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1243 2024-09-14T05:30:58.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.777 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:58.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:30:58.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219612+0000", 2024-09-14T05:30:58.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.09e-07 2024-09-14T05:30:58.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219616+0000", 2024-09-14T05:30:58.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.216e-06 2024-09-14T05:30:58.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219630+0000", 2024-09-14T05:30:58.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3064999999999999e-05 2024-09-14T05:30:58.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219637+0000", 2024-09-14T05:30:58.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7409999999999992e-06 2024-09-14T05:30:58.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219697+0000", 2024-09-14T05:30:58.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.02e-05 2024-09-14T05:30:58.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219833+0000", 2024-09-14T05:30:58.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013595600000000001 2024-09-14T05:30:58.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227477+0000", 2024-09-14T05:30:58.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076439710000000003 2024-09-14T05:30:58.780 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.780 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.780 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.781 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1264 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:58.781 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:58.781 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.535522126000004, 2024-09-14T05:30:58.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080867990000000004, 2024-09-14T05:30:58.781 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.781 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.781 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.781 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.781 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.781 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.781 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1264 2024-09-14T05:30:58.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.782 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:58.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:30:58.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227623+0000", 2024-09-14T05:30:58.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:30:58.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227627+0000", 2024-09-14T05:30:58.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.194e-06 2024-09-14T05:30:58.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227639+0000", 2024-09-14T05:30:58.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1296e-05 2024-09-14T05:30:58.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227648+0000", 2024-09-14T05:30:58.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2280000000000004e-06 2024-09-14T05:30:58.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227689+0000", 2024-09-14T05:30:58.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1329999999999999e-05 2024-09-14T05:30:58.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227741+0000", 2024-09-14T05:30:58.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1641000000000002e-05 2024-09-14T05:30:58.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:30:58.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227914+0000", 2024-09-14T05:30:58.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000173215 2024-09-14T05:30:58.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:58.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228474+0000", 2024-09-14T05:30:58.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055980199999999998 2024-09-14T05:30:58.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:58.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235608+0000", 2024-09-14T05:30:58.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0071343830000000002 2024-09-14T05:30:58.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235669+0000", 2024-09-14T05:30:58.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1068999999999994e-05 2024-09-14T05:30:58.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235709+0000", 2024-09-14T05:30:58.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9671e-05 2024-09-14T05:30:58.787 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.787 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.788 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.788 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1526 3.4 3:2695cbe4:::rbd_data.112dd25fb4c.0000000000000119:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:58.788 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:58.788 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.436981563000003, 2024-09-14T05:30:58.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0079537359999999994, 2024-09-14T05:30:58.788 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.788 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.789 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.789 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.789 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.789 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.789 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1526 2024-09-14T05:30:58.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.789 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:58.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:30:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3600000000000003e-07 2024-09-14T05:30:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326168+0000", 2024-09-14T05:30:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3460000000000004e-06 2024-09-14T05:30:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326178+0000", 2024-09-14T05:30:58.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0291e-05 2024-09-14T05:30:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326183+0000", 2024-09-14T05:30:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-14T05:30:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326230+0000", 2024-09-14T05:30:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6650999999999998e-05 2024-09-14T05:30:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326295+0000", 2024-09-14T05:30:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5158000000000002e-05 2024-09-14T05:30:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326408+0000", 2024-09-14T05:30:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113039 2024-09-14T05:30:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.331648+0000", 2024-09-14T05:30:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052404929999999997 2024-09-14T05:30:58.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334011+0000", 2024-09-14T05:30:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002362669 2024-09-14T05:30:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334064+0000", 2024-09-14T05:30:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3198999999999997e-05 2024-09-14T05:30:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334116+0000", 2024-09-14T05:30:58.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2133999999999999e-05 2024-09-14T05:30:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1531 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.434163397000006, 2024-09-14T05:30:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082114309999999999, 2024-09-14T05:30:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.795 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1531 2024-09-14T05:30:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:30:58.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328982+0000", 2024-09-14T05:30:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:30:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328985+0000", 2024-09-14T05:30:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7900000000000001e-06 2024-09-14T05:30:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328996+0000", 2024-09-14T05:30:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.028e-05 2024-09-14T05:30:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329001+0000", 2024-09-14T05:30:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0610000000000003e-06 2024-09-14T05:30:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329035+0000", 2024-09-14T05:30:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4366999999999998e-05 2024-09-14T05:30:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329082+0000", 2024-09-14T05:30:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6646e-05 2024-09-14T05:30:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329195+0000", 2024-09-14T05:30:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113279 2024-09-14T05:30:58.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334293+0000", 2024-09-14T05:30:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050980399999999999 2024-09-14T05:30:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337101+0000", 2024-09-14T05:30:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0028076170000000001 2024-09-14T05:30:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337154+0000", 2024-09-14T05:30:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3641999999999999e-05 2024-09-14T05:30:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337192+0000", 2024-09-14T05:30:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7932000000000002e-05 2024-09-14T05:30:58.801 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:2131 3.b e30/24 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head v 30'36, mlcod=30'34)", 2024-09-14T05:30:58.804 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.970816041999996, 2024-09-14T05:30:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031871709999999998, 2024-09-14T05:30:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:30:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792331+0000", 2024-09-14T05:30:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6900000000000001e-06 2024-09-14T05:30:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797319+0000", 2024-09-14T05:30:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049876349999999998 2024-09-14T05:30:58.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797349+0000", 2024-09-14T05:30:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0046e-05 2024-09-14T05:30:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797359+0000", 2024-09-14T05:30:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0037999999999999e-05 2024-09-14T05:30:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797445+0000", 2024-09-14T05:30:58.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6132999999999997e-05 2024-09-14T05:30:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797472+0000", 2024-09-14T05:30:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7478999999999999e-05 2024-09-14T05:30:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824161+0000", 2024-09-14T05:30:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026688714999999998 2024-09-14T05:30:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824200+0000", 2024-09-14T05:30:58.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8974000000000003e-05 2024-09-14T05:30:58.809 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.809 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.809 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.809 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4259 3.4 3:25f21dc9:::rbd_data.112d3d611e54.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 294912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.809 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:58.809 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.371643427999999, 2024-09-14T05:30:58.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010043553, 2024-09-14T05:30:58.809 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.809 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.809 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.810 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.810 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.810 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.810 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4259 2024-09-14T05:30:58.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.810 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:58.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:30:58.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391503+0000", 2024-09-14T05:30:58.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9099999999999999e-06 2024-09-14T05:30:58.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391821+0000", 2024-09-14T05:30:58.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031871800000000001 2024-09-14T05:30:58.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391844+0000", 2024-09-14T05:30:58.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2045e-05 2024-09-14T05:30:58.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391853+0000", 2024-09-14T05:30:58.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7249999999999992e-06 2024-09-14T05:30:58.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391917+0000", 2024-09-14T05:30:58.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3458000000000001e-05 2024-09-14T05:30:58.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392037+0000", 2024-09-14T05:30:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120449 2024-09-14T05:30:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392305+0000", 2024-09-14T05:30:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026739299999999999 2024-09-14T05:30:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.393609+0000", 2024-09-14T05:30:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013042450000000001 2024-09-14T05:30:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401430+0000", 2024-09-14T05:30:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078211670000000004 2024-09-14T05:30:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401499+0000", 2024-09-14T05:30:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9374000000000004e-05 2024-09-14T05:30:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401544+0000", 2024-09-14T05:30:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5068999999999998e-05 2024-09-14T05:30:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4554 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.778742617000006, 2024-09-14T05:30:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069998999999999999, 2024-09-14T05:30:58.815 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4554 2024-09-14T05:30:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:30:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9299999999999999e-07 2024-09-14T05:30:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984404+0000", 2024-09-14T05:30:58.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.855e-06 2024-09-14T05:30:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984407+0000", 2024-09-14T05:30:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-14T05:30:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984410+0000", 2024-09-14T05:30:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8159999999999998e-06 2024-09-14T05:30:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985511+0000", 2024-09-14T05:30:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00110053 2024-09-14T05:30:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985561+0000", 2024-09-14T05:30:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0389000000000002e-05 2024-09-14T05:30:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:30:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985772+0000", 2024-09-14T05:30:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000210571 2024-09-14T05:30:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.987075+0000", 2024-09-14T05:30:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001303128 2024-09-14T05:30:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991303+0000", 2024-09-14T05:30:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042283310000000001 2024-09-14T05:30:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991359+0000", 2024-09-14T05:30:58.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6335999999999997e-05 2024-09-14T05:30:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991402+0000", 2024-09-14T05:30:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2074999999999999e-05 2024-09-14T05:30:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4711 3.4 3:226df663:::rbd_data.112d3d611e54.000000000000002d:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:30:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.596071460999994, 2024-09-14T05:30:58.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011304193000000001, 2024-09-14T05:30:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4711 2024-09-14T05:30:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:30:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167074+0000", 2024-09-14T05:30:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0999999999999997e-07 2024-09-14T05:30:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167078+0000", 2024-09-14T05:30:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.878e-06 2024-09-14T05:30:58.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167087+0000", 2024-09-14T05:30:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6940000000000007e-06 2024-09-14T05:30:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167093+0000", 2024-09-14T05:30:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7740000000000001e-06 2024-09-14T05:30:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167128+0000", 2024-09-14T05:30:58.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5231999999999997e-05 2024-09-14T05:30:58.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167207+0000", 2024-09-14T05:30:58.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9036999999999996e-05 2024-09-14T05:30:58.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:30:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.169140+0000", 2024-09-14T05:30:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001932897 2024-09-14T05:30:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178258+0000", 2024-09-14T05:30:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091180819999999996 2024-09-14T05:30:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178314+0000", 2024-09-14T05:30:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5571e-05 2024-09-14T05:30:58.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178348+0000", 2024-09-14T05:30:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3747000000000001e-05 2024-09-14T05:30:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178377+0000", 2024-09-14T05:30:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9471e-05 2024-09-14T05:30:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:72582 3.b e47/24 3:d5072567:::rbd_data.112d4a90444c.0000000000000119:head v 47'386, mlcod=46'383)", 2024-09-14T05:30:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.238117607999996, 2024-09-14T05:30:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0067720940000000002, 2024-09-14T05:30:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:58.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:58.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:30:58.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1100000000000002e-07 2024-09-14T05:30:58.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525031+0000", 2024-09-14T05:30:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0740000000000003e-06 2024-09-14T05:30:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525040+0000", 2024-09-14T05:30:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5329999999999994e-06 2024-09-14T05:30:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525045+0000", 2024-09-14T05:30:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:30:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525090+0000", 2024-09-14T05:30:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4743000000000002e-05 2024-09-14T05:30:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525102+0000", 2024-09-14T05:30:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2067e-05 2024-09-14T05:30:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531757+0000", 2024-09-14T05:30:58.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066550380000000003 2024-09-14T05:30:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531799+0000", 2024-09-14T05:30:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1640000000000001e-05 2024-09-14T05:30:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.7.0:509 3.b e100/24 3:d1b3949e:::rbd_data.112d46b1ca94.000000000000010c:11 v 100'523, mlcod=99'518)", 2024-09-14T05:30:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.009409+0000", 2024-09-14T05:30:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.7537354890000003, 2024-09-14T05:30:58.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075659999999999998, 2024-09-14T05:30:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009409+0000", 2024-09-14T05:30:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009409+0000", 2024-09-14T05:30:58.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009409+0000", 2024-09-14T05:30:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7599999999999997e-07 2024-09-14T05:30:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009421+0000", 2024-09-14T05:30:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1705999999999999e-05 2024-09-14T05:30:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009432+0000", 2024-09-14T05:30:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0769e-05 2024-09-14T05:30:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009439+0000", 2024-09-14T05:30:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5969999999999997e-06 2024-09-14T05:30:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009505+0000", 2024-09-14T05:30:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6310999999999996e-05 2024-09-14T05:30:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009521+0000", 2024-09-14T05:30:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.571e-05 2024-09-14T05:30:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.016931+0000", 2024-09-14T05:30:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0074101619999999997 2024-09-14T05:30:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.016975+0000", 2024-09-14T05:30:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4069000000000001e-05 2024-09-14T05:30:58.836 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.837 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.837 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.837 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.2.0:1434 3.5 e100/24 3:a529dc6d:::rbd_data.112d46b1ca94.00000000000000d4:11 v 100'327, mlcod=100'321)", 2024-09-14T05:30:58.837 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.013998+0000", 2024-09-14T05:30:58.837 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.7491460720000003, 2024-09-14T05:30:58.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0083544610000000005, 2024-09-14T05:30:58.837 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.837 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.837 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.837 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.013998+0000", 2024-09-14T05:30:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.013998+0000", 2024-09-14T05:30:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.013999+0000", 2024-09-14T05:30:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-14T05:30:58.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014003+0000", 2024-09-14T05:30:58.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.51e-06 2024-09-14T05:30:58.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014013+0000", 2024-09-14T05:30:58.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3379999999999994e-06 2024-09-14T05:30:58.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014018+0000", 2024-09-14T05:30:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5269999999999998e-06 2024-09-14T05:30:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014843+0000", 2024-09-14T05:30:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00082516000000000002 2024-09-14T05:30:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014871+0000", 2024-09-14T05:30:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7628e-05 2024-09-14T05:30:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.022313+0000", 2024-09-14T05:30:58.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.007441563 2024-09-14T05:30:58.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.022353+0000", 2024-09-14T05:30:58.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0055000000000002e-05 2024-09-14T05:30:58.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.841 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.0.0:1713 3.15 e100/24 3:a916a3d3:::rbd_data.112d46b1ca94.00000000000000d3:11 v 100'427, mlcod=99'422)", 2024-09-14T05:30:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.014451+0000", 2024-09-14T05:30:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.7486927879999996, 2024-09-14T05:30:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078576710000000001, 2024-09-14T05:30:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014451+0000", 2024-09-14T05:30:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014451+0000", 2024-09-14T05:30:58.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014452+0000", 2024-09-14T05:30:58.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9599999999999998e-07 2024-09-14T05:30:58.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014458+0000", 2024-09-14T05:30:58.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8699999999999997e-06 2024-09-14T05:30:58.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014469+0000", 2024-09-14T05:30:58.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0627e-05 2024-09-14T05:30:58.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014476+0000", 2024-09-14T05:30:58.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.046e-06 2024-09-14T05:30:58.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014995+0000", 2024-09-14T05:30:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051870599999999996 2024-09-14T05:30:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.015014+0000", 2024-09-14T05:30:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9768000000000001e-05 2024-09-14T05:30:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.022266+0000", 2024-09-14T05:30:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0072513430000000004 2024-09-14T05:30:58.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.022309+0000", 2024-09-14T05:30:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3514999999999999e-05 2024-09-14T05:30:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.7.0:510 3.b e100/24 3:d5c4519c:::rbd_data.112d46b1ca94.0000000000000024:11 v 100'525, mlcod=100'521)", 2024-09-14T05:30:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.017655+0000", 2024-09-14T05:30:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.7454893279999997, 2024-09-14T05:30:58.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0073483380000000003, 2024-09-14T05:30:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017655+0000", 2024-09-14T05:30:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017655+0000", 2024-09-14T05:30:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017656+0000", 2024-09-14T05:30:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8800000000000002e-07 2024-09-14T05:30:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017660+0000", 2024-09-14T05:30:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2370000000000003e-06 2024-09-14T05:30:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017669+0000", 2024-09-14T05:30:58.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7749999999999998e-06 2024-09-14T05:30:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017675+0000", 2024-09-14T05:30:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1990000000000003e-06 2024-09-14T05:30:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017727+0000", 2024-09-14T05:30:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1835000000000001e-05 2024-09-14T05:30:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017745+0000", 2024-09-14T05:30:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8233999999999999e-05 2024-09-14T05:30:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.024955+0000", 2024-09-14T05:30:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0072104279999999996 2024-09-14T05:30:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.025003+0000", 2024-09-14T05:30:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7942000000000003e-05 2024-09-14T05:30:58.850 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.5.0:2812 3.4 e100/39 3:2237b773:::rbd_data.112d46b1ca94.00000000000001f5:11 v 100'550, mlcod=100'546)", 2024-09-14T05:30:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.018029+0000", 2024-09-14T05:30:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.7451153030000004, 2024-09-14T05:30:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0071886470000000003, 2024-09-14T05:30:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018029+0000", 2024-09-14T05:30:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018029+0000", 2024-09-14T05:30:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018030+0000", 2024-09-14T05:30:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3900000000000004e-07 2024-09-14T05:30:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018036+0000", 2024-09-14T05:30:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9669999999999998e-06 2024-09-14T05:30:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018049+0000", 2024-09-14T05:30:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2897e-05 2024-09-14T05:30:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018055+0000", 2024-09-14T05:30:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8109999999999997e-06 2024-09-14T05:30:58.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018104+0000", 2024-09-14T05:30:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8408999999999997e-05 2024-09-14T05:30:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018125+0000", 2024-09-14T05:30:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0992000000000002e-05 2024-09-14T05:30:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.025174+0000", 2024-09-14T05:30:58.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0070491850000000003 2024-09-14T05:30:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.025218+0000", 2024-09-14T05:30:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3646999999999997e-05 2024-09-14T05:30:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.7.0:511 3.b e100/24 3:d76f30d4:::rbd_data.112d46b1ca94.0000000000000048:11 v 100'527, mlcod=100'521)", 2024-09-14T05:30:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.018634+0000", 2024-09-14T05:30:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.7445105269999992, 2024-09-14T05:30:58.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0090907680000000008, 2024-09-14T05:30:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018634+0000", 2024-09-14T05:30:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018634+0000", 2024-09-14T05:30:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018634+0000", 2024-09-14T05:30:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5899999999999996e-07 2024-09-14T05:30:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018639+0000", 2024-09-14T05:30:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5229999999999997e-06 2024-09-14T05:30:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018649+0000", 2024-09-14T05:30:58.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5999999999999996e-06 2024-09-14T05:30:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018656+0000", 2024-09-14T05:30:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5789999999999998e-06 2024-09-14T05:30:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.020500+0000", 2024-09-14T05:30:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0018434840000000001 2024-09-14T05:30:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.020522+0000", 2024-09-14T05:30:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2892999999999999e-05 2024-09-14T05:30:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.027685+0000", 2024-09-14T05:30:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.007162758 2024-09-14T05:30:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.027725+0000", 2024-09-14T05:30:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9271999999999999e-05 2024-09-14T05:30:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.859 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.2.0:1440 3.5 e100/24 3:a70fccf8:::rbd_data.112d46b1ca94.00000000000000ec:11 v 100'329, mlcod=100'325)", 2024-09-14T05:30:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.023047+0000", 2024-09-14T05:30:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.7400969899999996, 2024-09-14T05:30:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.007561229, 2024-09-14T05:30:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023047+0000", 2024-09-14T05:30:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023047+0000", 2024-09-14T05:30:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023048+0000", 2024-09-14T05:30:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0999999999999997e-07 2024-09-14T05:30:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023054+0000", 2024-09-14T05:30:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9510000000000003e-06 2024-09-14T05:30:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023067+0000", 2024-09-14T05:30:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2959000000000001e-05 2024-09-14T05:30:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023074+0000", 2024-09-14T05:30:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0450000000000003e-06 2024-09-14T05:30:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023155+0000", 2024-09-14T05:30:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1422999999999996e-05 2024-09-14T05:30:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023179+0000", 2024-09-14T05:30:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.376e-05 2024-09-14T05:30:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.030542+0000", 2024-09-14T05:30:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0073630859999999996 2024-09-14T05:30:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.030609+0000", 2024-09-14T05:30:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6194999999999999e-05 2024-09-14T05:30:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.0.0:1718 3.15 e100/24 3:add139e1:::rbd_data.112d46b1ca94.0000000000000007:11 v 100'429, mlcod=100'425)", 2024-09-14T05:30:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.023071+0000", 2024-09-14T05:30:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.7400731779999994, 2024-09-14T05:30:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076044830000000004, 2024-09-14T05:30:58.864 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023071+0000", 2024-09-14T05:30:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023071+0000", 2024-09-14T05:30:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023072+0000", 2024-09-14T05:30:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2099999999999996e-07 2024-09-14T05:30:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023077+0000", 2024-09-14T05:30:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3759999999999999e-06 2024-09-14T05:30:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023085+0000", 2024-09-14T05:30:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2009999999999999e-06 2024-09-14T05:30:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023091+0000", 2024-09-14T05:30:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3720000000000004e-06 2024-09-14T05:30:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023241+0000", 2024-09-14T05:30:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015081799999999999 2024-09-14T05:30:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023261+0000", 2024-09-14T05:30:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9076000000000002e-05 2024-09-14T05:30:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.030614+0000", 2024-09-14T05:30:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00735347 2024-09-14T05:30:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.030676+0000", 2024-09-14T05:30:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1549000000000003e-05 2024-09-14T05:30:58.868 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.0.0:1719 3.15 e100/24 3:adb89761:::rbd_data.112d46b1ca94.00000000000003a3:11 v 100'431, mlcod=100'425)", 2024-09-14T05:30:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.023954+0000", 2024-09-14T05:30:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.7391902409999993, 2024-09-14T05:30:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094961170000000001, 2024-09-14T05:30:58.869 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023954+0000", 2024-09-14T05:30:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023954+0000", 2024-09-14T05:30:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023955+0000", 2024-09-14T05:30:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7400000000000003e-07 2024-09-14T05:30:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023960+0000", 2024-09-14T05:30:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1460000000000004e-06 2024-09-14T05:30:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023970+0000", 2024-09-14T05:30:58.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0455000000000001e-05 2024-09-14T05:30:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023976+0000", 2024-09-14T05:30:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1659999999999999e-06 2024-09-14T05:30:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.025476+0000", 2024-09-14T05:30:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001499088 2024-09-14T05:30:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:58.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.025498+0000", 2024-09-14T05:30:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2041000000000001e-05 2024-09-14T05:30:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.033380+0000", 2024-09-14T05:30:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078826269999999997 2024-09-14T05:30:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.033450+0000", 2024-09-14T05:30:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9919999999999995e-05 2024-09-14T05:30:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.873 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.874 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:58.874 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:58.874 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:58.875 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:30:58.876 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:58.873+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:58.976 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:58.975+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:59.009 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is set 2024-09-14T05:30:59.021 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T05:30:59.021 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T05:30:59.021 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T05:30:59.021 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 0.991903 2024-09-14T05:30:59.021 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 0.9919032895753584 2024-09-14T05:30:59.025 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:59.025 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:59.025 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:30:59.026 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:59.035 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:30:59.078 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:30:59.077+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:30:59.147 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:30:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:30:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:30:59.147 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:59.156 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:30:59.198 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:30:59.196+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:30:59.279 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:30:59.277+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:30:59.303 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:30:59.303 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:30:59.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:30:59.303 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:30:59.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2131 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:30:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.527065187000005, 2024-09-14T05:30:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049526593000000001, 2024-09-14T05:30:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:30:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2131 2024-09-14T05:30:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.304 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:30:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774867+0000", 2024-09-14T05:30:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4430000000000002e-06 2024-09-14T05:30:59.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790357+0000", 2024-09-14T05:30:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015489466 2024-09-14T05:30:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790383+0000", 2024-09-14T05:30:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6247000000000001e-05 2024-09-14T05:30:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790394+0000", 2024-09-14T05:30:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0728e-05 2024-09-14T05:30:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790471+0000", 2024-09-14T05:30:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7539000000000004e-05 2024-09-14T05:30:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790639+0000", 2024-09-14T05:30:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016787700000000001 2024-09-14T05:30:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-14T05:30:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790998+0000", 2024-09-14T05:30:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035931299999999999 2024-09-14T05:30:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:30:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.823529+0000", 2024-09-14T05:30:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032530798 2024-09-14T05:30:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:30:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824324+0000", 2024-09-14T05:30:59.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00079442399999999995 2024-09-14T05:30:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:30:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824362+0000", 2024-09-14T05:30:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8398000000000001e-05 2024-09-14T05:30:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824391+0000", 2024-09-14T05:30:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.936e-05 2024-09-14T05:30:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.309 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3892 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7602176~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.945487189000005, 2024-09-14T05:30:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047082951999999997, 2024-09-14T05:30:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.310 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3892 2024-09-14T05:30:59.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.311 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:59.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:59.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:30:59.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0199999999999998e-07 2024-09-14T05:30:59.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356454+0000", 2024-09-14T05:30:59.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0414999999999999e-05 2024-09-14T05:30:59.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356463+0000", 2024-09-14T05:30:59.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6739999999999992e-06 2024-09-14T05:30:59.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356468+0000", 2024-09-14T05:30:59.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5680000000000001e-06 2024-09-14T05:30:59.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403143+0000", 2024-09-14T05:30:59.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046675051000000002 2024-09-14T05:30:59.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403206+0000", 2024-09-14T05:30:59.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3127999999999996e-05 2024-09-14T05:30:59.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403526+0000", 2024-09-14T05:30:59.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031981400000000002 2024-09-14T05:30:59.314 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.314 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.314 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.314 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3893 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8093696~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.314 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:59.315 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.945442352000001, 2024-09-14T05:30:59.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047516305000000002, 2024-09-14T05:30:59.315 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.315 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.315 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.315 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.315 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.315 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.315 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3893 2024-09-14T05:30:59.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.315 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:59.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:59.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:30:59.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6899999999999997e-07 2024-09-14T05:30:59.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356491+0000", 2024-09-14T05:30:59.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8109999999999999e-06 2024-09-14T05:30:59.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356498+0000", 2024-09-14T05:30:59.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5320000000000003e-06 2024-09-14T05:30:59.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356502+0000", 2024-09-14T05:30:59.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000002e-06 2024-09-14T05:30:59.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403579+0000", 2024-09-14T05:30:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047076706000000003 2024-09-14T05:30:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403664+0000", 2024-09-14T05:30:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5240000000000004e-05 2024-09-14T05:30:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404004+0000", 2024-09-14T05:30:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033974500000000002 2024-09-14T05:30:59.318 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3993 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 262144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.945420480999999, 2024-09-14T05:30:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048088476999999998, 2024-09-14T05:30:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.319 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3993 2024-09-14T05:30:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:59.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:30:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.46e-07 2024-09-14T05:30:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356512+0000", 2024-09-14T05:30:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.827e-06 2024-09-14T05:30:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356518+0000", 2024-09-14T05:30:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7780000000000004e-06 2024-09-14T05:30:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356522+0000", 2024-09-14T05:30:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.865e-06 2024-09-14T05:30:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404036+0000", 2024-09-14T05:30:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047513725999999999 2024-09-14T05:30:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404135+0000", 2024-09-14T05:30:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9352999999999996e-05 2024-09-14T05:30:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404598+0000", 2024-09-14T05:30:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046258200000000001 2024-09-14T05:30:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3994 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.945378469000005, 2024-09-14T05:30:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048274625000000002, 2024-09-14T05:30:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3994 2024-09-14T05:30:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:59.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:59.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:30:59.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3799999999999997e-07 2024-09-14T05:30:59.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356554+0000", 2024-09-14T05:30:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2259999999999999e-06 2024-09-14T05:30:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356560+0000", 2024-09-14T05:30:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-14T05:30:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356566+0000", 2024-09-14T05:30:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3909999999999997e-06 2024-09-14T05:30:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404632+0000", 2024-09-14T05:30:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048066154 2024-09-14T05:30:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404719+0000", 2024-09-14T05:30:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7440000000000003e-05 2024-09-14T05:30:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404826+0000", 2024-09-14T05:30:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106813 2024-09-14T05:30:59.327 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3995 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.945353330000003, 2024-09-14T05:30:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048828209999999997, 2024-09-14T05:30:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.328 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3995 2024-09-14T05:30:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:59.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:30:59.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6599999999999996e-07 2024-09-14T05:30:59.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356579+0000", 2024-09-14T05:30:59.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0719999999999998e-06 2024-09-14T05:30:59.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356585+0000", 2024-09-14T05:30:59.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.959e-06 2024-09-14T05:30:59.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356590+0000", 2024-09-14T05:30:59.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.464e-06 2024-09-14T05:30:59.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404875+0000", 2024-09-14T05:30:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048285649999999999 2024-09-14T05:30:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404985+0000", 2024-09-14T05:30:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109417 2024-09-14T05:30:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405405+0000", 2024-09-14T05:30:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042008200000000001 2024-09-14T05:30:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3996 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.945044447000001, 2024-09-14T05:30:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049061595, 2024-09-14T05:30:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.334 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.335 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.335 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.335 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.335 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.335 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3996 2024-09-14T05:30:59.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.335 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:59.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:30:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.61e-07 2024-09-14T05:30:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356892+0000", 2024-09-14T05:30:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9050000000000002e-06 2024-09-14T05:30:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356901+0000", 2024-09-14T05:30:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4149999999999999e-06 2024-09-14T05:30:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356907+0000", 2024-09-14T05:30:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0140000000000004e-06 2024-09-14T05:30:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405448+0000", 2024-09-14T05:30:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048541647 2024-09-14T05:30:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405542+0000", 2024-09-14T05:30:59.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3639000000000003e-05 2024-09-14T05:30:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405947+0000", 2024-09-14T05:30:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040521399999999999 2024-09-14T05:30:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.339 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3997 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2228224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.340 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:59.340 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.944830464999995, 2024-09-14T05:30:59.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049308774999999999, 2024-09-14T05:30:59.340 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.340 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.340 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.340 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.340 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.340 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.340 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3997 2024-09-14T05:30:59.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:30:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6499999999999999e-07 2024-09-14T05:30:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357104+0000", 2024-09-14T05:30:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4989999999999999e-06 2024-09-14T05:30:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357112+0000", 2024-09-14T05:30:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4279999999999996e-06 2024-09-14T05:30:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357117+0000", 2024-09-14T05:30:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6190000000000002e-06 2024-09-14T05:30:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405989+0000", 2024-09-14T05:30:59.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048872080999999998 2024-09-14T05:30:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406074+0000", 2024-09-14T05:30:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4676e-05 2024-09-14T05:30:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406408+0000", 2024-09-14T05:30:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033480700000000001 2024-09-14T05:30:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.344 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3998 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.944509035999999, 2024-09-14T05:30:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049462314, 2024-09-14T05:30:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.345 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3998 2024-09-14T05:30:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:30:59.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357422+0000", 2024-09-14T05:30:59.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3099999999999996e-07 2024-09-14T05:30:59.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357429+0000", 2024-09-14T05:30:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2880000000000004e-06 2024-09-14T05:30:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357439+0000", 2024-09-14T05:30:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.302e-06 2024-09-14T05:30:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357445+0000", 2024-09-14T05:30:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4269999999999996e-06 2024-09-14T05:30:59.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406448+0000", 2024-09-14T05:30:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049003096000000003 2024-09-14T05:30:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406537+0000", 2024-09-14T05:30:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9135999999999995e-05 2024-09-14T05:30:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406883+0000", 2024-09-14T05:30:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000346134 2024-09-14T05:30:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3999 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.944438679000001, 2024-09-14T05:30:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049797556, 2024-09-14T05:30:59.350 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3999 2024-09-14T05:30:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:59.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:30:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357492+0000", 2024-09-14T05:30:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8299999999999996e-07 2024-09-14T05:30:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357495+0000", 2024-09-14T05:30:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0450000000000001e-06 2024-09-14T05:30:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357503+0000", 2024-09-14T05:30:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9689999999999994e-06 2024-09-14T05:30:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357510+0000", 2024-09-14T05:30:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-14T05:30:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406914+0000", 2024-09-14T05:30:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049404293000000002 2024-09-14T05:30:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406965+0000", 2024-09-14T05:30:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0470999999999998e-05 2024-09-14T05:30:59.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407289+0000", 2024-09-14T05:30:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032434899999999998 2024-09-14T05:30:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4000 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3702784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:59.355 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.944092334000004, 2024-09-14T05:30:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049936728999999999, 2024-09-14T05:30:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4000 2024-09-14T05:30:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:30:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357839+0000", 2024-09-14T05:30:59.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-14T05:30:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357842+0000", 2024-09-14T05:30:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4850000000000001e-06 2024-09-14T05:30:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357851+0000", 2024-09-14T05:30:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1819999999999995e-06 2024-09-14T05:30:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357859+0000", 2024-09-14T05:30:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8129999999999996e-06 2024-09-14T05:30:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407337+0000", 2024-09-14T05:30:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049478084999999998 2024-09-14T05:30:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407442+0000", 2024-09-14T05:30:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000105302 2024-09-14T05:30:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407774+0000", 2024-09-14T05:30:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033200100000000001 2024-09-14T05:30:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4001 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.943996979000005, 2024-09-14T05:30:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050304205999999997, 2024-09-14T05:30:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.361 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4001 2024-09-14T05:30:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:30:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357934+0000", 2024-09-14T05:30:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000004e-07 2024-09-14T05:30:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357937+0000", 2024-09-14T05:30:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0869999999999998e-06 2024-09-14T05:30:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357946+0000", 2024-09-14T05:30:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1439999999999992e-06 2024-09-14T05:30:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357952+0000", 2024-09-14T05:30:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.609e-06 2024-09-14T05:30:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407820+0000", 2024-09-14T05:30:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049868580000000003 2024-09-14T05:30:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407909+0000", 2024-09-14T05:30:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8887000000000005e-05 2024-09-14T05:30:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408237+0000", 2024-09-14T05:30:59.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032824599999999997 2024-09-14T05:30:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4002 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.943780383999993, 2024-09-14T05:30:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050539529999999999, 2024-09-14T05:30:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.366 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4002 2024-09-14T05:30:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:30:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5099999999999999e-07 2024-09-14T05:30:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358154+0000", 2024-09-14T05:30:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6799999999999999e-06 2024-09-14T05:30:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358165+0000", 2024-09-14T05:30:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0899e-05 2024-09-14T05:30:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358172+0000", 2024-09-14T05:30:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8689999999999998e-06 2024-09-14T05:30:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408281+0000", 2024-09-14T05:30:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050108699 2024-09-14T05:30:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408369+0000", 2024-09-14T05:30:59.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8542999999999996e-05 2024-09-14T05:30:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408689+0000", 2024-09-14T05:30:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032008900000000002 2024-09-14T05:30:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4003 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5177344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.371 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.943740548999997, 2024-09-14T05:30:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050868759, 2024-09-14T05:30:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4003 2024-09-14T05:30:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.372 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:30:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-07 2024-09-14T05:30:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358193+0000", 2024-09-14T05:30:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.588e-06 2024-09-14T05:30:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358204+0000", 2024-09-14T05:30:59.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1525999999999999e-05 2024-09-14T05:30:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358213+0000", 2024-09-14T05:30:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5010000000000004e-06 2024-09-14T05:30:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408719+0000", 2024-09-14T05:30:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050506708999999997 2024-09-14T05:30:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408766+0000", 2024-09-14T05:30:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6579999999999998e-05 2024-09-14T05:30:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409058+0000", 2024-09-14T05:30:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000292365 2024-09-14T05:30:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4004 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5668864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.943317706000002, 2024-09-14T05:30:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050839848, 2024-09-14T05:30:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4004 2024-09-14T05:30:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:30:59.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358613+0000", 2024-09-14T05:30:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:30:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358618+0000", 2024-09-14T05:30:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6009999999999997e-06 2024-09-14T05:30:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358630+0000", 2024-09-14T05:30:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2122e-05 2024-09-14T05:30:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358638+0000", 2024-09-14T05:30:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3969999999999994e-06 2024-09-14T05:30:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409088+0000", 2024-09-14T05:30:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050449565000000002 2024-09-14T05:30:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409153+0000", 2024-09-14T05:30:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4728999999999996e-05 2024-09-14T05:30:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409452+0000", 2024-09-14T05:30:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029958900000000001 2024-09-14T05:30:59.381 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4005 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6160384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.943100771999994, 2024-09-14T05:30:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051054752000000002, 2024-09-14T05:30:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.382 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4005 2024-09-14T05:30:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:30:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358830+0000", 2024-09-14T05:30:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-14T05:30:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358833+0000", 2024-09-14T05:30:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9529999999999999e-06 2024-09-14T05:30:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358843+0000", 2024-09-14T05:30:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8069999999999994e-06 2024-09-14T05:30:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358849+0000", 2024-09-14T05:30:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1040000000000003e-06 2024-09-14T05:30:59.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409487+0000", 2024-09-14T05:30:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050638626999999999 2024-09-14T05:30:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409535+0000", 2024-09-14T05:30:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7525000000000003e-05 2024-09-14T05:30:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409884+0000", 2024-09-14T05:30:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034914800000000003 2024-09-14T05:30:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4006 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.943063660000007, 2024-09-14T05:30:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051425323000000002, 2024-09-14T05:30:59.387 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4006 2024-09-14T05:30:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:30:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358867+0000", 2024-09-14T05:30:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2900000000000004e-07 2024-09-14T05:30:59.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358869+0000", 2024-09-14T05:30:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4210000000000002e-06 2024-09-14T05:30:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358876+0000", 2024-09-14T05:30:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4980000000000003e-06 2024-09-14T05:30:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358881+0000", 2024-09-14T05:30:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3090000000000003e-06 2024-09-14T05:30:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409915+0000", 2024-09-14T05:30:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051033348999999999 2024-09-14T05:30:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409964+0000", 2024-09-14T05:30:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9858000000000003e-05 2024-09-14T05:30:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410292+0000", 2024-09-14T05:30:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032735899999999999 2024-09-14T05:30:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4007 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.943041066999996, 2024-09-14T05:30:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05184076, 2024-09-14T05:30:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4007 2024-09-14T05:30:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:59.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:30:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:30:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358891+0000", 2024-09-14T05:30:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9819999999999998e-06 2024-09-14T05:30:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358897+0000", 2024-09-14T05:30:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9390000000000002e-06 2024-09-14T05:30:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358902+0000", 2024-09-14T05:30:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6770000000000003e-06 2024-09-14T05:30:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410335+0000", 2024-09-14T05:30:59.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051432991999999997 2024-09-14T05:30:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410416+0000", 2024-09-14T05:30:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1316999999999994e-05 2024-09-14T05:30:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410730+0000", 2024-09-14T05:30:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000313419 2024-09-14T05:30:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4008 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7634944~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.943016705999995, 2024-09-14T05:30:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052208082000000003, 2024-09-14T05:30:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4008 2024-09-14T05:30:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:30:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358914+0000", 2024-09-14T05:30:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6100000000000001e-07 2024-09-14T05:30:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358916+0000", 2024-09-14T05:30:59.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.902e-06 2024-09-14T05:30:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358923+0000", 2024-09-14T05:30:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4780000000000002e-06 2024-09-14T05:30:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358929+0000", 2024-09-14T05:30:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0970000000000002e-06 2024-09-14T05:30:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410761+0000", 2024-09-14T05:30:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051831436000000002 2024-09-14T05:30:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410829+0000", 2024-09-14T05:30:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7874999999999996e-05 2024-09-14T05:30:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411121+0000", 2024-09-14T05:30:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029283299999999999 2024-09-14T05:30:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4009 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8126464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:30:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 87.942869481000002, 2024-09-14T05:30:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052518169000000003, 2024-09-14T05:30:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:30:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:30:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:30:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:30:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:30:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4009 2024-09-14T05:30:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:30:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:30:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:30:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:30:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:30:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:30:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0200000000000002e-07 2024-09-14T05:30:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:30:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359066+0000", 2024-09-14T05:30:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7899999999999999e-06 2024-09-14T05:30:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:30:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359074+0000", 2024-09-14T05:30:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2579999999999995e-06 2024-09-14T05:30:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359082+0000", 2024-09-14T05:30:59.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.464e-06 2024-09-14T05:30:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:30:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411160+0000", 2024-09-14T05:30:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052078078 2024-09-14T05:30:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:30:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411244+0000", 2024-09-14T05:30:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4041000000000004e-05 2024-09-14T05:30:59.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:30:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:30:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:30:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411579+0000", 2024-09-14T05:30:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033493599999999998 2024-09-14T05:30:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:30:59.408 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:30:59.408 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:30:59.410 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:30:59.411 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:59.409+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:59.511 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:30:59.510+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:30:59.513 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:59.513 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:59.514 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:30:59.514 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:59.522 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:30:59.612 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:30:59.612+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:30:59.665 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:59.666 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:30:59.666 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:30:59.666 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:30:59.666 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:59.674 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:30:59.684 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T05:30:59.713 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:30:59.712+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:30:59.814 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:30:59.813+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:30:59.822 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:30:59.822 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:30:59.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:30:59.822 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:30:59.822 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.822 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74187 3.f 3:f707b2d9:::rbd_data.112d46b1ca94.0000000000000061:11 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.822 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:59.822 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.252495534000005, 2024-09-14T05:30:59.822 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072367828999999995, 2024-09-14T05:30:59.822 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.823 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.823 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.823 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.823 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.823 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.823 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74187 2024-09-14T05:30:59.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.823 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.823 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.823 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:59.823 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:59.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:30:59.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0900000000000002e-07 2024-09-14T05:30:59.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.824 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.824 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568166+0000", 2024-09-14T05:30:59.824 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.683e-06 2024-09-14T05:30:59.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568172+0000", 2024-09-14T05:30:59.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0639999999999999e-06 2024-09-14T05:30:59.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.825 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.825 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568177+0000", 2024-09-14T05:30:59.825 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.967e-06 2024-09-14T05:30:59.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.828 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.828 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640193+0000", 2024-09-14T05:30:59.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072016284 2024-09-14T05:30:59.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640282+0000", 2024-09-14T05:30:59.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9011999999999994e-05 2024-09-14T05:30:59.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.829 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.829 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640532+0000", 2024-09-14T05:30:59.829 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024940999999999999 2024-09-14T05:30:59.829 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.829 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.830 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.830 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74188 3.f 3:f707b2d9:::rbd_data.112d46b1ca94.0000000000000061:11 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.830 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:59.830 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.252476161000004, 2024-09-14T05:30:59.830 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072600425999999996, 2024-09-14T05:30:59.830 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.830 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.830 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.830 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.830 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.831 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.831 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74188 2024-09-14T05:30:59.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.831 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:59.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.831 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.831 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:30:59.831 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568184+0000", 2024-09-14T05:30:59.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8700000000000001e-07 2024-09-14T05:30:59.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.832 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568200+0000", 2024-09-14T05:30:59.832 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6603e-05 2024-09-14T05:30:59.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.832 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568208+0000", 2024-09-14T05:30:59.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4800000000000004e-06 2024-09-14T05:30:59.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568213+0000", 2024-09-14T05:30:59.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6860000000000003e-06 2024-09-14T05:30:59.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.833 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.833 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640563+0000", 2024-09-14T05:30:59.833 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072349438000000002 2024-09-14T05:30:59.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640607+0000", 2024-09-14T05:30:59.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4014999999999998e-05 2024-09-14T05:30:59.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.834 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.834 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640784+0000", 2024-09-14T05:30:59.834 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017681700000000001 2024-09-14T05:30:59.834 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.834 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.834 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.835 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74194 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.835 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:59.835 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.252438029000004, 2024-09-14T05:30:59.835 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072795199000000005, 2024-09-14T05:30:59.835 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.835 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.835 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.835 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.835 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.835 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.835 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74194 2024-09-14T05:30:59.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.835 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:59.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:30:59.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.836 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.836 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568222+0000", 2024-09-14T05:30:59.836 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1300000000000001e-07 2024-09-14T05:30:59.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568224+0000", 2024-09-14T05:30:59.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0660000000000002e-06 2024-09-14T05:30:59.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568230+0000", 2024-09-14T05:30:59.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4470000000000003e-06 2024-09-14T05:30:59.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.837 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.837 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568235+0000", 2024-09-14T05:30:59.837 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1220000000000003e-06 2024-09-14T05:30:59.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640824+0000", 2024-09-14T05:30:59.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072588707000000002 2024-09-14T05:30:59.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640915+0000", 2024-09-14T05:30:59.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1008999999999995e-05 2024-09-14T05:30:59.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.838 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.838 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641017+0000", 2024-09-14T05:30:59.838 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010143499999999999 2024-09-14T05:30:59.838 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.839 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.839 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.839 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74208 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2693120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.839 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:59.839 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.252417453000007, 2024-09-14T05:30:59.839 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072905555999999996, 2024-09-14T05:30:59.839 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.839 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.839 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.839 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.839 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.839 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.839 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74208 2024-09-14T05:30:59.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.840 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:59.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.840 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.840 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:59.840 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.840 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:30:59.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9799999999999999e-07 2024-09-14T05:30:59.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568244+0000", 2024-09-14T05:30:59.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0339999999999999e-06 2024-09-14T05:30:59.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.841 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.841 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568251+0000", 2024-09-14T05:30:59.841 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4060000000000002e-06 2024-09-14T05:30:59.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568255+0000", 2024-09-14T05:30:59.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6589999999999998e-06 2024-09-14T05:30:59.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641048+0000", 2024-09-14T05:30:59.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072792647000000002 2024-09-14T05:30:59.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.842 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.842 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641086+0000", 2024-09-14T05:30:59.842 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8031999999999998e-05 2024-09-14T05:30:59.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.843 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.843 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641147+0000", 2024-09-14T05:30:59.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1379999999999998e-05 2024-09-14T05:30:59.843 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.843 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.843 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.843 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74236 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2662400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.843 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:59.843 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.251783790000005, 2024-09-14T05:30:59.843 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072517776000000006, 2024-09-14T05:30:59.843 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.844 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.844 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.844 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.844 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.844 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.844 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74236 2024-09-14T05:30:59.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.844 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.844 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:59.844 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.844 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:30:59.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568881+0000", 2024-09-14T05:30:59.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2340000000000002e-06 2024-09-14T05:30:59.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.845 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.845 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568886+0000", 2024-09-14T05:30:59.845 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9450000000000001e-06 2024-09-14T05:30:59.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568898+0000", 2024-09-14T05:30:59.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2150000000000001e-05 2024-09-14T05:30:59.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568907+0000", 2024-09-14T05:30:59.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9239999999999996e-06 2024-09-14T05:30:59.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.846 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.846 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641185+0000", 2024-09-14T05:30:59.846 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.07227807 2024-09-14T05:30:59.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641265+0000", 2024-09-14T05:30:59.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0575999999999996e-05 2024-09-14T05:30:59.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.847 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.847 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641393+0000", 2024-09-14T05:30:59.847 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012787700000000001 2024-09-14T05:30:59.847 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.847 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.847 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.847 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74239 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2695168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.847 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:59.848 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.251739396999994, 2024-09-14T05:30:59.848 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072593816000000005, 2024-09-14T05:30:59.848 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.848 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.848 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.848 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.848 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.848 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.848 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74239 2024-09-14T05:30:59.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.848 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.848 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:59.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:59.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.849 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.849 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:30:59.849 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4300000000000003e-07 2024-09-14T05:30:59.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568923+0000", 2024-09-14T05:30:59.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2019999999999998e-06 2024-09-14T05:30:59.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568933+0000", 2024-09-14T05:30:59.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0376e-05 2024-09-14T05:30:59.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.850 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.850 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568938+0000", 2024-09-14T05:30:59.850 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2850000000000002e-06 2024-09-14T05:30:59.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641424+0000", 2024-09-14T05:30:59.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072485230999999997 2024-09-14T05:30:59.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.851 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641464+0000", 2024-09-14T05:30:59.851 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0574000000000002e-05 2024-09-14T05:30:59.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.851 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.852 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641514+0000", 2024-09-14T05:30:59.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9604999999999997e-05 2024-09-14T05:30:59.852 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.852 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.852 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.852 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74252 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2664448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.852 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:59.852 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.251559482999994, 2024-09-14T05:30:59.852 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072631237000000001, 2024-09-14T05:30:59.852 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.852 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.852 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.853 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.853 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.853 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.853 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74252 2024-09-14T05:30:59.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.853 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:59.853 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.853 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.853 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:30:59.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569101+0000", 2024-09-14T05:30:59.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8700000000000005e-07 2024-09-14T05:30:59.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.854 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569104+0000", 2024-09-14T05:30:59.854 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.163e-06 2024-09-14T05:30:59.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.854 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569115+0000", 2024-09-14T05:30:59.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1335000000000001e-05 2024-09-14T05:30:59.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569121+0000", 2024-09-14T05:30:59.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0800000000000002e-06 2024-09-14T05:30:59.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.855 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.855 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641552+0000", 2024-09-14T05:30:59.855 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072430743000000006 2024-09-14T05:30:59.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641631+0000", 2024-09-14T05:30:59.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9388999999999995e-05 2024-09-14T05:30:59.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.856 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.856 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641731+0000", 2024-09-14T05:30:59.856 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9840000000000006e-05 2024-09-14T05:30:59.856 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.856 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.856 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.857 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74256 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2697216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.857 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:59.857 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.251528426999997, 2024-09-14T05:30:59.857 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072721090000000002, 2024-09-14T05:30:59.857 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.857 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.857 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.857 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.857 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.857 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.857 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74256 2024-09-14T05:30:59.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.857 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:59.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:59.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.858 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.858 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:30:59.858 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0299999999999999e-07 2024-09-14T05:30:59.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569133+0000", 2024-09-14T05:30:59.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.046e-06 2024-09-14T05:30:59.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569139+0000", 2024-09-14T05:30:59.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0279999999999997e-06 2024-09-14T05:30:59.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.859 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.859 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569144+0000", 2024-09-14T05:30:59.859 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6929999999999998e-06 2024-09-14T05:30:59.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641762+0000", 2024-09-14T05:30:59.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072617904999999996 2024-09-14T05:30:59.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.860 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641801+0000", 2024-09-14T05:30:59.860 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8902999999999997e-05 2024-09-14T05:30:59.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.860 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.861 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641852+0000", 2024-09-14T05:30:59.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1011999999999999e-05 2024-09-14T05:30:59.861 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.861 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.861 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.861 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74285 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2672640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.861 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:59.861 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.250682991000005, 2024-09-14T05:30:59.861 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072422069000000006, 2024-09-14T05:30:59.861 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.861 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.861 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.862 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.862 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.862 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.862 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74285 2024-09-14T05:30:59.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.862 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.862 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:59.862 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.862 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:30:59.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569977+0000", 2024-09-14T05:30:59.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3099999999999998e-07 2024-09-14T05:30:59.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.863 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569978+0000", 2024-09-14T05:30:59.863 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.598e-06 2024-09-14T05:30:59.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.863 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569984+0000", 2024-09-14T05:30:59.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4689999999999996e-06 2024-09-14T05:30:59.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569989+0000", 2024-09-14T05:30:59.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8409999999999999e-06 2024-09-14T05:30:59.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.864 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.864 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642229+0000", 2024-09-14T05:30:59.864 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072240046000000002 2024-09-14T05:30:59.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642306+0000", 2024-09-14T05:30:59.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7347000000000003e-05 2024-09-14T05:30:59.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.865 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.865 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642398+0000", 2024-09-14T05:30:59.865 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2336999999999994e-05 2024-09-14T05:30:59.865 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.865 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.865 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.866 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74292 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2705408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.866 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:59.866 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.250428283000005, 2024-09-14T05:30:59.866 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072361018999999999, 2024-09-14T05:30:59.866 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.866 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.866 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.866 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.866 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.866 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.866 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74292 2024-09-14T05:30:59.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.867 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:59.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:30:59.867 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.867 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.867 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570232+0000", 2024-09-14T05:30:59.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.06e-07 2024-09-14T05:30:59.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570235+0000", 2024-09-14T05:30:59.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3189999999999999e-06 2024-09-14T05:30:59.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.868 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.868 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570248+0000", 2024-09-14T05:30:59.868 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3006e-05 2024-09-14T05:30:59.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570254+0000", 2024-09-14T05:30:59.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.908e-06 2024-09-14T05:30:59.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642433+0000", 2024-09-14T05:30:59.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072179192000000003 2024-09-14T05:30:59.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.869 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.869 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642500+0000", 2024-09-14T05:30:59.869 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7080000000000001e-05 2024-09-14T05:30:59.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.870 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.870 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642592+0000", 2024-09-14T05:30:59.870 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1807999999999999e-05 2024-09-14T05:30:59.870 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.870 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.870 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.870 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74304 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2707456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.870 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:59.870 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.250071617000003, 2024-09-14T05:30:59.871 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072444960000000003, 2024-09-14T05:30:59.871 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.871 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.871 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.871 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.871 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.871 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.871 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74304 2024-09-14T05:30:59.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.871 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.871 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.871 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:59.871 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.872 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:59.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.872 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:30:59.872 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9800000000000004e-07 2024-09-14T05:30:59.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.872 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.873 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570590+0000", 2024-09-14T05:30:59.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0810000000000001e-06 2024-09-14T05:30:59.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.873 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.873 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570596+0000", 2024-09-14T05:30:59.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9789999999999998e-06 2024-09-14T05:30:59.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.873 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.873 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570601+0000", 2024-09-14T05:30:59.873 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6750000000000001e-06 2024-09-14T05:30:59.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642835+0000", 2024-09-14T05:30:59.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.07223359 2024-09-14T05:30:59.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642917+0000", 2024-09-14T05:30:59.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2013000000000002e-05 2024-09-14T05:30:59.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.874 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.874 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643033+0000", 2024-09-14T05:30:59.874 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116124 2024-09-14T05:30:59.875 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.875 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.875 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.875 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74321 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2709504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.875 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:59.875 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.249589627000006, 2024-09-14T05:30:59.875 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072350611999999995, 2024-09-14T05:30:59.875 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.875 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.875 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.875 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.875 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.876 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.876 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74321 2024-09-14T05:30:59.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.876 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:59.876 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.876 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.876 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:59.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:30:59.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3e-07 2024-09-14T05:30:59.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.877 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571073+0000", 2024-09-14T05:30:59.877 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6879999999999999e-06 2024-09-14T05:30:59.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.877 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571080+0000", 2024-09-14T05:30:59.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0319999999999998e-06 2024-09-14T05:30:59.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571085+0000", 2024-09-14T05:30:59.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2109999999999997e-06 2024-09-14T05:30:59.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.878 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.878 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643309+0000", 2024-09-14T05:30:59.878 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072223812999999998 2024-09-14T05:30:59.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643351+0000", 2024-09-14T05:30:59.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2239e-05 2024-09-14T05:30:59.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.879 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.879 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643420+0000", 2024-09-14T05:30:59.879 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8998999999999994e-05 2024-09-14T05:30:59.879 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.879 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.879 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.880 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74332 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2678784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.880 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:59.880 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.249266464000002, 2024-09-14T05:30:59.880 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072259062999999998, 2024-09-14T05:30:59.880 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.880 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.880 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.880 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.880 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.880 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.880 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74332 2024-09-14T05:30:59.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.880 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:59.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:30:59.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.881 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.881 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571394+0000", 2024-09-14T05:30:59.881 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:30:59.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571397+0000", 2024-09-14T05:30:59.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8469999999999998e-06 2024-09-14T05:30:59.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.882 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571411+0000", 2024-09-14T05:30:59.882 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.34e-05 2024-09-14T05:30:59.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.882 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571418+0000", 2024-09-14T05:30:59.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9149999999999999e-06 2024-09-14T05:30:59.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643458+0000", 2024-09-14T05:30:59.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072039881 2024-09-14T05:30:59.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.883 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.883 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643539+0000", 2024-09-14T05:30:59.883 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1650000000000006e-05 2024-09-14T05:30:59.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.884 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.884 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643652+0000", 2024-09-14T05:30:59.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000112593 2024-09-14T05:30:59.884 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.884 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.884 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.884 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74336 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2711552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.884 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:59.884 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.249230014000005, 2024-09-14T05:30:59.884 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072354038999999995, 2024-09-14T05:30:59.885 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.885 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.885 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.885 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.885 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.885 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.885 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74336 2024-09-14T05:30:59.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.885 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.885 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.885 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:59.885 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:30:59.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571430+0000", 2024-09-14T05:30:59.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8999999999999996e-07 2024-09-14T05:30:59.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.886 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.886 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571432+0000", 2024-09-14T05:30:59.886 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4159999999999998e-06 2024-09-14T05:30:59.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571439+0000", 2024-09-14T05:30:59.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.657e-06 2024-09-14T05:30:59.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571444+0000", 2024-09-14T05:30:59.887 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1329999999999998e-06 2024-09-14T05:30:59.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.887 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.887 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643690+0000", 2024-09-14T05:30:59.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072246088 2024-09-14T05:30:59.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643731+0000", 2024-09-14T05:30:59.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0311000000000001e-05 2024-09-14T05:30:59.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.888 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.888 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643783+0000", 2024-09-14T05:30:59.888 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2843999999999998e-05 2024-09-14T05:30:59.888 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.888 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.889 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.889 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74349 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.889 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:59.889 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.249209004999997, 2024-09-14T05:30:59.889 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072554297000000004, 2024-09-14T05:30:59.889 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.889 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.889 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.889 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.889 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.889 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.890 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74349 2024-09-14T05:30:59.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.890 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:59.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.890 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.890 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:30:59.890 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571451+0000", 2024-09-14T05:30:59.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.41e-07 2024-09-14T05:30:59.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571453+0000", 2024-09-14T05:30:59.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8360000000000001e-06 2024-09-14T05:30:59.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.891 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.891 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571475+0000", 2024-09-14T05:30:59.891 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2452000000000001e-05 2024-09-14T05:30:59.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571480+0000", 2024-09-14T05:30:59.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4430000000000002e-06 2024-09-14T05:30:59.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.892 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643821+0000", 2024-09-14T05:30:59.892 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072340347999999999 2024-09-14T05:30:59.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.892 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643904+0000", 2024-09-14T05:30:59.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3534000000000004e-05 2024-09-14T05:30:59.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.893 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.893 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644005+0000", 2024-09-14T05:30:59.893 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100343 2024-09-14T05:30:59.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.893 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.893 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74357 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2713600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.893 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:59.894 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.249164972000003, 2024-09-14T05:30:59.894 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072651144000000001, 2024-09-14T05:30:59.894 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.894 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.894 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.894 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.894 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74357 2024-09-14T05:30:59.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.894 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.894 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:59.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:30:59.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.895 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.895 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571495+0000", 2024-09-14T05:30:59.895 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8899999999999999e-07 2024-09-14T05:30:59.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571498+0000", 2024-09-14T05:30:59.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5840000000000002e-06 2024-09-14T05:30:59.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.896 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.896 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571504+0000", 2024-09-14T05:30:59.896 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6549999999999998e-06 2024-09-14T05:30:59.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571509+0000", 2024-09-14T05:30:59.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6500000000000004e-06 2024-09-14T05:30:59.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644043+0000", 2024-09-14T05:30:59.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072533832000000006 2024-09-14T05:30:59.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.897 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.897 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644090+0000", 2024-09-14T05:30:59.897 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7426000000000003e-05 2024-09-14T05:30:59.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.898 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.898 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644146+0000", 2024-09-14T05:30:59.898 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5408000000000002e-05 2024-09-14T05:30:59.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.898 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.898 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74364 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2682880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.898 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:59.899 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.248828200000005, 2024-09-14T05:30:59.899 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072531227000000004, 2024-09-14T05:30:59.899 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.899 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.899 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.899 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.899 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.899 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.899 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74364 2024-09-14T05:30:59.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.899 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.899 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.899 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:59.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:30:59.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.900 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.900 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571832+0000", 2024-09-14T05:30:59.900 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4e-07 2024-09-14T05:30:59.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571838+0000", 2024-09-14T05:30:59.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0519999999999998e-06 2024-09-14T05:30:59.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571865+0000", 2024-09-14T05:30:59.901 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7109000000000001e-05 2024-09-14T05:30:59.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.901 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.901 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571883+0000", 2024-09-14T05:30:59.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7428e-05 2024-09-14T05:30:59.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644182+0000", 2024-09-14T05:30:59.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072299551000000004 2024-09-14T05:30:59.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644260+0000", 2024-09-14T05:30:59.902 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7551000000000003e-05 2024-09-14T05:30:59.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.902 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.902 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644362+0000", 2024-09-14T05:30:59.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010259600000000001 2024-09-14T05:30:59.903 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.903 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.903 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.903 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74367 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2715648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.903 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:59.903 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.248464893000005, 2024-09-14T05:30:59.903 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072368259000000004, 2024-09-14T05:30:59.903 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.903 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.903 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.904 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.904 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.904 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.904 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74367 2024-09-14T05:30:59.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.904 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.904 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:59.904 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.904 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:30:59.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572195+0000", 2024-09-14T05:30:59.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.78e-07 2024-09-14T05:30:59.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.905 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.905 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572200+0000", 2024-09-14T05:30:59.905 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2520000000000001e-06 2024-09-14T05:30:59.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572213+0000", 2024-09-14T05:30:59.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3618000000000001e-05 2024-09-14T05:30:59.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572226+0000", 2024-09-14T05:30:59.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3040999999999999e-05 2024-09-14T05:30:59.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.906 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.906 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644409+0000", 2024-09-14T05:30:59.906 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072182982000000007 2024-09-14T05:30:59.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644489+0000", 2024-09-14T05:30:59.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9429000000000001e-05 2024-09-14T05:30:59.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.907 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.907 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644563+0000", 2024-09-14T05:30:59.907 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4059000000000004e-05 2024-09-14T05:30:59.907 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.907 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.907 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.908 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74380 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2684928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.908 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:59.908 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.248331114999999, 2024-09-14T05:30:59.908 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072442180999999994, 2024-09-14T05:30:59.908 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.908 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.908 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.908 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.908 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.908 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.908 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74380 2024-09-14T05:30:59.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.909 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:59.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:30:59.909 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.909 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.909 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572329+0000", 2024-09-14T05:30:59.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3300000000000001e-07 2024-09-14T05:30:59.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572333+0000", 2024-09-14T05:30:59.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8e-06 2024-09-14T05:30:59.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572346+0000", 2024-09-14T05:30:59.910 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.307e-05 2024-09-14T05:30:59.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.910 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.910 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572353+0000", 2024-09-14T05:30:59.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9e-06 2024-09-14T05:30:59.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644597+0000", 2024-09-14T05:30:59.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072244770999999999 2024-09-14T05:30:59.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.911 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.911 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644672+0000", 2024-09-14T05:30:59.911 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4945999999999997e-05 2024-09-14T05:30:59.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.912 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.912 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644770+0000", 2024-09-14T05:30:59.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7961000000000006e-05 2024-09-14T05:30:59.912 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.912 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.912 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.912 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74383 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2717696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:30:59.912 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:59.912 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 70.248075120999999, 2024-09-14T05:30:59.912 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072397258000000006, 2024-09-14T05:30:59.912 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:30:59.913 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:30:59.913 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:30:59.913 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:30:59.913 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:30:59.913 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:30:59.913 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74383 2024-09-14T05:30:59.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.913 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:30:59.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:30:59.913 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:59.913 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.913 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:30:59.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:30:59.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:30:59.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.914 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:30:59.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572585+0000", 2024-09-14T05:30:59.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:30:59.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.914 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:30:59.914 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572589+0000", 2024-09-14T05:30:59.914 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:30:59.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:30:59.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572602+0000", 2024-09-14T05:30:59.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2917e-05 2024-09-14T05:30:59.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:30:59.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572610+0000", 2024-09-14T05:30:59.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.943e-06 2024-09-14T05:30:59.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.915 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:30:59.915 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644791+0000", 2024-09-14T05:30:59.915 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072181113000000005 2024-09-14T05:30:59.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:30:59.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644861+0000", 2024-09-14T05:30:59.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0650999999999998e-05 2024-09-14T05:30:59.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:30:59.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:30:59.916 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:30:59.916 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644981+0000", 2024-09-14T05:30:59.916 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000120185 2024-09-14T05:30:59.916 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.916 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.916 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.917 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:30:59.917 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:30:59.917 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:30:59.918 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:30:59.919 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:30:59.919 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:30:59.919 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:30:59.920 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:30:59.920 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:30:59.920 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:30:59.920 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:30:59.921 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:30:59.921 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:30:59.921 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:30:59.921 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:30:59.917+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:31:00.004 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.7 primary-affinity to 0.991903 (8650052) 2024-09-14T05:31:00.019 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:31:00.018+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:31:00.076 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:31:00.076 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:31:00.086 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:31:00.124 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:31:00.125 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:31:00.125 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:31:00.124+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:31:00.142 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:31:00.142 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:31:00.159 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:31:00.159 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:31:00.195 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:31:00.195 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:31:00.204 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:31:00.246 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:31:00.245+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:31:00.323 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:31:00.321+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:31:00.347 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:31:00.348 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:31:00.356 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:31:00.424 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:31:00.423+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:31:00.459 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:31:00.459 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:31:00.468 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:31:00.524 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:31:00.523+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:31:00.611 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:31:00.612 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:31:00.620 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:31:00.625 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:31:00.624+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:31:00.723 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:31:00.723 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:31:00.726 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:31:00.725+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:31:00.731 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:31:00.828 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:31:00.827+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:31:00.874 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:31:00.874 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:31:00.882 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:31:00.928 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:31:00.928+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:31:00.985 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:31:00.985 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:31:00.993 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:31:01.030 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T05:31:01.036 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:31:01.035+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:31:01.131 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:31:01.130+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:31:01.152 INFO:teuthology.orchestra.run.smithi071.stdout:ERROR: (22) Invalid argument 2024-09-14T05:31:01.152 INFO:teuthology.orchestra.run.smithi071.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:31:01.163 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:31:01.232 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:31:01.231+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:31:01.271 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:31:01.271 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:31:01.280 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:31:01.332 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:31:01.332+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:31:01.382 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:31:01.382 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:31:01.390 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:31:01.434 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-14T05:31:01.433+0000 7f9616b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33604) UID: 0 2024-09-14T05:31:01.494 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:31:01.495 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:31:01.503 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:31:01.545 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:31:01.543+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:31:01.636 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:31:01.635+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:31:01.647 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:31:01.648 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:31:01.656 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:31:01.737 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-14T05:31:01.735+0000 7f3cd1cc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33607) UID: 0 2024-09-14T05:31:01.759 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:31:01.759 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:31:01.768 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:31:01.838 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:31:01.838+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:31:01.914 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:31:01.914 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:31:01.922 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:31:01.967 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:31:01.965+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:31:02.012 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is set 2024-09-14T05:31:02.031 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:31:02.031 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:31:02.040 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:31:02.038+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:31:02.041 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:31:02.141 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:31:02.140+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:31:02.171 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T05:31:02.188 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:31:02.188 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:31:02.197 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:31:02.242 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:31:02.241+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:31:02.292 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0: {} 2024-09-14T05:31:02.292 INFO:teuthology.orchestra.run.smithi071.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T05:31:02.300 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-14T05:31:02.300 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T05:31:02.309 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:31:02.309 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1: {} 2024-09-14T05:31:02.310 INFO:teuthology.orchestra.run.smithi071.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T05:31:02.324 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2: {} 2024-09-14T05:31:02.325 INFO:teuthology.orchestra.run.smithi071.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T05:31:02.340 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3: {} 2024-09-14T05:31:02.340 INFO:teuthology.orchestra.run.smithi071.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T05:31:02.342 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-14T05:31:02.342+0000 7fbc2f513640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33544) UID: 0 2024-09-14T05:31:02.357 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4: {} 2024-09-14T05:31:02.357 INFO:teuthology.orchestra.run.smithi071.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T05:31:02.374 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5: {} 2024-09-14T05:31:02.374 INFO:teuthology.orchestra.run.smithi071.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T05:31:02.391 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6: {} 2024-09-14T05:31:02.391 INFO:teuthology.orchestra.run.smithi071.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T05:31:02.407 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7: {} 2024-09-14T05:31:02.407 INFO:teuthology.orchestra.run.smithi071.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T05:31:02.415 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:31:02.415 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:31:02.415 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:31:02.415 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:31:02.424 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:31:02.459 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-14T05:31:02.459+0000 7f64703e4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33555) UID: 0 2024-09-14T05:31:02.526 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:31:02.526 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:31:02.526 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:31:02.527 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:31:02.527 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:31:02.535 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:31:02.544 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:31:02.543+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:31:02.640 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:31:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:31:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:31:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:31:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2131 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:31:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:31:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.863434885999993, 2024-09-14T05:31:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049526593000000001, 2024-09-14T05:31:02.640 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2131 2024-09-14T05:31:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:31:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:31:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774867+0000", 2024-09-14T05:31:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4430000000000002e-06 2024-09-14T05:31:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790357+0000", 2024-09-14T05:31:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015489466 2024-09-14T05:31:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790383+0000", 2024-09-14T05:31:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6247000000000001e-05 2024-09-14T05:31:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790394+0000", 2024-09-14T05:31:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0728e-05 2024-09-14T05:31:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790471+0000", 2024-09-14T05:31:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7539000000000004e-05 2024-09-14T05:31:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790639+0000", 2024-09-14T05:31:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016787700000000001 2024-09-14T05:31:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-14T05:31:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790998+0000", 2024-09-14T05:31:02.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035931299999999999 2024-09-14T05:31:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:31:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.823529+0000", 2024-09-14T05:31:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032530798 2024-09-14T05:31:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:31:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824324+0000", 2024-09-14T05:31:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00079442399999999995 2024-09-14T05:31:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824362+0000", 2024-09-14T05:31:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8398000000000001e-05 2024-09-14T05:31:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824391+0000", 2024-09-14T05:31:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.936e-05 2024-09-14T05:31:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3892 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7602176~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:31:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.281856887999993, 2024-09-14T05:31:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047082951999999997, 2024-09-14T05:31:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3892 2024-09-14T05:31:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.647 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:31:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:31:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:31:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0199999999999998e-07 2024-09-14T05:31:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356454+0000", 2024-09-14T05:31:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0414999999999999e-05 2024-09-14T05:31:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356463+0000", 2024-09-14T05:31:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6739999999999992e-06 2024-09-14T05:31:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356468+0000", 2024-09-14T05:31:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5680000000000001e-06 2024-09-14T05:31:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403143+0000", 2024-09-14T05:31:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046675051000000002 2024-09-14T05:31:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403206+0000", 2024-09-14T05:31:02.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3127999999999996e-05 2024-09-14T05:31:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403526+0000", 2024-09-14T05:31:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031981400000000002 2024-09-14T05:31:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3893 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8093696~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:31:02.651 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.281812051000003, 2024-09-14T05:31:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047516305000000002, 2024-09-14T05:31:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3893 2024-09-14T05:31:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:31:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:31:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:31:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6899999999999997e-07 2024-09-14T05:31:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356491+0000", 2024-09-14T05:31:02.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8109999999999999e-06 2024-09-14T05:31:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356498+0000", 2024-09-14T05:31:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5320000000000003e-06 2024-09-14T05:31:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356502+0000", 2024-09-14T05:31:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000002e-06 2024-09-14T05:31:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403579+0000", 2024-09-14T05:31:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047076706000000003 2024-09-14T05:31:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403664+0000", 2024-09-14T05:31:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5240000000000004e-05 2024-09-14T05:31:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404004+0000", 2024-09-14T05:31:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033974500000000002 2024-09-14T05:31:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3993 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 262144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:31:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.281790180000002, 2024-09-14T05:31:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048088476999999998, 2024-09-14T05:31:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3993 2024-09-14T05:31:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:31:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:31:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:31:02.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.46e-07 2024-09-14T05:31:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356512+0000", 2024-09-14T05:31:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.827e-06 2024-09-14T05:31:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356518+0000", 2024-09-14T05:31:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7780000000000004e-06 2024-09-14T05:31:02.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356522+0000", 2024-09-14T05:31:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.865e-06 2024-09-14T05:31:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404036+0000", 2024-09-14T05:31:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047513725999999999 2024-09-14T05:31:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404135+0000", 2024-09-14T05:31:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9352999999999996e-05 2024-09-14T05:31:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404598+0000", 2024-09-14T05:31:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046258200000000001 2024-09-14T05:31:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3994 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.660 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:31:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.281748168000007, 2024-09-14T05:31:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048274625000000002, 2024-09-14T05:31:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3994 2024-09-14T05:31:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:31:02.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:31:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:31:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3799999999999997e-07 2024-09-14T05:31:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356554+0000", 2024-09-14T05:31:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2259999999999999e-06 2024-09-14T05:31:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356560+0000", 2024-09-14T05:31:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-14T05:31:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356566+0000", 2024-09-14T05:31:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3909999999999997e-06 2024-09-14T05:31:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404632+0000", 2024-09-14T05:31:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048066154 2024-09-14T05:31:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404719+0000", 2024-09-14T05:31:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7440000000000003e-05 2024-09-14T05:31:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404826+0000", 2024-09-14T05:31:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106813 2024-09-14T05:31:02.664 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.665 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.665 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.665 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3995 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.665 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:31:02.665 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.281723029000005, 2024-09-14T05:31:02.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048828209999999997, 2024-09-14T05:31:02.665 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.665 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.665 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.665 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.665 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.666 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.666 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3995 2024-09-14T05:31:02.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.666 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:31:02.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:31:02.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:31:02.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6599999999999996e-07 2024-09-14T05:31:02.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356579+0000", 2024-09-14T05:31:02.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0719999999999998e-06 2024-09-14T05:31:02.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356585+0000", 2024-09-14T05:31:02.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.959e-06 2024-09-14T05:31:02.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356590+0000", 2024-09-14T05:31:02.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.464e-06 2024-09-14T05:31:02.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404875+0000", 2024-09-14T05:31:02.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048285649999999999 2024-09-14T05:31:02.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404985+0000", 2024-09-14T05:31:02.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109417 2024-09-14T05:31:02.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405405+0000", 2024-09-14T05:31:02.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042008200000000001 2024-09-14T05:31:02.669 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.669 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.669 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.669 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3996 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.669 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:31:02.669 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.281414146000003, 2024-09-14T05:31:02.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049061595, 2024-09-14T05:31:02.669 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.669 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.670 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.670 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.670 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.670 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.670 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3996 2024-09-14T05:31:02.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.670 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:31:02.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:31:02.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:31:02.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.61e-07 2024-09-14T05:31:02.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356892+0000", 2024-09-14T05:31:02.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9050000000000002e-06 2024-09-14T05:31:02.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356901+0000", 2024-09-14T05:31:02.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4149999999999999e-06 2024-09-14T05:31:02.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356907+0000", 2024-09-14T05:31:02.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0140000000000004e-06 2024-09-14T05:31:02.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405448+0000", 2024-09-14T05:31:02.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048541647 2024-09-14T05:31:02.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405542+0000", 2024-09-14T05:31:02.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3639000000000003e-05 2024-09-14T05:31:02.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405947+0000", 2024-09-14T05:31:02.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040521399999999999 2024-09-14T05:31:02.673 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.673 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.673 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.674 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3997 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2228224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.674 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:31:02.674 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.281200163999998, 2024-09-14T05:31:02.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049308774999999999, 2024-09-14T05:31:02.674 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.674 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.674 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.674 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.674 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.674 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.674 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3997 2024-09-14T05:31:02.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.674 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:31:02.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:31:02.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:31:02.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6499999999999999e-07 2024-09-14T05:31:02.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357104+0000", 2024-09-14T05:31:02.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4989999999999999e-06 2024-09-14T05:31:02.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357112+0000", 2024-09-14T05:31:02.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4279999999999996e-06 2024-09-14T05:31:02.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357117+0000", 2024-09-14T05:31:02.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6190000000000002e-06 2024-09-14T05:31:02.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405989+0000", 2024-09-14T05:31:02.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048872080999999998 2024-09-14T05:31:02.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406074+0000", 2024-09-14T05:31:02.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4676e-05 2024-09-14T05:31:02.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406408+0000", 2024-09-14T05:31:02.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033480700000000001 2024-09-14T05:31:02.678 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.678 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.678 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.678 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3998 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.678 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:31:02.678 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.280878735000002, 2024-09-14T05:31:02.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049462314, 2024-09-14T05:31:02.678 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.678 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.678 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.678 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.678 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.679 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.679 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3998 2024-09-14T05:31:02.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.679 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:31:02.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:31:02.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357422+0000", 2024-09-14T05:31:02.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3099999999999996e-07 2024-09-14T05:31:02.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357429+0000", 2024-09-14T05:31:02.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2880000000000004e-06 2024-09-14T05:31:02.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357439+0000", 2024-09-14T05:31:02.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.302e-06 2024-09-14T05:31:02.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357445+0000", 2024-09-14T05:31:02.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4269999999999996e-06 2024-09-14T05:31:02.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406448+0000", 2024-09-14T05:31:02.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049003096000000003 2024-09-14T05:31:02.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406537+0000", 2024-09-14T05:31:02.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9135999999999995e-05 2024-09-14T05:31:02.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406883+0000", 2024-09-14T05:31:02.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000346134 2024-09-14T05:31:02.682 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.682 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.682 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.683 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3999 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.683 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:31:02.683 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.280808378000003, 2024-09-14T05:31:02.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049797556, 2024-09-14T05:31:02.683 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.683 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.683 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.683 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.683 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.683 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.683 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3999 2024-09-14T05:31:02.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.683 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:31:02.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:31:02.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357492+0000", 2024-09-14T05:31:02.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8299999999999996e-07 2024-09-14T05:31:02.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357495+0000", 2024-09-14T05:31:02.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0450000000000001e-06 2024-09-14T05:31:02.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357503+0000", 2024-09-14T05:31:02.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9689999999999994e-06 2024-09-14T05:31:02.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357510+0000", 2024-09-14T05:31:02.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-14T05:31:02.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406914+0000", 2024-09-14T05:31:02.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049404293000000002 2024-09-14T05:31:02.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406965+0000", 2024-09-14T05:31:02.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0470999999999998e-05 2024-09-14T05:31:02.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407289+0000", 2024-09-14T05:31:02.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032434899999999998 2024-09-14T05:31:02.687 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.687 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.687 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.687 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4000 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3702784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.687 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:31:02.687 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.280462033000006, 2024-09-14T05:31:02.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049936728999999999, 2024-09-14T05:31:02.687 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.687 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.687 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.687 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.687 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.687 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.687 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4000 2024-09-14T05:31:02.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.688 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:31:02.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:31:02.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357839+0000", 2024-09-14T05:31:02.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-14T05:31:02.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357842+0000", 2024-09-14T05:31:02.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4850000000000001e-06 2024-09-14T05:31:02.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357851+0000", 2024-09-14T05:31:02.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1819999999999995e-06 2024-09-14T05:31:02.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357859+0000", 2024-09-14T05:31:02.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8129999999999996e-06 2024-09-14T05:31:02.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407337+0000", 2024-09-14T05:31:02.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049478084999999998 2024-09-14T05:31:02.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407442+0000", 2024-09-14T05:31:02.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000105302 2024-09-14T05:31:02.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407774+0000", 2024-09-14T05:31:02.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033200100000000001 2024-09-14T05:31:02.691 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.691 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.691 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.691 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4001 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.691 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:31:02.691 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.280366677999993, 2024-09-14T05:31:02.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050304205999999997, 2024-09-14T05:31:02.692 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.692 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.692 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.692 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.692 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.692 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.692 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4001 2024-09-14T05:31:02.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.692 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:31:02.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:31:02.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357934+0000", 2024-09-14T05:31:02.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000004e-07 2024-09-14T05:31:02.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357937+0000", 2024-09-14T05:31:02.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0869999999999998e-06 2024-09-14T05:31:02.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357946+0000", 2024-09-14T05:31:02.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1439999999999992e-06 2024-09-14T05:31:02.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357952+0000", 2024-09-14T05:31:02.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.609e-06 2024-09-14T05:31:02.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407820+0000", 2024-09-14T05:31:02.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049868580000000003 2024-09-14T05:31:02.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407909+0000", 2024-09-14T05:31:02.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8887000000000005e-05 2024-09-14T05:31:02.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408237+0000", 2024-09-14T05:31:02.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032824599999999997 2024-09-14T05:31:02.695 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.695 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.695 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.695 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4002 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.696 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:31:02.696 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.280150082999995, 2024-09-14T05:31:02.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050539529999999999, 2024-09-14T05:31:02.696 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.696 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.696 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.696 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.696 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.696 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.696 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4002 2024-09-14T05:31:02.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.696 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:31:02.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:31:02.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:31:02.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5099999999999999e-07 2024-09-14T05:31:02.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358154+0000", 2024-09-14T05:31:02.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6799999999999999e-06 2024-09-14T05:31:02.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358165+0000", 2024-09-14T05:31:02.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0899e-05 2024-09-14T05:31:02.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358172+0000", 2024-09-14T05:31:02.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8689999999999998e-06 2024-09-14T05:31:02.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408281+0000", 2024-09-14T05:31:02.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050108699 2024-09-14T05:31:02.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408369+0000", 2024-09-14T05:31:02.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8542999999999996e-05 2024-09-14T05:31:02.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408689+0000", 2024-09-14T05:31:02.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032008900000000002 2024-09-14T05:31:02.700 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.700 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.700 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.700 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4003 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5177344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.700 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:31:02.700 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.280110248, 2024-09-14T05:31:02.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050868759, 2024-09-14T05:31:02.700 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.701 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.701 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.701 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.701 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.701 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.701 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4003 2024-09-14T05:31:02.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.701 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:31:02.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:31:02.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:31:02.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-07 2024-09-14T05:31:02.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358193+0000", 2024-09-14T05:31:02.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.588e-06 2024-09-14T05:31:02.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358204+0000", 2024-09-14T05:31:02.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1525999999999999e-05 2024-09-14T05:31:02.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358213+0000", 2024-09-14T05:31:02.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5010000000000004e-06 2024-09-14T05:31:02.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408719+0000", 2024-09-14T05:31:02.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050506708999999997 2024-09-14T05:31:02.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408766+0000", 2024-09-14T05:31:02.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6579999999999998e-05 2024-09-14T05:31:02.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409058+0000", 2024-09-14T05:31:02.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000292365 2024-09-14T05:31:02.704 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.704 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.705 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.705 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4004 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5668864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.705 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:31:02.705 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.279687405000004, 2024-09-14T05:31:02.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050839848, 2024-09-14T05:31:02.705 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.705 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.705 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.705 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.705 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.705 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.705 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4004 2024-09-14T05:31:02.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.706 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:31:02.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:31:02.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358613+0000", 2024-09-14T05:31:02.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:31:02.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358618+0000", 2024-09-14T05:31:02.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6009999999999997e-06 2024-09-14T05:31:02.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358630+0000", 2024-09-14T05:31:02.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2122e-05 2024-09-14T05:31:02.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358638+0000", 2024-09-14T05:31:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3969999999999994e-06 2024-09-14T05:31:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409088+0000", 2024-09-14T05:31:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050449565000000002 2024-09-14T05:31:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409153+0000", 2024-09-14T05:31:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4728999999999996e-05 2024-09-14T05:31:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409452+0000", 2024-09-14T05:31:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029958900000000001 2024-09-14T05:31:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4005 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6160384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:31:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.279470470999996, 2024-09-14T05:31:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051054752000000002, 2024-09-14T05:31:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4005 2024-09-14T05:31:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:31:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:31:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358830+0000", 2024-09-14T05:31:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-14T05:31:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358833+0000", 2024-09-14T05:31:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9529999999999999e-06 2024-09-14T05:31:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358843+0000", 2024-09-14T05:31:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8069999999999994e-06 2024-09-14T05:31:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358849+0000", 2024-09-14T05:31:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1040000000000003e-06 2024-09-14T05:31:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409487+0000", 2024-09-14T05:31:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050638626999999999 2024-09-14T05:31:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409535+0000", 2024-09-14T05:31:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7525000000000003e-05 2024-09-14T05:31:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409884+0000", 2024-09-14T05:31:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034914800000000003 2024-09-14T05:31:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4006 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:31:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.279433358999995, 2024-09-14T05:31:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051425323000000002, 2024-09-14T05:31:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4006 2024-09-14T05:31:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:31:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:31:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358867+0000", 2024-09-14T05:31:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2900000000000004e-07 2024-09-14T05:31:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358869+0000", 2024-09-14T05:31:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4210000000000002e-06 2024-09-14T05:31:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358876+0000", 2024-09-14T05:31:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4980000000000003e-06 2024-09-14T05:31:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358881+0000", 2024-09-14T05:31:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3090000000000003e-06 2024-09-14T05:31:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409915+0000", 2024-09-14T05:31:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051033348999999999 2024-09-14T05:31:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409964+0000", 2024-09-14T05:31:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9858000000000003e-05 2024-09-14T05:31:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410292+0000", 2024-09-14T05:31:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032735899999999999 2024-09-14T05:31:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4007 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:31:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.279410765999998, 2024-09-14T05:31:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05184076, 2024-09-14T05:31:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4007 2024-09-14T05:31:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:31:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:31:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:31:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:31:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358891+0000", 2024-09-14T05:31:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9819999999999998e-06 2024-09-14T05:31:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358897+0000", 2024-09-14T05:31:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9390000000000002e-06 2024-09-14T05:31:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358902+0000", 2024-09-14T05:31:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6770000000000003e-06 2024-09-14T05:31:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410335+0000", 2024-09-14T05:31:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051432991999999997 2024-09-14T05:31:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410416+0000", 2024-09-14T05:31:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1316999999999994e-05 2024-09-14T05:31:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410730+0000", 2024-09-14T05:31:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000313419 2024-09-14T05:31:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4008 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7634944~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:31:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.279386404999997, 2024-09-14T05:31:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052208082000000003, 2024-09-14T05:31:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4008 2024-09-14T05:31:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:31:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:31:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358914+0000", 2024-09-14T05:31:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6100000000000001e-07 2024-09-14T05:31:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358916+0000", 2024-09-14T05:31:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.902e-06 2024-09-14T05:31:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358923+0000", 2024-09-14T05:31:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4780000000000002e-06 2024-09-14T05:31:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358929+0000", 2024-09-14T05:31:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0970000000000002e-06 2024-09-14T05:31:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410761+0000", 2024-09-14T05:31:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051831436000000002 2024-09-14T05:31:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410829+0000", 2024-09-14T05:31:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7874999999999996e-05 2024-09-14T05:31:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411121+0000", 2024-09-14T05:31:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029283299999999999 2024-09-14T05:31:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4009 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8126464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:31:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 91.279239180000005, 2024-09-14T05:31:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052518169000000003, 2024-09-14T05:31:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4009 2024-09-14T05:31:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:31:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:31:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:31:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0200000000000002e-07 2024-09-14T05:31:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359066+0000", 2024-09-14T05:31:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7899999999999999e-06 2024-09-14T05:31:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359074+0000", 2024-09-14T05:31:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2579999999999995e-06 2024-09-14T05:31:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359082+0000", 2024-09-14T05:31:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.464e-06 2024-09-14T05:31:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411160+0000", 2024-09-14T05:31:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052078078 2024-09-14T05:31:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411244+0000", 2024-09-14T05:31:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4041000000000004e-05 2024-09-14T05:31:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411579+0000", 2024-09-14T05:31:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033493599999999998 2024-09-14T05:31:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:02.732 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:02.732 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:31:02.733 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-14T05:31:02.732+0000 7f77d4c86640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33600) UID: 0 2024-09-14T05:31:02.733 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T05:31:02.834 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-14T05:31:02.833+0000 7f312e7c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33559) UID: 0 2024-09-14T05:31:02.836 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:02.836 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:31:02.837 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:31:02.837 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:02.845 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T05:31:02.935 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-14T05:31:02.934+0000 7f9c551bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33540) UID: 0 2024-09-14T05:31:02.988 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:02.988 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:31:02.988 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:31:02.988 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:31:02.989 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:02.997 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T05:31:03.036 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-14T05:31:03.035+0000 7fca42ce7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33602) UID: 0 2024-09-14T05:31:03.103 INFO:teuthology.orchestra.run.smithi071.stdout:All operations completed A-OK! 2024-09-14T05:31:03.113 DEBUG:teuthology.parallel:result is None 2024-09-14T05:31:03.113 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-14T05:31:03.123 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-14T05:31:03.143 INFO:tasks.thrashosds:joining thrashosds 2024-09-14T05:31:03.148 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:03.148 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:31:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:31:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:31:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74187 3.f 3:f707b2d9:::rbd_data.112d46b1ca94.0000000000000061:11 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:31:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.579260175000002, 2024-09-14T05:31:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072367828999999995, 2024-09-14T05:31:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74187 2024-09-14T05:31:03.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:31:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:31:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568164+0000", 2024-09-14T05:31:03.150 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0900000000000002e-07 2024-09-14T05:31:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568166+0000", 2024-09-14T05:31:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.683e-06 2024-09-14T05:31:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568172+0000", 2024-09-14T05:31:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0639999999999999e-06 2024-09-14T05:31:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.151 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568177+0000", 2024-09-14T05:31:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.967e-06 2024-09-14T05:31:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640193+0000", 2024-09-14T05:31:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072016284 2024-09-14T05:31:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640282+0000", 2024-09-14T05:31:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9011999999999994e-05 2024-09-14T05:31:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640532+0000", 2024-09-14T05:31:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00024940999999999999 2024-09-14T05:31:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74188 3.f 3:f707b2d9:::rbd_data.112d46b1ca94.0000000000000061:11 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.153 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:31:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.579240802000001, 2024-09-14T05:31:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072600425999999996, 2024-09-14T05:31:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74188 2024-09-14T05:31:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:31:03.154 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568183+0000", 2024-09-14T05:31:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568184+0000", 2024-09-14T05:31:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8700000000000001e-07 2024-09-14T05:31:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.155 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568200+0000", 2024-09-14T05:31:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.6603e-05 2024-09-14T05:31:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568208+0000", 2024-09-14T05:31:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4800000000000004e-06 2024-09-14T05:31:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568213+0000", 2024-09-14T05:31:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6860000000000003e-06 2024-09-14T05:31:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640563+0000", 2024-09-14T05:31:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072349438000000002 2024-09-14T05:31:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640607+0000", 2024-09-14T05:31:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4014999999999998e-05 2024-09-14T05:31:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640784+0000", 2024-09-14T05:31:03.157 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00017681700000000001 2024-09-14T05:31:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74194 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:31:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.579202670000001, 2024-09-14T05:31:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072795199000000005, 2024-09-14T05:31:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.158 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74194 2024-09-14T05:31:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:31:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568221+0000", 2024-09-14T05:31:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568222+0000", 2024-09-14T05:31:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1300000000000001e-07 2024-09-14T05:31:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568224+0000", 2024-09-14T05:31:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0660000000000002e-06 2024-09-14T05:31:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568230+0000", 2024-09-14T05:31:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4470000000000003e-06 2024-09-14T05:31:03.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568235+0000", 2024-09-14T05:31:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1220000000000003e-06 2024-09-14T05:31:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640824+0000", 2024-09-14T05:31:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072588707000000002 2024-09-14T05:31:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.640915+0000", 2024-09-14T05:31:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1008999999999995e-05 2024-09-14T05:31:03.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641017+0000", 2024-09-14T05:31:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010143499999999999 2024-09-14T05:31:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74208 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2693120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:31:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.579182094000004, 2024-09-14T05:31:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072905555999999996, 2024-09-14T05:31:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.162 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74208 2024-09-14T05:31:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:31:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:31:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568242+0000", 2024-09-14T05:31:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9799999999999999e-07 2024-09-14T05:31:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568244+0000", 2024-09-14T05:31:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0339999999999999e-06 2024-09-14T05:31:03.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568251+0000", 2024-09-14T05:31:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4060000000000002e-06 2024-09-14T05:31:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568255+0000", 2024-09-14T05:31:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6589999999999998e-06 2024-09-14T05:31:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641048+0000", 2024-09-14T05:31:03.165 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072792647000000002 2024-09-14T05:31:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641086+0000", 2024-09-14T05:31:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8031999999999998e-05 2024-09-14T05:31:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641147+0000", 2024-09-14T05:31:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1379999999999998e-05 2024-09-14T05:31:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.166 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74236 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2662400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:31:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.578548431000002, 2024-09-14T05:31:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072517776000000006, 2024-09-14T05:31:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74236 2024-09-14T05:31:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:31:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568876+0000", 2024-09-14T05:31:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568881+0000", 2024-09-14T05:31:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2340000000000002e-06 2024-09-14T05:31:03.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568886+0000", 2024-09-14T05:31:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9450000000000001e-06 2024-09-14T05:31:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568898+0000", 2024-09-14T05:31:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2150000000000001e-05 2024-09-14T05:31:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568907+0000", 2024-09-14T05:31:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9239999999999996e-06 2024-09-14T05:31:03.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641185+0000", 2024-09-14T05:31:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.07227807 2024-09-14T05:31:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641265+0000", 2024-09-14T05:31:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0575999999999996e-05 2024-09-14T05:31:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641393+0000", 2024-09-14T05:31:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00012787700000000001 2024-09-14T05:31:03.170 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74239 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2695168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:31:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.578504038000005, 2024-09-14T05:31:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072593816000000005, 2024-09-14T05:31:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.171 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74239 2024-09-14T05:31:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:31:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:31:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.172 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568920+0000", 2024-09-14T05:31:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4300000000000003e-07 2024-09-14T05:31:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568923+0000", 2024-09-14T05:31:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2019999999999998e-06 2024-09-14T05:31:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568933+0000", 2024-09-14T05:31:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0376e-05 2024-09-14T05:31:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.568938+0000", 2024-09-14T05:31:03.173 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2850000000000002e-06 2024-09-14T05:31:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641424+0000", 2024-09-14T05:31:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072485230999999997 2024-09-14T05:31:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641464+0000", 2024-09-14T05:31:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0574000000000002e-05 2024-09-14T05:31:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641514+0000", 2024-09-14T05:31:03.174 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9604999999999997e-05 2024-09-14T05:31:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74252 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2664448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:31:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.578324124000005, 2024-09-14T05:31:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072631237000000001, 2024-09-14T05:31:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.175 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74252 2024-09-14T05:31:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:31:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569100+0000", 2024-09-14T05:31:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569101+0000", 2024-09-14T05:31:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8700000000000005e-07 2024-09-14T05:31:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569104+0000", 2024-09-14T05:31:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.163e-06 2024-09-14T05:31:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569115+0000", 2024-09-14T05:31:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1335000000000001e-05 2024-09-14T05:31:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569121+0000", 2024-09-14T05:31:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0800000000000002e-06 2024-09-14T05:31:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641552+0000", 2024-09-14T05:31:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072430743000000006 2024-09-14T05:31:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641631+0000", 2024-09-14T05:31:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9388999999999995e-05 2024-09-14T05:31:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641731+0000", 2024-09-14T05:31:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9840000000000006e-05 2024-09-14T05:31:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74256 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2697216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:31:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.578293067999994, 2024-09-14T05:31:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072721090000000002, 2024-09-14T05:31:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.179 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74256 2024-09-14T05:31:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:31:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:31:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569131+0000", 2024-09-14T05:31:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0299999999999999e-07 2024-09-14T05:31:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569133+0000", 2024-09-14T05:31:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.046e-06 2024-09-14T05:31:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569139+0000", 2024-09-14T05:31:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0279999999999997e-06 2024-09-14T05:31:03.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569144+0000", 2024-09-14T05:31:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6929999999999998e-06 2024-09-14T05:31:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641762+0000", 2024-09-14T05:31:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072617904999999996 2024-09-14T05:31:03.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641801+0000", 2024-09-14T05:31:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8902999999999997e-05 2024-09-14T05:31:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.641852+0000", 2024-09-14T05:31:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1011999999999999e-05 2024-09-14T05:31:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74285 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2672640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:31:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.577447632000002, 2024-09-14T05:31:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072422069000000006, 2024-09-14T05:31:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74285 2024-09-14T05:31:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:31:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569976+0000", 2024-09-14T05:31:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569977+0000", 2024-09-14T05:31:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3099999999999998e-07 2024-09-14T05:31:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569978+0000", 2024-09-14T05:31:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.598e-06 2024-09-14T05:31:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569984+0000", 2024-09-14T05:31:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4689999999999996e-06 2024-09-14T05:31:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.569989+0000", 2024-09-14T05:31:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8409999999999999e-06 2024-09-14T05:31:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642229+0000", 2024-09-14T05:31:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072240046000000002 2024-09-14T05:31:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642306+0000", 2024-09-14T05:31:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7347000000000003e-05 2024-09-14T05:31:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642398+0000", 2024-09-14T05:31:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2336999999999994e-05 2024-09-14T05:31:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74292 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2705408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:31:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.577192924000002, 2024-09-14T05:31:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072361018999999999, 2024-09-14T05:31:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74292 2024-09-14T05:31:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:31:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570231+0000", 2024-09-14T05:31:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570232+0000", 2024-09-14T05:31:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.06e-07 2024-09-14T05:31:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570235+0000", 2024-09-14T05:31:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3189999999999999e-06 2024-09-14T05:31:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570248+0000", 2024-09-14T05:31:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3006e-05 2024-09-14T05:31:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570254+0000", 2024-09-14T05:31:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.908e-06 2024-09-14T05:31:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642433+0000", 2024-09-14T05:31:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072179192000000003 2024-09-14T05:31:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642500+0000", 2024-09-14T05:31:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7080000000000001e-05 2024-09-14T05:31:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642592+0000", 2024-09-14T05:31:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1807999999999999e-05 2024-09-14T05:31:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74304 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2707456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:31:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.576836258, 2024-09-14T05:31:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072444960000000003, 2024-09-14T05:31:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74304 2024-09-14T05:31:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:31:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:31:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570588+0000", 2024-09-14T05:31:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9800000000000004e-07 2024-09-14T05:31:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570590+0000", 2024-09-14T05:31:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0810000000000001e-06 2024-09-14T05:31:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570596+0000", 2024-09-14T05:31:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9789999999999998e-06 2024-09-14T05:31:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.570601+0000", 2024-09-14T05:31:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6750000000000001e-06 2024-09-14T05:31:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642835+0000", 2024-09-14T05:31:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.07223359 2024-09-14T05:31:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.642917+0000", 2024-09-14T05:31:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2013000000000002e-05 2024-09-14T05:31:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643033+0000", 2024-09-14T05:31:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000116124 2024-09-14T05:31:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74321 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2709504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:31:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.576354268000003, 2024-09-14T05:31:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072350611999999995, 2024-09-14T05:31:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74321 2024-09-14T05:31:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:31:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:31:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571070+0000", 2024-09-14T05:31:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3e-07 2024-09-14T05:31:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571073+0000", 2024-09-14T05:31:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6879999999999999e-06 2024-09-14T05:31:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571080+0000", 2024-09-14T05:31:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0319999999999998e-06 2024-09-14T05:31:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571085+0000", 2024-09-14T05:31:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2109999999999997e-06 2024-09-14T05:31:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643309+0000", 2024-09-14T05:31:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072223812999999998 2024-09-14T05:31:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643351+0000", 2024-09-14T05:31:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2239e-05 2024-09-14T05:31:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643420+0000", 2024-09-14T05:31:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8998999999999994e-05 2024-09-14T05:31:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74332 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2678784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:31:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.576031104999998, 2024-09-14T05:31:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072259062999999998, 2024-09-14T05:31:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74332 2024-09-14T05:31:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:31:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571393+0000", 2024-09-14T05:31:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571394+0000", 2024-09-14T05:31:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:31:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571397+0000", 2024-09-14T05:31:03.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8469999999999998e-06 2024-09-14T05:31:03.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571411+0000", 2024-09-14T05:31:03.203 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.34e-05 2024-09-14T05:31:03.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.203 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.203 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571418+0000", 2024-09-14T05:31:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9149999999999999e-06 2024-09-14T05:31:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643458+0000", 2024-09-14T05:31:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072039881 2024-09-14T05:31:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643539+0000", 2024-09-14T05:31:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1650000000000006e-05 2024-09-14T05:31:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643652+0000", 2024-09-14T05:31:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000112593 2024-09-14T05:31:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74336 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2711552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:31:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.575994655000002, 2024-09-14T05:31:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072354038999999995, 2024-09-14T05:31:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.206 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.206 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.206 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.206 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74336 2024-09-14T05:31:03.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.206 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.206 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.206 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:31:03.206 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.206 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571429+0000", 2024-09-14T05:31:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571430+0000", 2024-09-14T05:31:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8999999999999996e-07 2024-09-14T05:31:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571432+0000", 2024-09-14T05:31:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4159999999999998e-06 2024-09-14T05:31:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571439+0000", 2024-09-14T05:31:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.657e-06 2024-09-14T05:31:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571444+0000", 2024-09-14T05:31:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1329999999999998e-06 2024-09-14T05:31:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643690+0000", 2024-09-14T05:31:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072246088 2024-09-14T05:31:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643731+0000", 2024-09-14T05:31:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0311000000000001e-05 2024-09-14T05:31:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643783+0000", 2024-09-14T05:31:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2843999999999998e-05 2024-09-14T05:31:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.210 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74349 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.210 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:31:03.210 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.575973645999994, 2024-09-14T05:31:03.210 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072554297000000004, 2024-09-14T05:31:03.210 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.210 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.210 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.210 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.210 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.210 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.210 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74349 2024-09-14T05:31:03.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:31:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571450+0000", 2024-09-14T05:31:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571451+0000", 2024-09-14T05:31:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.41e-07 2024-09-14T05:31:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571453+0000", 2024-09-14T05:31:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8360000000000001e-06 2024-09-14T05:31:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571475+0000", 2024-09-14T05:31:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2452000000000001e-05 2024-09-14T05:31:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571480+0000", 2024-09-14T05:31:03.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4430000000000002e-06 2024-09-14T05:31:03.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643821+0000", 2024-09-14T05:31:03.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072340347999999999 2024-09-14T05:31:03.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.213 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.213 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.643904+0000", 2024-09-14T05:31:03.213 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3534000000000004e-05 2024-09-14T05:31:03.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644005+0000", 2024-09-14T05:31:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000100343 2024-09-14T05:31:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74357 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2713600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:31:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.575929613, 2024-09-14T05:31:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072651144000000001, 2024-09-14T05:31:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.215 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.215 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.215 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.215 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.215 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74357 2024-09-14T05:31:03.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.215 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.215 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.215 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:31:03.215 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571494+0000", 2024-09-14T05:31:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571495+0000", 2024-09-14T05:31:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8899999999999999e-07 2024-09-14T05:31:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571498+0000", 2024-09-14T05:31:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5840000000000002e-06 2024-09-14T05:31:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571504+0000", 2024-09-14T05:31:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6549999999999998e-06 2024-09-14T05:31:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571509+0000", 2024-09-14T05:31:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6500000000000004e-06 2024-09-14T05:31:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644043+0000", 2024-09-14T05:31:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072533832000000006 2024-09-14T05:31:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644090+0000", 2024-09-14T05:31:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7426000000000003e-05 2024-09-14T05:31:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644146+0000", 2024-09-14T05:31:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5408000000000002e-05 2024-09-14T05:31:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74364 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2682880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:31:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.575592841000002, 2024-09-14T05:31:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072531227000000004, 2024-09-14T05:31:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74364 2024-09-14T05:31:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:31:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571831+0000", 2024-09-14T05:31:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571832+0000", 2024-09-14T05:31:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4e-07 2024-09-14T05:31:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571838+0000", 2024-09-14T05:31:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0519999999999998e-06 2024-09-14T05:31:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571865+0000", 2024-09-14T05:31:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7109000000000001e-05 2024-09-14T05:31:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.571883+0000", 2024-09-14T05:31:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7428e-05 2024-09-14T05:31:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644182+0000", 2024-09-14T05:31:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072299551000000004 2024-09-14T05:31:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644260+0000", 2024-09-14T05:31:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7551000000000003e-05 2024-09-14T05:31:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644362+0000", 2024-09-14T05:31:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00010259600000000001 2024-09-14T05:31:03.223 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.223 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.223 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.223 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74367 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2715648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.223 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:31:03.223 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.575229534000002, 2024-09-14T05:31:03.223 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072368259000000004, 2024-09-14T05:31:03.223 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.223 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.223 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.223 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74367 2024-09-14T05:31:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:31:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572194+0000", 2024-09-14T05:31:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572195+0000", 2024-09-14T05:31:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.78e-07 2024-09-14T05:31:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572200+0000", 2024-09-14T05:31:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2520000000000001e-06 2024-09-14T05:31:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572213+0000", 2024-09-14T05:31:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3618000000000001e-05 2024-09-14T05:31:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572226+0000", 2024-09-14T05:31:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3040999999999999e-05 2024-09-14T05:31:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644409+0000", 2024-09-14T05:31:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072182982000000007 2024-09-14T05:31:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644489+0000", 2024-09-14T05:31:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9429000000000001e-05 2024-09-14T05:31:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644563+0000", 2024-09-14T05:31:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4059000000000004e-05 2024-09-14T05:31:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74380 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2684928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:31:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.575095755999996, 2024-09-14T05:31:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072442180999999994, 2024-09-14T05:31:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74380 2024-09-14T05:31:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:31:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572328+0000", 2024-09-14T05:31:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572329+0000", 2024-09-14T05:31:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3300000000000001e-07 2024-09-14T05:31:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572333+0000", 2024-09-14T05:31:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8e-06 2024-09-14T05:31:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572346+0000", 2024-09-14T05:31:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.307e-05 2024-09-14T05:31:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572353+0000", 2024-09-14T05:31:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.9e-06 2024-09-14T05:31:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644597+0000", 2024-09-14T05:31:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072244770999999999 2024-09-14T05:31:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644672+0000", 2024-09-14T05:31:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.4945999999999997e-05 2024-09-14T05:31:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644770+0000", 2024-09-14T05:31:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7961000000000006e-05 2024-09-14T05:31:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74383 3.f 3:f36fb223:::rbd_data.112d641bd628.0000000000000004:10 [read 2717696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:31:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 73.574839761999996, 2024-09-14T05:31:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072397258000000006, 2024-09-14T05:31:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74383 2024-09-14T05:31:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:31:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572584+0000", 2024-09-14T05:31:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572585+0000", 2024-09-14T05:31:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:31:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572589+0000", 2024-09-14T05:31:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:31:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572602+0000", 2024-09-14T05:31:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2917e-05 2024-09-14T05:31:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.572610+0000", 2024-09-14T05:31:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.943e-06 2024-09-14T05:31:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644791+0000", 2024-09-14T05:31:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.072181113000000005 2024-09-14T05:31:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644861+0000", 2024-09-14T05:31:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0650999999999998e-05 2024-09-14T05:31:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.644981+0000", 2024-09-14T05:31:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000120185 2024-09-14T05:31:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.236 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:03.237 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T05:31:03.379 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:03.380 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:31:03.380 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:31:03.380 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:03.388 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T05:31:03.490 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:03.491 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:31:03.491 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:31:03.491 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:31:03.491 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:03.499 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T05:31:03.603 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:03.603 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:31:03.604 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:31:03.604 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:31:03.604 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.604 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4206.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T05:31:03.604 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:31:03.604 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 115.825677008, 2024-09-14T05:31:03.604 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.24562984099999999, 2024-09-14T05:31:03.604 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.604 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.604 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:03.604 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.604 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4206", 2024-09-14T05:31:03.604 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1863085609", 2024-09-14T05:31:03.605 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 1 2024-09-14T05:31:03.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.605 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.605 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.605 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:31:03.605 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.605 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.605 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776563+0000", 2024-09-14T05:31:03.605 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.605 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776565+0000", 2024-09-14T05:31:03.606 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.4780000000000001e-06 2024-09-14T05:31:03.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.606 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776571+0000", 2024-09-14T05:31:03.606 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7669999999999998e-06 2024-09-14T05:31:03.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.606 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776580+0000", 2024-09-14T05:31:03.606 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9830000000000002e-06 2024-09-14T05:31:03.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.606 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.606 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776589+0000", 2024-09-14T05:31:03.607 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7579999999999998e-06 2024-09-14T05:31:03.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.607 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.607 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776662+0000", 2024-09-14T05:31:03.607 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.3332999999999997e-05 2024-09-14T05:31:03.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.607 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:31:03.607 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:07.776670+0000", 2024-09-14T05:31:03.607 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9850000000000006e-06 2024-09-14T05:31:03.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.607 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.607 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.020867+0000", 2024-09-14T05:31:03.607 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.244196265 2024-09-14T05:31:03.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.608 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.608 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.608 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021022+0000", 2024-09-14T05:31:03.608 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000155699 2024-09-14T05:31:03.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.608 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.608 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 5", 2024-09-14T05:31:03.608 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021169+0000", 2024-09-14T05:31:03.608 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014661700000000001 2024-09-14T05:31:03.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.608 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.608 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:31:03.608 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.021891+0000", 2024-09-14T05:31:03.608 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00072146700000000005 2024-09-14T05:31:03.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:31:03.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022124+0000", 2024-09-14T05:31:03.609 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00023354199999999999 2024-09-14T05:31:03.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:31:03.609 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022164+0000", 2024-09-14T05:31:03.609 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9897000000000001e-05 2024-09-14T05:31:03.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.609 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.610 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:08.022193+0000", 2024-09-14T05:31:03.610 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9050000000000001e-05 2024-09-14T05:31:03.610 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.610 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.610 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.610 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.610 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_repop(client.4397.0:1989 3.1f e30/24 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head v 30'29, mlcod=30'27)", 2024-09-14T05:31:03.610 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:31:03.610 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 93.008906021000001, 2024-09-14T05:31:03.610 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.064242492999999998, 2024-09-14T05:31:03.610 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.610 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.610 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:03.610 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.611 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.611 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:31:03.611 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.611 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.611 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593334+0000", 2024-09-14T05:31:03.611 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.611 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.611 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.593336+0000", 2024-09-14T05:31:03.611 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3909999999999999e-06 2024-09-14T05:31:03.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.612 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.612 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616632+0000", 2024-09-14T05:31:03.612 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.023296598000000002 2024-09-14T05:31:03.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.612 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.612 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616657+0000", 2024-09-14T05:31:03.612 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5228e-05 2024-09-14T05:31:03.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.612 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.612 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616684+0000", 2024-09-14T05:31:03.612 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6441999999999999e-05 2024-09-14T05:31:03.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.613 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.613 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.613 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616755+0000", 2024-09-14T05:31:03.613 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1452000000000006e-05 2024-09-14T05:31:03.613 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.613 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.613 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.613 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.616780+0000", 2024-09-14T05:31:03.613 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4436000000000001e-05 2024-09-14T05:31:03.613 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.613 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.613 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:31:03.613 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657527+0000", 2024-09-14T05:31:03.613 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.040747323000000002 2024-09-14T05:31:03.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.614 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.614 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.614 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:30.657577+0000", 2024-09-14T05:31:03.614 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9623000000000002e-05 2024-09-14T05:31:03.614 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.614 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.614 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.614 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.614 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124262 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4481024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:31:03.614 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867845+0000", 2024-09-14T05:31:03.614 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.734394821999999, 2024-09-14T05:31:03.614 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048759513999999997, 2024-09-14T05:31:03.614 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.614 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.615 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.615 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.615 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.615 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.615 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124262 2024-09-14T05:31:03.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.615 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.615 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867845+0000", 2024-09-14T05:31:03.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.615 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.615 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867845+0000", 2024-09-14T05:31:03.615 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.616 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.616 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867846+0000", 2024-09-14T05:31:03.616 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2e-07 2024-09-14T05:31:03.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.616 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.616 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867848+0000", 2024-09-14T05:31:03.616 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0339999999999999e-06 2024-09-14T05:31:03.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.616 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.617 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867854+0000", 2024-09-14T05:31:03.617 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.7350000000000001e-06 2024-09-14T05:31:03.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.617 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.617 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867859+0000", 2024-09-14T05:31:03.617 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9690000000000002e-06 2024-09-14T05:31:03.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.617 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.617 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916436+0000", 2024-09-14T05:31:03.617 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048577694999999997 2024-09-14T05:31:03.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.618 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.618 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916515+0000", 2024-09-14T05:31:03.618 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8982999999999999e-05 2024-09-14T05:31:03.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.618 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.618 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916605+0000", 2024-09-14T05:31:03.618 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9578000000000002e-05 2024-09-14T05:31:03.618 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.618 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.618 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.618 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124263 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4587520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:31:03.619 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:31:03.619 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.734375194999998, 2024-09-14T05:31:03.619 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048866746000000003, 2024-09-14T05:31:03.619 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.619 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.619 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.619 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.619 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.619 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.619 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124263 2024-09-14T05:31:03.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.619 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.619 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.619 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:31:03.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.620 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.620 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:31:03.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.620 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.620 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867865+0000", 2024-09-14T05:31:03.620 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2199999999999999e-07 2024-09-14T05:31:03.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.620 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.620 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867867+0000", 2024-09-14T05:31:03.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.091e-06 2024-09-14T05:31:03.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867873+0000", 2024-09-14T05:31:03.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8869999999999997e-06 2024-09-14T05:31:03.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.621 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.621 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867884+0000", 2024-09-14T05:31:03.621 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0784999999999999e-05 2024-09-14T05:31:03.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916636+0000", 2024-09-14T05:31:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048752141999999998 2024-09-14T05:31:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916680+0000", 2024-09-14T05:31:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3621000000000001e-05 2024-09-14T05:31:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916732+0000", 2024-09-14T05:31:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1798000000000001e-05 2024-09-14T05:31:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.623 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.623 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.623 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124264 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4694016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:31:03.623 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867891+0000", 2024-09-14T05:31:03.623 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.734348840000003, 2024-09-14T05:31:03.623 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049046027999999998, 2024-09-14T05:31:03.623 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.623 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.623 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.623 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.623 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.623 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.623 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124264 2024-09-14T05:31:03.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.624 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.624 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.624 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867891+0000", 2024-09-14T05:31:03.624 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.624 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.624 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867891+0000", 2024-09-14T05:31:03.624 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.624 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.624 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867892+0000", 2024-09-14T05:31:03.625 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.39e-07 2024-09-14T05:31:03.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.625 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.625 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867894+0000", 2024-09-14T05:31:03.625 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.278e-06 2024-09-14T05:31:03.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.625 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.625 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867900+0000", 2024-09-14T05:31:03.625 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2469999999999997e-06 2024-09-14T05:31:03.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.625 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867905+0000", 2024-09-14T05:31:03.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0209999999999999e-06 2024-09-14T05:31:03.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916764+0000", 2024-09-14T05:31:03.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.048859163999999997 2024-09-14T05:31:03.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.626 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916845+0000", 2024-09-14T05:31:03.626 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0309999999999995e-05 2024-09-14T05:31:03.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.626 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.627 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916937+0000", 2024-09-14T05:31:03.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2569000000000001e-05 2024-09-14T05:31:03.627 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.627 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.627 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.627 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124265 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:31:03.627 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:31:03.627 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.734328634999997, 2024-09-14T05:31:03.627 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049155040999999997, 2024-09-14T05:31:03.627 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.627 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.627 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.627 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.628 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.628 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.628 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124265 2024-09-14T05:31:03.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.628 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.628 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.628 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:31:03.628 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.628 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.628 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:31:03.628 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.629 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.629 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867912+0000", 2024-09-14T05:31:03.629 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7099999999999997e-07 2024-09-14T05:31:03.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.629 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.629 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867914+0000", 2024-09-14T05:31:03.629 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9810000000000002e-06 2024-09-14T05:31:03.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.629 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.629 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867920+0000", 2024-09-14T05:31:03.630 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.643e-06 2024-09-14T05:31:03.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.630 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.630 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867924+0000", 2024-09-14T05:31:03.630 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.605e-06 2024-09-14T05:31:03.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.630 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.630 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.916970+0000", 2024-09-14T05:31:03.630 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049045501999999998 2024-09-14T05:31:03.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.631 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.631 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917013+0000", 2024-09-14T05:31:03.631 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3857999999999999e-05 2024-09-14T05:31:03.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.631 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.631 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917067+0000", 2024-09-14T05:31:03.631 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3081000000000002e-05 2024-09-14T05:31:03.631 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.631 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.631 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.632 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124266 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 4907008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:31:03.632 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867930+0000", 2024-09-14T05:31:03.632 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.734309776000003, 2024-09-14T05:31:03.632 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049337389000000002, 2024-09-14T05:31:03.632 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.632 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.632 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.632 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.632 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.632 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.632 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124266 2024-09-14T05:31:03.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.632 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867930+0000", 2024-09-14T05:31:03.633 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867930+0000", 2024-09-14T05:31:03.633 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.633 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.633 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867931+0000", 2024-09-14T05:31:03.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9999999999999998e-07 2024-09-14T05:31:03.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.634 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.634 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867933+0000", 2024-09-14T05:31:03.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.046e-06 2024-09-14T05:31:03.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.634 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.634 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867938+0000", 2024-09-14T05:31:03.634 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5589999999999996e-06 2024-09-14T05:31:03.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.635 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.635 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867941+0000", 2024-09-14T05:31:03.635 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.791e-06 2024-09-14T05:31:03.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.635 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.635 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917100+0000", 2024-09-14T05:31:03.635 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049158544999999998 2024-09-14T05:31:03.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.635 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.635 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917179+0000", 2024-09-14T05:31:03.635 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9350000000000004e-05 2024-09-14T05:31:03.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.636 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.636 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917268+0000", 2024-09-14T05:31:03.636 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8697999999999997e-05 2024-09-14T05:31:03.636 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.636 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.636 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.636 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.636 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124267 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5013504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:31:03.636 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:31:03.636 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.73429522, 2024-09-14T05:31:03.636 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049449223, 2024-09-14T05:31:03.636 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.636 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.637 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.637 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.637 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.637 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.637 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124267 2024-09-14T05:31:03.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.637 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.637 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.637 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:31:03.637 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.637 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.638 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:31:03.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.638 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.638 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867945+0000", 2024-09-14T05:31:03.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.28e-07 2024-09-14T05:31:03.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.638 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.638 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867946+0000", 2024-09-14T05:31:03.638 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.116e-06 2024-09-14T05:31:03.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867950+0000", 2024-09-14T05:31:03.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2049999999999998e-06 2024-09-14T05:31:03.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867952+0000", 2024-09-14T05:31:03.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6800000000000002e-06 2024-09-14T05:31:03.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.639 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.639 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917300+0000", 2024-09-14T05:31:03.639 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049347633000000002 2024-09-14T05:31:03.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.640 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917343+0000", 2024-09-14T05:31:03.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3022000000000002e-05 2024-09-14T05:31:03.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.640 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.640 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917394+0000", 2024-09-14T05:31:03.640 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1338999999999998e-05 2024-09-14T05:31:03.640 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.640 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.640 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.641 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124268 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5120000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:31:03.641 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867955+0000", 2024-09-14T05:31:03.641 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.734284813000002, 2024-09-14T05:31:03.641 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049638350999999997, 2024-09-14T05:31:03.641 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.641 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.641 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.641 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.641 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.641 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.641 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124268 2024-09-14T05:31:03.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.641 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.642 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.642 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867955+0000", 2024-09-14T05:31:03.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.642 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.642 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.642 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867955+0000", 2024-09-14T05:31:03.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.642 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.642 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.642 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867956+0000", 2024-09-14T05:31:03.642 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1299999999999999e-07 2024-09-14T05:31:03.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.643 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.643 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867961+0000", 2024-09-14T05:31:03.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4269999999999999e-06 2024-09-14T05:31:03.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.643 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.643 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867965+0000", 2024-09-14T05:31:03.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.563e-06 2024-09-14T05:31:03.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.643 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.643 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867967+0000", 2024-09-14T05:31:03.643 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6469999999999998e-06 2024-09-14T05:31:03.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917427+0000", 2024-09-14T05:31:03.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049460102999999998 2024-09-14T05:31:03.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.644 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.644 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917506+0000", 2024-09-14T05:31:03.644 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8246999999999998e-05 2024-09-14T05:31:03.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.645 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.645 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917594+0000", 2024-09-14T05:31:03.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8151000000000004e-05 2024-09-14T05:31:03.645 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.645 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.645 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.645 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124269 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:31:03.645 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:31:03.645 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.734269196, 2024-09-14T05:31:03.645 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049749122999999999, 2024-09-14T05:31:03.645 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.645 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.646 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.646 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.646 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.646 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.646 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124269 2024-09-14T05:31:03.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.646 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.646 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:31:03.646 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.646 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:31:03.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.647 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867971+0000", 2024-09-14T05:31:03.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2499999999999999e-07 2024-09-14T05:31:03.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.647 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.647 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867972+0000", 2024-09-14T05:31:03.647 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0440000000000001e-06 2024-09-14T05:31:03.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.648 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.648 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867975+0000", 2024-09-14T05:31:03.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9699999999999999e-06 2024-09-14T05:31:03.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.648 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.648 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867978+0000", 2024-09-14T05:31:03.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5909999999999998e-06 2024-09-14T05:31:03.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.648 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.648 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917625+0000", 2024-09-14T05:31:03.648 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049647450000000003 2024-09-14T05:31:03.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.649 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.649 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917668+0000", 2024-09-14T05:31:03.649 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3108000000000001e-05 2024-09-14T05:31:03.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.649 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.649 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917720+0000", 2024-09-14T05:31:03.649 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1734999999999998e-05 2024-09-14T05:31:03.649 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.649 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.649 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.650 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124270 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5292032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:31:03.650 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:31:03.650 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.734259256000001, 2024-09-14T05:31:03.650 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049946404, 2024-09-14T05:31:03.650 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.650 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.650 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.650 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.650 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.650 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.650 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124270 2024-09-14T05:31:03.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.650 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.651 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.651 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:31:03.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.651 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.651 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:31:03.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.651 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.651 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867981+0000", 2024-09-14T05:31:03.651 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5600000000000002e-07 2024-09-14T05:31:03.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.652 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867982+0000", 2024-09-14T05:31:03.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.95e-07 2024-09-14T05:31:03.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.652 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867985+0000", 2024-09-14T05:31:03.652 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8890000000000002e-06 2024-09-14T05:31:03.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.652 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.652 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.867988+0000", 2024-09-14T05:31:03.653 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5900000000000002e-06 2024-09-14T05:31:03.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.653 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.653 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917753+0000", 2024-09-14T05:31:03.653 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049765113 2024-09-14T05:31:03.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.653 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.653 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917833+0000", 2024-09-14T05:31:03.653 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0055000000000005e-05 2024-09-14T05:31:03.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.653 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.654 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917927+0000", 2024-09-14T05:31:03.654 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.4506000000000006e-05 2024-09-14T05:31:03.654 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.654 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.654 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.654 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.654 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124271 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5398528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:31:03.654 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868010+0000", 2024-09-14T05:31:03.654 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.734230060999998, 2024-09-14T05:31:03.654 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050049336, 2024-09-14T05:31:03.654 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.654 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.654 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.654 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.655 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.655 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.655 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124271 2024-09-14T05:31:03.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.655 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.655 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.655 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.655 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868010+0000", 2024-09-14T05:31:03.655 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.655 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.655 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.655 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868010+0000", 2024-09-14T05:31:03.655 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.656 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.656 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868011+0000", 2024-09-14T05:31:03.656 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:31:03.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.656 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.656 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868012+0000", 2024-09-14T05:31:03.656 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5719999999999999e-06 2024-09-14T05:31:03.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.656 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.656 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868016+0000", 2024-09-14T05:31:03.656 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.861e-06 2024-09-14T05:31:03.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.657 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.657 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.657 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868019+0000", 2024-09-14T05:31:03.657 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7580000000000001e-06 2024-09-14T05:31:03.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.657 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.657 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.657 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.917961+0000", 2024-09-14T05:31:03.657 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.049941885999999998 2024-09-14T05:31:03.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.657 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.657 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.657 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918007+0000", 2024-09-14T05:31:03.658 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6187999999999999e-05 2024-09-14T05:31:03.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.658 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.658 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.658 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918059+0000", 2024-09-14T05:31:03.658 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2645000000000002e-05 2024-09-14T05:31:03.658 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.658 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.658 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.658 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.658 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124272 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:31:03.658 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:31:03.658 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.734218034000001, 2024-09-14T05:31:03.659 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050239936999999998, 2024-09-14T05:31:03.659 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.659 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.659 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.659 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.659 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.659 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.659 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124272 2024-09-14T05:31:03.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.659 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.659 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.659 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.659 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:31:03.660 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.660 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.660 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.660 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:31:03.660 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.660 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.660 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.660 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868022+0000", 2024-09-14T05:31:03.660 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.16e-07 2024-09-14T05:31:03.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.660 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.660 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.661 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868023+0000", 2024-09-14T05:31:03.661 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7999999999999993e-07 2024-09-14T05:31:03.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.661 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.661 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868026+0000", 2024-09-14T05:31:03.661 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9859999999999999e-06 2024-09-14T05:31:03.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.661 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.661 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868029+0000", 2024-09-14T05:31:03.661 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.039e-06 2024-09-14T05:31:03.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.662 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.662 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918093+0000", 2024-09-14T05:31:03.662 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050063689000000001 2024-09-14T05:31:03.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.662 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.662 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918173+0000", 2024-09-14T05:31:03.662 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0131999999999998e-05 2024-09-14T05:31:03.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.662 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.662 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918262+0000", 2024-09-14T05:31:03.662 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8894999999999995e-05 2024-09-14T05:31:03.663 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.663 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.663 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.663 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124273 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:31:03.663 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868032+0000", 2024-09-14T05:31:03.663 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.734207885000004, 2024-09-14T05:31:03.663 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050357587000000002, 2024-09-14T05:31:03.663 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.663 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.663 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.663 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.663 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.663 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.664 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124273 2024-09-14T05:31:03.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.664 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.664 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.664 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868032+0000", 2024-09-14T05:31:03.664 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.664 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.664 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868032+0000", 2024-09-14T05:31:03.664 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.665 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.665 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868033+0000", 2024-09-14T05:31:03.665 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9799999999999999e-07 2024-09-14T05:31:03.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.665 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.665 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868034+0000", 2024-09-14T05:31:03.665 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1119999999999999e-06 2024-09-14T05:31:03.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.665 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.665 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868037+0000", 2024-09-14T05:31:03.665 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.0189999999999998e-06 2024-09-14T05:31:03.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.666 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.666 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868039+0000", 2024-09-14T05:31:03.666 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.57e-06 2024-09-14T05:31:03.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.666 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.666 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918294+0000", 2024-09-14T05:31:03.666 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050254778999999999 2024-09-14T05:31:03.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.666 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.666 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918338+0000", 2024-09-14T05:31:03.666 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4288e-05 2024-09-14T05:31:03.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.667 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.667 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918390+0000", 2024-09-14T05:31:03.667 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1520999999999997e-05 2024-09-14T05:31:03.667 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.667 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.667 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.667 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124274 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:31:03.667 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:31:03.667 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.734198042999999, 2024-09-14T05:31:03.667 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050548334, 2024-09-14T05:31:03.667 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.667 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.668 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.668 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.668 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.668 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.668 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124274 2024-09-14T05:31:03.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.668 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.668 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.668 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.668 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:31:03.668 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.668 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.668 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.669 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:31:03.669 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.669 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.669 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.669 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.669 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868042+0000", 2024-09-14T05:31:03.669 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1799999999999999e-07 2024-09-14T05:31:03.669 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.669 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.669 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.669 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868043+0000", 2024-09-14T05:31:03.669 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9999999999999995e-07 2024-09-14T05:31:03.669 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.669 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.669 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868046+0000", 2024-09-14T05:31:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8880000000000001e-06 2024-09-14T05:31:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868049+0000", 2024-09-14T05:31:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7e-06 2024-09-14T05:31:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918423+0000", 2024-09-14T05:31:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050374249000000003 2024-09-14T05:31:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918503+0000", 2024-09-14T05:31:03.671 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0049000000000006e-05 2024-09-14T05:31:03.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.671 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.671 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.671 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918590+0000", 2024-09-14T05:31:03.671 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7230000000000003e-05 2024-09-14T05:31:03.671 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.671 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.671 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.671 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.671 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124275 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5824512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:31:03.671 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:31:03.671 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.734188457000002, 2024-09-14T05:31:03.671 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050665241999999999, 2024-09-14T05:31:03.672 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.672 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.672 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.672 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.672 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.672 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.672 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124275 2024-09-14T05:31:03.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.672 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.672 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.672 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:31:03.672 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.673 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.673 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:31:03.673 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.673 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.673 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.673 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868052+0000", 2024-09-14T05:31:03.673 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1e-07 2024-09-14T05:31:03.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.673 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.673 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.673 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868053+0000", 2024-09-14T05:31:03.673 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.049e-06 2024-09-14T05:31:03.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.674 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.674 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868056+0000", 2024-09-14T05:31:03.674 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9519999999999999e-06 2024-09-14T05:31:03.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.674 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.674 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868058+0000", 2024-09-14T05:31:03.674 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4789999999999999e-06 2024-09-14T05:31:03.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.674 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.674 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918622+0000", 2024-09-14T05:31:03.674 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050563679 2024-09-14T05:31:03.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.675 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.675 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918666+0000", 2024-09-14T05:31:03.675 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3600000000000003e-05 2024-09-14T05:31:03.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.675 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.675 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918717+0000", 2024-09-14T05:31:03.675 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1273000000000002e-05 2024-09-14T05:31:03.675 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.675 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.675 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.676 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124276 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:31:03.676 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:31:03.676 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.734178913000001, 2024-09-14T05:31:03.676 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050860996999999998, 2024-09-14T05:31:03.676 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.676 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.676 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.676 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.676 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.676 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.676 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124276 2024-09-14T05:31:03.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.676 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.677 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.677 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:31:03.677 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.677 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.677 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:31:03.677 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.677 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.677 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868061+0000", 2024-09-14T05:31:03.677 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1e-07 2024-09-14T05:31:03.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.678 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.678 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.678 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868062+0000", 2024-09-14T05:31:03.678 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2099999999999995e-07 2024-09-14T05:31:03.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.678 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.678 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.678 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868065+0000", 2024-09-14T05:31:03.678 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9500000000000001e-06 2024-09-14T05:31:03.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.678 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.678 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.678 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868068+0000", 2024-09-14T05:31:03.678 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4760000000000001e-06 2024-09-14T05:31:03.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.679 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.679 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918750+0000", 2024-09-14T05:31:03.679 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050681926000000002 2024-09-14T05:31:03.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.679 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.679 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918829+0000", 2024-09-14T05:31:03.679 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9264000000000005e-05 2024-09-14T05:31:03.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.679 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.679 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918922+0000", 2024-09-14T05:31:03.679 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.3250000000000003e-05 2024-09-14T05:31:03.679 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.680 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.680 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.680 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.680 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124277 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 6037504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:31:03.680 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:31:03.680 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.734169305000002, 2024-09-14T05:31:03.680 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050980939000000003, 2024-09-14T05:31:03.680 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.680 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.680 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.680 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.680 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.680 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.680 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124277 2024-09-14T05:31:03.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.681 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.681 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.681 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:31:03.681 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.681 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.681 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:31:03.681 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.681 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.682 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868071+0000", 2024-09-14T05:31:03.682 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.11e-07 2024-09-14T05:31:03.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.682 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.682 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.682 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868072+0000", 2024-09-14T05:31:03.682 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.9999999999999995e-07 2024-09-14T05:31:03.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.682 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.682 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.682 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868075+0000", 2024-09-14T05:31:03.682 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9969999999999999e-06 2024-09-14T05:31:03.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.682 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.682 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.683 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868078+0000", 2024-09-14T05:31:03.683 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5349999999999999e-06 2024-09-14T05:31:03.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.683 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.683 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.683 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.918956+0000", 2024-09-14T05:31:03.683 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050878067999999999 2024-09-14T05:31:03.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.683 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.683 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.683 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919000+0000", 2024-09-14T05:31:03.683 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3837000000000001e-05 2024-09-14T05:31:03.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.683 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.684 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.684 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919052+0000", 2024-09-14T05:31:03.684 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2290999999999998e-05 2024-09-14T05:31:03.684 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.684 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.684 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.684 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.684 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.684 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124278 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 6144000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:31:03.685 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868080+0000", 2024-09-14T05:31:03.685 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.734159867000002, 2024-09-14T05:31:03.685 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.051170446000000001, 2024-09-14T05:31:03.685 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.685 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.685 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.685 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.685 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.685 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.685 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124278 2024-09-14T05:31:03.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.685 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.685 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.685 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.686 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868080+0000", 2024-09-14T05:31:03.686 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.686 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.686 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.686 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868080+0000", 2024-09-14T05:31:03.686 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.686 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.686 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.686 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868081+0000", 2024-09-14T05:31:03.686 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1500000000000001e-07 2024-09-14T05:31:03.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.686 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.686 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.687 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868081+0000", 2024-09-14T05:31:03.687 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9400000000000004e-07 2024-09-14T05:31:03.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.687 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.687 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.687 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868084+0000", 2024-09-14T05:31:03.687 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.943e-06 2024-09-14T05:31:03.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.687 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.687 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.687 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868087+0000", 2024-09-14T05:31:03.687 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6929999999999999e-06 2024-09-14T05:31:03.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.687 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.687 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.688 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919084+0000", 2024-09-14T05:31:03.688 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.050996581999999999 2024-09-14T05:31:03.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.688 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.688 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919163+0000", 2024-09-14T05:31:03.688 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8931999999999996e-05 2024-09-14T05:31:03.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.688 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.688 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919251+0000", 2024-09-14T05:31:03.688 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.8187000000000001e-05 2024-09-14T05:31:03.688 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.688 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.689 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.689 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:124279 3.10 3:08652b2a:::rbd_data.112d5502ed06.0000000000000000:27 [read 6250496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T05:31:03.689 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:31:03.689 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 45.734150257000003, 2024-09-14T05:31:03.689 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.051284430999999998, 2024-09-14T05:31:03.689 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:03.689 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:03.689 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:03.689 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:03.689 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:03.689 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:03.690 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 124279 2024-09-14T05:31:03.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.690 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:03.690 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.690 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:03.690 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:31:03.690 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.690 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.690 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:03.690 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:31:03.690 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:03.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.690 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.690 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:03.691 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868090+0000", 2024-09-14T05:31:03.691 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.11e-07 2024-09-14T05:31:03.691 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.691 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.691 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:03.691 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868091+0000", 2024-09-14T05:31:03.691 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.8299999999999995e-07 2024-09-14T05:31:03.691 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.691 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.691 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:03.691 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868094+0000", 2024-09-14T05:31:03.691 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.8870000000000001e-06 2024-09-14T05:31:03.691 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.691 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.692 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:03.692 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.868097+0000", 2024-09-14T05:31:03.692 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5330000000000001e-06 2024-09-14T05:31:03.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.692 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.692 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:03.692 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919276+0000", 2024-09-14T05:31:03.692 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.051179740000000001 2024-09-14T05:31:03.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.692 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.692 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:03.692 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919322+0000", 2024-09-14T05:31:03.692 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5475000000000001e-05 2024-09-14T05:31:03.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:03.692 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:03.692 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:03.693 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:17.919374+0000", 2024-09-14T05:31:03.693 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2602000000000002e-05 2024-09-14T05:31:03.693 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.693 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.693 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.693 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:03.693 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:03.693 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:03.694 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T05:31:03.798 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:03.798 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:31:03.798 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:31:03.798 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:03.808 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T05:31:03.910 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:03.910 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:31:03.910 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:31:03.910 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:31:03.910 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:03.918 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T05:31:04.027 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:04.028 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:31:04.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:31:04.028 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:31:04.028 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.028 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73760 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2848768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.028 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:31:04.028 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.466393729000004, 2024-09-14T05:31:04.028 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.04469911, 2024-09-14T05:31:04.028 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.028 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.028 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.028 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.028 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.029 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.029 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73760 2024-09-14T05:31:04.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.029 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:31:04.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.029 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.029 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:31:04.029 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559699+0000", 2024-09-14T05:31:04.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:31:04.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559701+0000", 2024-09-14T05:31:04.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1909999999999999e-06 2024-09-14T05:31:04.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.030 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.030 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559708+0000", 2024-09-14T05:31:04.030 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.7279999999999998e-06 2024-09-14T05:31:04.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559713+0000", 2024-09-14T05:31:04.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8389999999999997e-06 2024-09-14T05:31:04.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604310+0000", 2024-09-14T05:31:04.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044597664000000002 2024-09-14T05:31:04.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.031 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.031 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.031 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604352+0000", 2024-09-14T05:31:04.031 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1236000000000003e-05 2024-09-14T05:31:04.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.032 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.032 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604398+0000", 2024-09-14T05:31:04.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6091000000000003e-05 2024-09-14T05:31:04.032 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.032 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.032 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.032 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73773 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2801664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.032 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:31:04.032 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.466373559999994, 2024-09-14T05:31:04.032 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941026000000002, 2024-09-14T05:31:04.032 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.033 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.033 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.033 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.033 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.033 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.033 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73773 2024-09-14T05:31:04.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.033 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.033 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:31:04.033 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.033 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:31:04.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559719+0000", 2024-09-14T05:31:04.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8599999999999999e-07 2024-09-14T05:31:04.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559721+0000", 2024-09-14T05:31:04.034 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0540000000000002e-06 2024-09-14T05:31:04.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.034 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.034 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559729+0000", 2024-09-14T05:31:04.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1359999999999997e-06 2024-09-14T05:31:04.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559734+0000", 2024-09-14T05:31:04.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2140000000000004e-06 2024-09-14T05:31:04.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604441+0000", 2024-09-14T05:31:04.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044706828999999997 2024-09-14T05:31:04.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.035 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.035 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604522+0000", 2024-09-14T05:31:04.035 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0603000000000001e-05 2024-09-14T05:31:04.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.036 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.036 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604660+0000", 2024-09-14T05:31:04.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000137804 2024-09-14T05:31:04.036 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.036 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.036 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.036 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73777 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2834432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.036 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:31:04.036 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.466351158999998, 2024-09-14T05:31:04.036 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045116270999999999, 2024-09-14T05:31:04.037 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.037 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.037 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.037 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.037 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.037 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.037 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73777 2024-09-14T05:31:04.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.037 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.037 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.037 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:31:04.037 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559741+0000", 2024-09-14T05:31:04.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559742+0000", 2024-09-14T05:31:04.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4099999999999999e-07 2024-09-14T05:31:04.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.038 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.038 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559744+0000", 2024-09-14T05:31:04.038 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0040000000000002e-06 2024-09-14T05:31:04.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559749+0000", 2024-09-14T05:31:04.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.9739999999999999e-06 2024-09-14T05:31:04.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559754+0000", 2024-09-14T05:31:04.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9540000000000003e-06 2024-09-14T05:31:04.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.039 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.039 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.039 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604695+0000", 2024-09-14T05:31:04.039 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044941003 2024-09-14T05:31:04.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604766+0000", 2024-09-14T05:31:04.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0296999999999995e-05 2024-09-14T05:31:04.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.040 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.040 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604857+0000", 2024-09-14T05:31:04.040 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.1598e-05 2024-09-14T05:31:04.040 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.040 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.040 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.041 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73790 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.041 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:31:04.041 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.466331224000001, 2024-09-14T05:31:04.041 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316437000000001, 2024-09-14T05:31:04.041 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.041 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.041 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.041 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.041 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.041 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.041 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73790 2024-09-14T05:31:04.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.041 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:31:04.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:31:04.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.042 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.042 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559761+0000", 2024-09-14T05:31:04.042 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7599999999999998e-07 2024-09-14T05:31:04.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559763+0000", 2024-09-14T05:31:04.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0880000000000002e-06 2024-09-14T05:31:04.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559769+0000", 2024-09-14T05:31:04.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6740000000000002e-06 2024-09-14T05:31:04.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.043 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.043 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559774+0000", 2024-09-14T05:31:04.043 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6110000000000004e-06 2024-09-14T05:31:04.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604905+0000", 2024-09-14T05:31:04.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045130827999999998 2024-09-14T05:31:04.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.604982+0000", 2024-09-14T05:31:04.044 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.7410999999999995e-05 2024-09-14T05:31:04.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.044 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.044 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605077+0000", 2024-09-14T05:31:04.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5449e-05 2024-09-14T05:31:04.045 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.045 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.045 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.045 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73793 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2836480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.045 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:31:04.045 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.466312020000004, 2024-09-14T05:31:04.045 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045441212000000002, 2024-09-14T05:31:04.045 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.045 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.045 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.045 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.046 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.046 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.046 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73793 2024-09-14T05:31:04.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.046 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:31:04.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.046 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.046 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.046 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559780+0000", 2024-09-14T05:31:04.046 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559781+0000", 2024-09-14T05:31:04.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2899999999999999e-07 2024-09-14T05:31:04.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559783+0000", 2024-09-14T05:31:04.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.075e-06 2024-09-14T05:31:04.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.047 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.047 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559788+0000", 2024-09-14T05:31:04.047 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5369999999999997e-06 2024-09-14T05:31:04.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.048 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.048 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559793+0000", 2024-09-14T05:31:04.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:31:04.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.048 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.048 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605109+0000", 2024-09-14T05:31:04.048 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045316229999999999 2024-09-14T05:31:04.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605168+0000", 2024-09-14T05:31:04.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8949000000000001e-05 2024-09-14T05:31:04.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.049 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.049 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605221+0000", 2024-09-14T05:31:04.049 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3260000000000002e-05 2024-09-14T05:31:04.049 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.049 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.049 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.050 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73806 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.050 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:31:04.050 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.466293394000004, 2024-09-14T05:31:04.050 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045526189000000002, 2024-09-14T05:31:04.050 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.050 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.050 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.050 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.050 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.050 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.050 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73806 2024-09-14T05:31:04.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.050 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:31:04.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:31:04.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.051 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.051 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559799+0000", 2024-09-14T05:31:04.051 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:31:04.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559801+0000", 2024-09-14T05:31:04.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.995e-06 2024-09-14T05:31:04.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559807+0000", 2024-09-14T05:31:04.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5450000000000003e-06 2024-09-14T05:31:04.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.052 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559811+0000", 2024-09-14T05:31:04.052 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5090000000000004e-06 2024-09-14T05:31:04.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.052 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.053 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605237+0000", 2024-09-14T05:31:04.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045426133 2024-09-14T05:31:04.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.053 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.053 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605278+0000", 2024-09-14T05:31:04.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1084999999999997e-05 2024-09-14T05:31:04.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.053 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.053 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605325+0000", 2024-09-14T05:31:04.053 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6582999999999998e-05 2024-09-14T05:31:04.053 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.053 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.054 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.054 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.054 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73809 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2838528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.054 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:31:04.054 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.466269889000003, 2024-09-14T05:31:04.054 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045596062999999999, 2024-09-14T05:31:04.054 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.054 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.054 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.054 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.054 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.054 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.055 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73809 2024-09-14T05:31:04.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.055 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:31:04.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.055 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559822+0000", 2024-09-14T05:31:04.055 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.055 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559823+0000", 2024-09-14T05:31:04.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.08e-07 2024-09-14T05:31:04.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559825+0000", 2024-09-14T05:31:04.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.26e-06 2024-09-14T05:31:04.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.056 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.056 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559831+0000", 2024-09-14T05:31:04.056 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6690000000000003e-06 2024-09-14T05:31:04.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.056 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559835+0000", 2024-09-14T05:31:04.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5120000000000002e-06 2024-09-14T05:31:04.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605342+0000", 2024-09-14T05:31:04.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045506957000000001 2024-09-14T05:31:04.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.057 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.057 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605375+0000", 2024-09-14T05:31:04.057 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3250999999999997e-05 2024-09-14T05:31:04.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.058 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.058 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.058 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605418+0000", 2024-09-14T05:31:04.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3006000000000001e-05 2024-09-14T05:31:04.058 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.058 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.058 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.058 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.058 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.058 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73822 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2807808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.058 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:31:04.058 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.466242235999999, 2024-09-14T05:31:04.058 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045661405000000002, 2024-09-14T05:31:04.058 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.058 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.058 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.059 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.059 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.059 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.059 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73822 2024-09-14T05:31:04.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.059 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.059 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.059 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:31:04.059 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.059 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.059 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559850+0000", 2024-09-14T05:31:04.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.060 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559851+0000", 2024-09-14T05:31:04.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4000000000000002e-07 2024-09-14T05:31:04.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.060 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559853+0000", 2024-09-14T05:31:04.060 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.5380000000000001e-06 2024-09-14T05:31:04.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.060 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.060 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559859+0000", 2024-09-14T05:31:04.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0619999999999997e-06 2024-09-14T05:31:04.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559864+0000", 2024-09-14T05:31:04.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.977e-06 2024-09-14T05:31:04.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.061 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.061 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605435+0000", 2024-09-14T05:31:04.061 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045570672 2024-09-14T05:31:04.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605467+0000", 2024-09-14T05:31:04.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2486e-05 2024-09-14T05:31:04.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.062 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.062 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605511+0000", 2024-09-14T05:31:04.062 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4129999999999999e-05 2024-09-14T05:31:04.062 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.062 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.062 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.062 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:73825 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2840576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.062 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:31:04.063 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.466218871999999, 2024-09-14T05:31:04.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045728694, 2024-09-14T05:31:04.063 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.063 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.063 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.063 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.063 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.063 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.063 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 73825 2024-09-14T05:31:04.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.063 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.063 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.063 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:31:04.063 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559873+0000", 2024-09-14T05:31:04.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559874+0000", 2024-09-14T05:31:04.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5400000000000002e-07 2024-09-14T05:31:04.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.064 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.064 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559876+0000", 2024-09-14T05:31:04.064 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.131e-06 2024-09-14T05:31:04.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559888+0000", 2024-09-14T05:31:04.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2216e-05 2024-09-14T05:31:04.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.559893+0000", 2024-09-14T05:31:04.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6539999999999998e-06 2024-09-14T05:31:04.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.065 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.065 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.065 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605526+0000", 2024-09-14T05:31:04.065 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045633119 2024-09-14T05:31:04.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605560+0000", 2024-09-14T05:31:04.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.3825999999999998e-05 2024-09-14T05:31:04.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.066 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.066 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605602+0000", 2024-09-14T05:31:04.066 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2293999999999999e-05 2024-09-14T05:31:04.066 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.066 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.066 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.066 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74023 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2863104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.067 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:31:04.067 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.465032897, 2024-09-14T05:31:04.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044635385999999999, 2024-09-14T05:31:04.067 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.067 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.067 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.067 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.067 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.067 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.067 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74023 2024-09-14T05:31:04.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.067 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.067 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.067 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:31:04.067 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561059+0000", 2024-09-14T05:31:04.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561060+0000", 2024-09-14T05:31:04.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0699999999999998e-07 2024-09-14T05:31:04.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.068 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.068 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561072+0000", 2024-09-14T05:31:04.068 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.2261e-05 2024-09-14T05:31:04.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.069 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.069 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561079+0000", 2024-09-14T05:31:04.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8399999999999997e-06 2024-09-14T05:31:04.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.069 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.069 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561084+0000", 2024-09-14T05:31:04.069 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7380000000000002e-06 2024-09-14T05:31:04.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.069 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.069 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.069 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605620+0000", 2024-09-14T05:31:04.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044536146999999998 2024-09-14T05:31:04.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605653+0000", 2024-09-14T05:31:04.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2969000000000002e-05 2024-09-14T05:31:04.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.070 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.070 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605695+0000", 2024-09-14T05:31:04.070 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2024000000000003e-05 2024-09-14T05:31:04.070 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.070 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.070 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.071 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74026 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2895872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.071 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:31:04.071 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.465001435000005, 2024-09-14T05:31:04.071 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044692717, 2024-09-14T05:31:04.071 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.071 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.071 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.071 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.071 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.071 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.071 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74026 2024-09-14T05:31:04.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.071 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.071 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.071 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:31:04.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:31:04.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561091+0000", 2024-09-14T05:31:04.072 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2599999999999998e-07 2024-09-14T05:31:04.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.072 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.072 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.072 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561093+0000", 2024-09-14T05:31:04.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9429999999999999e-06 2024-09-14T05:31:04.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.073 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.073 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561099+0000", 2024-09-14T05:31:04.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0399999999999998e-06 2024-09-14T05:31:04.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.073 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.073 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561104+0000", 2024-09-14T05:31:04.073 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8180000000000003e-06 2024-09-14T05:31:04.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.073 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.073 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605710+0000", 2024-09-14T05:31:04.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044605657 2024-09-14T05:31:04.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.074 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.074 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.074 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605742+0000", 2024-09-14T05:31:04.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.1949000000000002e-05 2024-09-14T05:31:04.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.074 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.074 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.074 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605784+0000", 2024-09-14T05:31:04.074 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1884000000000001e-05 2024-09-14T05:31:04.074 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.074 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.074 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.075 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74039 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.075 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:31:04.075 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.464874836999996, 2024-09-14T05:31:04.075 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044654905000000002, 2024-09-14T05:31:04.075 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.075 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.075 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.075 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.075 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.075 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.075 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74039 2024-09-14T05:31:04.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.075 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.075 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.075 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.076 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:31:04.076 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.076 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.076 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561217+0000", 2024-09-14T05:31:04.076 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.076 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.076 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561218+0000", 2024-09-14T05:31:04.076 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4000000000000001e-07 2024-09-14T05:31:04.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.076 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561221+0000", 2024-09-14T05:31:04.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.9730000000000002e-06 2024-09-14T05:31:04.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.077 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.077 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561229+0000", 2024-09-14T05:31:04.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1759999999999993e-06 2024-09-14T05:31:04.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.077 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.077 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.077 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561235+0000", 2024-09-14T05:31:04.077 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0109999999999997e-06 2024-09-14T05:31:04.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.078 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.078 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.078 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605798+0000", 2024-09-14T05:31:04.078 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044563285000000001 2024-09-14T05:31:04.078 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.078 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.078 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.078 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605831+0000", 2024-09-14T05:31:04.078 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2054999999999997e-05 2024-09-14T05:31:04.078 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.078 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.078 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.078 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605872+0000", 2024-09-14T05:31:04.078 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1764999999999997e-05 2024-09-14T05:31:04.078 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.078 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.079 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.079 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.079 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.079 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74042 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.079 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:31:04.079 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.464826384000006, 2024-09-14T05:31:04.079 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044773234000000002, 2024-09-14T05:31:04.079 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.079 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.079 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.079 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.079 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.079 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.079 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74042 2024-09-14T05:31:04.079 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.079 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.080 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.080 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.080 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:31:04.080 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.080 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.080 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.080 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.080 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:31:04.080 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.080 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.080 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.080 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.080 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561266+0000", 2024-09-14T05:31:04.080 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4099999999999998e-07 2024-09-14T05:31:04.080 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.080 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.081 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.081 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561269+0000", 2024-09-14T05:31:04.081 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7930000000000002e-06 2024-09-14T05:31:04.081 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.081 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.081 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.081 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561278+0000", 2024-09-14T05:31:04.081 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.5150000000000006e-06 2024-09-14T05:31:04.081 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.081 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.081 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.081 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561284+0000", 2024-09-14T05:31:04.081 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2029999999999997e-06 2024-09-14T05:31:04.081 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.082 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.082 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.082 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605934+0000", 2024-09-14T05:31:04.082 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044650449000000002 2024-09-14T05:31:04.082 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.082 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.082 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.082 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605983+0000", 2024-09-14T05:31:04.082 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8943999999999998e-05 2024-09-14T05:31:04.082 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.082 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.082 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.082 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606039+0000", 2024-09-14T05:31:04.082 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5689000000000002e-05 2024-09-14T05:31:04.082 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.083 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.083 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.083 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.083 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.083 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74052 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.083 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:31:04.083 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.464799509000002, 2024-09-14T05:31:04.083 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044830474000000002, 2024-09-14T05:31:04.083 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.083 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.083 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.083 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.083 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.083 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.084 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74052 2024-09-14T05:31:04.084 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.084 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.084 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.084 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.084 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:31:04.084 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.084 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.084 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.084 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.084 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:31:04.084 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.084 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.084 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.085 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.085 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561293+0000", 2024-09-14T05:31:04.085 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1099999999999996e-07 2024-09-14T05:31:04.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.085 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.085 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.085 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561295+0000", 2024-09-14T05:31:04.085 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7770000000000001e-06 2024-09-14T05:31:04.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.085 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.085 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.085 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561303+0000", 2024-09-14T05:31:04.085 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.6860000000000006e-06 2024-09-14T05:31:04.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.085 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.086 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.086 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561308+0000", 2024-09-14T05:31:04.086 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.023e-06 2024-09-14T05:31:04.086 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.086 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.086 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.086 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605893+0000", 2024-09-14T05:31:04.086 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044585458000000001 2024-09-14T05:31:04.086 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.086 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.086 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.086 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.605933+0000", 2024-09-14T05:31:04.086 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.9551999999999997e-05 2024-09-14T05:31:04.086 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.087 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.087 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.087 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606123+0000", 2024-09-14T05:31:04.087 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019046699999999999 2024-09-14T05:31:04.087 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.087 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.087 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.087 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.087 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.087 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74054 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.087 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:31:04.087 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.464777482000002, 2024-09-14T05:31:04.087 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093610999999999, 2024-09-14T05:31:04.087 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.087 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.087 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.088 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.088 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.088 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.088 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74054 2024-09-14T05:31:04.088 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.088 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.088 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.088 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.088 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:31:04.088 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.088 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.088 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.088 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.088 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:31:04.089 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.089 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.089 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.089 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.089 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561315+0000", 2024-09-14T05:31:04.089 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9699999999999996e-07 2024-09-14T05:31:04.089 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.089 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.089 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.089 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561317+0000", 2024-09-14T05:31:04.089 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.9740000000000001e-06 2024-09-14T05:31:04.089 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.089 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.089 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.089 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561324+0000", 2024-09-14T05:31:04.090 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4590000000000003e-06 2024-09-14T05:31:04.090 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.090 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.090 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.090 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561329+0000", 2024-09-14T05:31:04.090 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5969999999999999e-06 2024-09-14T05:31:04.090 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.090 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.090 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.090 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606142+0000", 2024-09-14T05:31:04.090 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044812347000000002 2024-09-14T05:31:04.090 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.090 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.090 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.090 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606184+0000", 2024-09-14T05:31:04.091 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2441999999999998e-05 2024-09-14T05:31:04.091 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.091 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.091 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.091 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606408+0000", 2024-09-14T05:31:04.091 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022429500000000001 2024-09-14T05:31:04.091 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.091 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.091 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.091 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.091 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.091 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74056 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.091 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:31:04.091 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.464582778999997, 2024-09-14T05:31:04.091 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045191685000000002, 2024-09-14T05:31:04.092 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.092 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.092 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.092 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.092 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.092 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.092 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74056 2024-09-14T05:31:04.092 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.092 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.092 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.092 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.092 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:31:04.092 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.093 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.093 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.093 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.093 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561509+0000", 2024-09-14T05:31:04.093 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.093 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.093 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.093 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.093 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561510+0000", 2024-09-14T05:31:04.093 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2399999999999998e-07 2024-09-14T05:31:04.093 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.093 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.094 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.094 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561513+0000", 2024-09-14T05:31:04.094 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.7219999999999999e-06 2024-09-14T05:31:04.094 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.094 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.094 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.094 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561524+0000", 2024-09-14T05:31:04.094 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.1357e-05 2024-09-14T05:31:04.094 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.094 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.094 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.095 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561530+0000", 2024-09-14T05:31:04.095 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2890000000000003e-06 2024-09-14T05:31:04.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.095 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.095 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.095 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606438+0000", 2024-09-14T05:31:04.095 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044907326999999997 2024-09-14T05:31:04.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.095 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.095 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.095 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606520+0000", 2024-09-14T05:31:04.095 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1924999999999999e-05 2024-09-14T05:31:04.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.095 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.096 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.096 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606701+0000", 2024-09-14T05:31:04.096 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00018144100000000001 2024-09-14T05:31:04.096 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.096 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.096 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.096 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.096 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.096 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74058 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.096 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:31:04.096 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.464104105999994, 2024-09-14T05:31:04.096 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045093368000000002, 2024-09-14T05:31:04.096 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.096 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.097 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.097 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.097 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.097 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.097 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74058 2024-09-14T05:31:04.097 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.097 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.097 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.097 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.097 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:31:04.097 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.097 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.097 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.097 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.098 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561988+0000", 2024-09-14T05:31:04.098 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.098 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.098 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.098 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.098 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561989+0000", 2024-09-14T05:31:04.098 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3300000000000001e-07 2024-09-14T05:31:04.098 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.098 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.098 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.098 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.561993+0000", 2024-09-14T05:31:04.098 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1790000000000001e-06 2024-09-14T05:31:04.098 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.098 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.098 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.099 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562007+0000", 2024-09-14T05:31:04.099 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.3434e-05 2024-09-14T05:31:04.099 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.099 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.099 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.099 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562014+0000", 2024-09-14T05:31:04.099 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.537e-06 2024-09-14T05:31:04.099 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.099 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.099 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.099 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606736+0000", 2024-09-14T05:31:04.099 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.044722117999999998 2024-09-14T05:31:04.099 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.099 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.099 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.100 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.606829+0000", 2024-09-14T05:31:04.100 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.2410999999999993e-05 2024-09-14T05:31:04.100 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.100 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.100 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.100 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607082+0000", 2024-09-14T05:31:04.100 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025285599999999998 2024-09-14T05:31:04.100 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.100 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.100 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.100 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.100 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.100 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74059 3.e 3:761d1174:::rbd_data.112d46b1ca94.000000000000005c:11 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.100 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:31:04.100 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.464046115000002, 2024-09-14T05:31:04.101 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045479928000000003, 2024-09-14T05:31:04.101 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.101 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.101 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.101 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.101 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.101 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.101 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74059 2024-09-14T05:31:04.101 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.101 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.101 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.101 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.101 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:31:04.101 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.102 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.102 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.102 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.102 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562046+0000", 2024-09-14T05:31:04.102 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.102 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.102 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.102 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.102 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562047+0000", 2024-09-14T05:31:04.102 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6899999999999997e-07 2024-09-14T05:31:04.102 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.102 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.102 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.103 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562050+0000", 2024-09-14T05:31:04.103 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.943e-06 2024-09-14T05:31:04.103 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.103 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.103 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.103 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562058+0000", 2024-09-14T05:31:04.103 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.0120000000000005e-06 2024-09-14T05:31:04.103 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.103 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.103 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.103 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562063+0000", 2024-09-14T05:31:04.103 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:31:04.103 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.103 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.104 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.104 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607186+0000", 2024-09-14T05:31:04.104 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045122348999999999 2024-09-14T05:31:04.104 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.104 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.104 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.104 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607265+0000", 2024-09-14T05:31:04.104 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9462999999999995e-05 2024-09-14T05:31:04.104 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.104 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.104 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.104 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607526+0000", 2024-09-14T05:31:04.104 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00026090400000000001 2024-09-14T05:31:04.104 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.105 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.105 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.105 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.105 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.105 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:74060 3.16 3:6b16d72f:::rbd_data.112d641bd628.0000000000000005:10 [read 2850816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-14T05:31:04.105 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:31:04.105 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 74.464020738000002, 2024-09-14T05:31:04.105 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045200092999999997, 2024-09-14T05:31:04.105 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.105 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.105 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.105 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.105 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.105 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.106 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 74060 2024-09-14T05:31:04.106 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.106 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.106 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.106 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.106 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:31:04.106 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.106 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.106 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.106 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.106 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:31:04.106 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.106 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.106 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.106 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.106 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562072+0000", 2024-09-14T05:31:04.107 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5200000000000002e-07 2024-09-14T05:31:04.107 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.107 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.107 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.107 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562075+0000", 2024-09-14T05:31:04.107 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5200000000000002e-06 2024-09-14T05:31:04.107 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.107 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.107 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.107 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562082+0000", 2024-09-14T05:31:04.107 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8929999999999999e-06 2024-09-14T05:31:04.107 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.107 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.107 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.107 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.562088+0000", 2024-09-14T05:31:04.108 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.5570000000000003e-06 2024-09-14T05:31:04.108 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.108 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.108 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.108 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607111+0000", 2024-09-14T05:31:04.108 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.045023464999999999 2024-09-14T05:31:04.108 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.108 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.108 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.108 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607190+0000", 2024-09-14T05:31:04.108 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.8721999999999996e-05 2024-09-14T05:31:04.108 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.108 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.108 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.108 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:49.607272+0000", 2024-09-14T05:31:04.109 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.1483999999999994e-05 2024-09-14T05:31:04.109 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.109 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.109 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.109 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.109 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.109 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:136816 3.7 3:e5cc52c3:::rbd_id.image_client.0-clone31:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-14T05:31:04.109 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:30:30.438431+0000", 2024-09-14T05:31:04.109 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 33.587661034999996, 2024-09-14T05:31:04.109 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.96261680299999997, 2024-09-14T05:31:04.109 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.109 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.109 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:04.109 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.109 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.110 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.110 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 136816 2024-09-14T05:31:04.110 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.110 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.110 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.110 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.110 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438431+0000", 2024-09-14T05:31:04.110 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.110 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.110 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.110 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.110 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438431+0000", 2024-09-14T05:31:04.110 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.110 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.111 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.111 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.111 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438432+0000", 2024-09-14T05:31:04.111 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0419999999999999e-06 2024-09-14T05:31:04.111 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.111 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.111 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.111 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438437+0000", 2024-09-14T05:31:04.111 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.5000000000000001e-06 2024-09-14T05:31:04.111 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.111 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.111 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.111 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438446+0000", 2024-09-14T05:31:04.111 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5740000000000002e-06 2024-09-14T05:31:04.111 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.111 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.112 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.112 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438456+0000", 2024-09-14T05:31:04.112 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0056e-05 2024-09-14T05:31:04.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.112 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.112 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.112 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438500+0000", 2024-09-14T05:31:04.112 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3738e-05 2024-09-14T05:31:04.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.112 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.112 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:31:04.112 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:30.438506+0000", 2024-09-14T05:31:04.112 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1659999999999999e-06 2024-09-14T05:31:04.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.112 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.113 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.113 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.399222+0000", 2024-09-14T05:31:04.113 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.96071578999999996 2024-09-14T05:31:04.113 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.113 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.113 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.113 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.399381+0000", 2024-09-14T05:31:04.113 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000158834 2024-09-14T05:31:04.113 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.113 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.113 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for subops from 4", 2024-09-14T05:31:04.113 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.399530+0000", 2024-09-14T05:31:04.113 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00014862400000000001 2024-09-14T05:31:04.113 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.113 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.114 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "op_commit", 2024-09-14T05:31:04.114 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.400507+0000", 2024-09-14T05:31:04.114 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.0009779369999999999 2024-09-14T05:31:04.114 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.114 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.114 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:31:04.114 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.400969+0000", 2024-09-14T05:31:04.114 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00046119800000000001 2024-09-14T05:31:04.114 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.114 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.114 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "commit_sent", 2024-09-14T05:31:04.114 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.401006+0000", 2024-09-14T05:31:04.114 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7594e-05 2024-09-14T05:31:04.114 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.115 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.115 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.115 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:30:31.401048+0000", 2024-09-14T05:31:04.115 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1749999999999998e-05 2024-09-14T05:31:04.115 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.115 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.115 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.115 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.115 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.115 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:04.115 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T05:31:04.117 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T05:31:04.263 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:04.263 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:31:04.263 INFO:teuthology.orchestra.run.smithi071.stdout: "num_ops": 0 2024-09-14T05:31:04.263 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:04.273 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T05:31:04.377 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [], 2024-09-14T05:31:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: "complaint_time": 30, 2024-09-14T05:31:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: "num_blocked_ops": 0 2024-09-14T05:31:04.378 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:04.386 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T05:31:04.491 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:04.491 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 20, 2024-09-14T05:31:04.491 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 600, 2024-09-14T05:31:04.491 INFO:teuthology.orchestra.run.smithi071.stdout: "ops": [ 2024-09-14T05:31:04.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.491 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4235.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-14T05:31:04.492 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:31:04.492 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 110.649981748, 2024-09-14T05:31:04.492 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12887758399999999, 2024-09-14T05:31:04.492 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.492 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.492 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "reached pg", 2024-09-14T05:31:04.492 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.492 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4235", 2024-09-14T05:31:04.492 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/3211559243", 2024-09-14T05:31:04.492 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 2 2024-09-14T05:31:04.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.492 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.492 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.493 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:31:04.493 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.493 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839832+0000", 2024-09-14T05:31:04.493 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.493 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.493 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839833+0000", 2024-09-14T05:31:04.493 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:31:04.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839838+0000", 2024-09-14T05:31:04.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.2440000000000001e-06 2024-09-14T05:31:04.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839848+0000", 2024-09-14T05:31:04.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.0030000000000001e-05 2024-09-14T05:31:04.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.494 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.494 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839854+0000", 2024-09-14T05:31:04.494 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3750000000000002e-06 2024-09-14T05:31:04.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839908+0000", 2024-09-14T05:31:04.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3965999999999998e-05 2024-09-14T05:31:04.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "waiting for peered", 2024-09-14T05:31:04.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.839914+0000", 2024-09-14T05:31:04.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6139999999999997e-06 2024-09-14T05:31:04.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.495 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.495 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968558+0000", 2024-09-14T05:31:04.495 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.12864361199999999 2024-09-14T05:31:04.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.496 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.496 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:13.968710+0000", 2024-09-14T05:31:04.496 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00015177300000000001 2024-09-14T05:31:04.496 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.496 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.496 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.496 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27764 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 188416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.496 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:31:04.496 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.278655365999995, 2024-09-14T05:31:04.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080826920999999996, 2024-09-14T05:31:04.497 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.497 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.497 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.497 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.497 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.497 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.497 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27764 2024-09-14T05:31:04.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.497 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.497 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.497 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:31:04.497 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211158+0000", 2024-09-14T05:31:04.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.498 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211159+0000", 2024-09-14T05:31:04.498 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1800000000000001e-07 2024-09-14T05:31:04.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.498 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211161+0000", 2024-09-14T05:31:04.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.968e-06 2024-09-14T05:31:04.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211169+0000", 2024-09-14T05:31:04.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.4800000000000001e-06 2024-09-14T05:31:04.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.499 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.499 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211174+0000", 2024-09-14T05:31:04.499 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.1880000000000001e-06 2024-09-14T05:31:04.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291670+0000", 2024-09-14T05:31:04.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080495199000000003 2024-09-14T05:31:04.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291775+0000", 2024-09-14T05:31:04.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000105375 2024-09-14T05:31:04.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.500 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.500 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.291985+0000", 2024-09-14T05:31:04.500 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00021029299999999999 2024-09-14T05:31:04.501 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.501 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.501 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.501 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27765 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.501 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:31:04.501 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.278632615999996, 2024-09-14T05:31:04.501 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081115197999999999, 2024-09-14T05:31:04.501 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.501 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.501 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.501 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.501 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.501 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.502 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27765 2024-09-14T05:31:04.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.502 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:31:04.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.502 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.502 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211181+0000", 2024-09-14T05:31:04.502 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211182+0000", 2024-09-14T05:31:04.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.3599999999999999e-07 2024-09-14T05:31:04.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211184+0000", 2024-09-14T05:31:04.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.03e-06 2024-09-14T05:31:04.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.503 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.503 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211192+0000", 2024-09-14T05:31:04.503 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.2309999999999996e-06 2024-09-14T05:31:04.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211197+0000", 2024-09-14T05:31:04.504 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4199999999999998e-06 2024-09-14T05:31:04.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.504 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.504 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292017+0000", 2024-09-14T05:31:04.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.080819516999999994 2024-09-14T05:31:04.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292087+0000", 2024-09-14T05:31:04.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0056999999999997e-05 2024-09-14T05:31:04.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.505 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.505 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292296+0000", 2024-09-14T05:31:04.505 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000209507 2024-09-14T05:31:04.505 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.505 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.505 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.506 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27766 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.506 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:31:04.506 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.278609402000001, 2024-09-14T05:31:04.506 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081359077000000002, 2024-09-14T05:31:04.506 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.506 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.506 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.506 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.506 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.506 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.506 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27766 2024-09-14T05:31:04.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.506 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:31:04.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.507 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211204+0000", 2024-09-14T05:31:04.507 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.507 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211205+0000", 2024-09-14T05:31:04.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7e-07 2024-09-14T05:31:04.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211207+0000", 2024-09-14T05:31:04.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.4540000000000001e-06 2024-09-14T05:31:04.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.508 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.508 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211216+0000", 2024-09-14T05:31:04.508 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.3680000000000002e-06 2024-09-14T05:31:04.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211221+0000", 2024-09-14T05:31:04.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3499999999999996e-06 2024-09-14T05:31:04.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.509 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292329+0000", 2024-09-14T05:31:04.509 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081108240999999998 2024-09-14T05:31:04.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.509 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292370+0000", 2024-09-14T05:31:04.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.121e-05 2024-09-14T05:31:04.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.510 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.510 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292563+0000", 2024-09-14T05:31:04.510 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019298399999999999 2024-09-14T05:31:04.510 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.510 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.510 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.510 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27767 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.510 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:31:04.510 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.278585887999995, 2024-09-14T05:31:04.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081623119999999993, 2024-09-14T05:31:04.511 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.511 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.511 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.511 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.511 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.511 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.511 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27767 2024-09-14T05:31:04.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.511 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.511 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.511 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:31:04.511 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:31:04.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.512 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.512 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211228+0000", 2024-09-14T05:31:04.512 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.0499999999999999e-07 2024-09-14T05:31:04.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211230+0000", 2024-09-14T05:31:04.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.0700000000000001e-06 2024-09-14T05:31:04.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211236+0000", 2024-09-14T05:31:04.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.8429999999999998e-06 2024-09-14T05:31:04.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.513 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.513 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211241+0000", 2024-09-14T05:31:04.513 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7319999999999998e-06 2024-09-14T05:31:04.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292594+0000", 2024-09-14T05:31:04.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081352868999999994 2024-09-14T05:31:04.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292631+0000", 2024-09-14T05:31:04.514 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7083999999999999e-05 2024-09-14T05:31:04.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.514 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.514 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292851+0000", 2024-09-14T05:31:04.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022011700000000001 2024-09-14T05:31:04.515 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.515 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.515 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.515 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27768 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 274432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.515 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:31:04.515 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.278567351999996, 2024-09-14T05:31:04.515 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081875032, 2024-09-14T05:31:04.515 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.515 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.515 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.515 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.516 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.516 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.516 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27768 2024-09-14T05:31:04.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.516 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:31:04.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.516 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.516 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211246+0000", 2024-09-14T05:31:04.516 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211247+0000", 2024-09-14T05:31:04.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.46e-07 2024-09-14T05:31:04.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211248+0000", 2024-09-14T05:31:04.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7349999999999999e-06 2024-09-14T05:31:04.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.517 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.517 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211253+0000", 2024-09-14T05:31:04.517 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.899e-06 2024-09-14T05:31:04.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211258+0000", 2024-09-14T05:31:04.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2810000000000002e-06 2024-09-14T05:31:04.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292888+0000", 2024-09-14T05:31:04.518 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081629955000000004 2024-09-14T05:31:04.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.518 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.518 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.292926+0000", 2024-09-14T05:31:04.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7947000000000001e-05 2024-09-14T05:31:04.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.519 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.519 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293121+0000", 2024-09-14T05:31:04.519 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019586899999999999 2024-09-14T05:31:04.519 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.519 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.519 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.519 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27769 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 294912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.519 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:31:04.519 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.278549263000002, 2024-09-14T05:31:04.520 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082181899000000003, 2024-09-14T05:31:04.520 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.520 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.520 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.520 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.520 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.520 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.520 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27769 2024-09-14T05:31:04.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.520 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.520 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.520 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:31:04.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211264+0000", 2024-09-14T05:31:04.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.521 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.521 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211265+0000", 2024-09-14T05:31:04.521 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2199999999999999e-07 2024-09-14T05:31:04.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.522 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211267+0000", 2024-09-14T05:31:04.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.2239999999999998e-06 2024-09-14T05:31:04.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.522 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211273+0000", 2024-09-14T05:31:04.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.6540000000000004e-06 2024-09-14T05:31:04.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.522 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.522 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211277+0000", 2024-09-14T05:31:04.522 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6299999999999997e-06 2024-09-14T05:31:04.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.523 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293150+0000", 2024-09-14T05:31:04.523 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.081872575000000003 2024-09-14T05:31:04.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.523 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293245+0000", 2024-09-14T05:31:04.523 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 9.5327000000000004e-05 2024-09-14T05:31:04.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.523 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.523 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293446+0000", 2024-09-14T05:31:04.523 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000201067 2024-09-14T05:31:04.524 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.524 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.524 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.524 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27770 3.d 3:b194523f:::rbd_data.112d641bd628.0000000000000003:10 [read 315392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.524 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:31:04.524 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.278529101000004, 2024-09-14T05:31:04.524 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082663707000000003, 2024-09-14T05:31:04.524 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.524 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.524 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.524 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.525 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.525 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.525 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27770 2024-09-14T05:31:04.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.525 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:31:04.525 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.525 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.525 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:31:04.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211285+0000", 2024-09-14T05:31:04.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.1199999999999998e-07 2024-09-14T05:31:04.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.526 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.526 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211292+0000", 2024-09-14T05:31:04.526 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1609999999999997e-06 2024-09-14T05:31:04.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211299+0000", 2024-09-14T05:31:04.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3300000000000004e-06 2024-09-14T05:31:04.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211303+0000", 2024-09-14T05:31:04.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.7859999999999996e-06 2024-09-14T05:31:04.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.527 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.527 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293628+0000", 2024-09-14T05:31:04.527 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082324283999999998 2024-09-14T05:31:04.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293694+0000", 2024-09-14T05:31:04.528 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.6277000000000003e-05 2024-09-14T05:31:04.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.528 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.528 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293948+0000", 2024-09-14T05:31:04.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00025445699999999998 2024-09-14T05:31:04.529 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.529 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.529 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.529 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27790 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.529 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:31:04.529 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.278502696999993, 2024-09-14T05:31:04.529 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082461006000000003, 2024-09-14T05:31:04.529 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.529 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.529 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.530 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.530 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.530 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.530 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27790 2024-09-14T05:31:04.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.530 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.530 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:31:04.530 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.530 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211311+0000", 2024-09-14T05:31:04.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.531 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211312+0000", 2024-09-14T05:31:04.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9299999999999998e-07 2024-09-14T05:31:04.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.531 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.531 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211314+0000", 2024-09-14T05:31:04.531 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.1160000000000002e-06 2024-09-14T05:31:04.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211320+0000", 2024-09-14T05:31:04.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1589999999999998e-06 2024-09-14T05:31:04.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211325+0000", 2024-09-14T05:31:04.532 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9609999999999996e-06 2024-09-14T05:31:04.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.532 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.532 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293474+0000", 2024-09-14T05:31:04.533 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082149080999999999 2024-09-14T05:31:04.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.533 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.533 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293542+0000", 2024-09-14T05:31:04.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8497000000000005e-05 2024-09-14T05:31:04.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.536 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.536 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293772+0000", 2024-09-14T05:31:04.536 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022969900000000001 2024-09-14T05:31:04.537 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.537 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.537 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.537 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27791 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.537 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:31:04.537 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.278482382999997, 2024-09-14T05:31:04.537 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082700838999999998, 2024-09-14T05:31:04.537 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.537 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.537 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.537 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.538 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.538 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.538 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27791 2024-09-14T05:31:04.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.538 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:31:04.538 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.538 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.538 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211331+0000", 2024-09-14T05:31:04.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211332+0000", 2024-09-14T05:31:04.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.7399999999999999e-07 2024-09-14T05:31:04.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.539 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.539 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211334+0000", 2024-09-14T05:31:04.539 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.8500000000000001e-06 2024-09-14T05:31:04.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211340+0000", 2024-09-14T05:31:04.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3010000000000003e-06 2024-09-14T05:31:04.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211345+0000", 2024-09-14T05:31:04.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.938e-06 2024-09-14T05:31:04.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.540 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.540 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293797+0000", 2024-09-14T05:31:04.540 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082452152000000001 2024-09-14T05:31:04.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.293869+0000", 2024-09-14T05:31:04.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2236000000000004e-05 2024-09-14T05:31:04.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.541 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.541 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294032+0000", 2024-09-14T05:31:04.541 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00016298800000000001 2024-09-14T05:31:04.541 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.541 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.541 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.542 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27792 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.542 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:31:04.542 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.278462852000004, 2024-09-14T05:31:04.542 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082897304000000005, 2024-09-14T05:31:04.542 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.542 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.542 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.542 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.542 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.542 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.542 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27792 2024-09-14T05:31:04.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.543 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.543 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.543 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:31:04.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.543 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.543 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:31:04.543 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.543 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211351+0000", 2024-09-14T05:31:04.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.2000000000000001e-07 2024-09-14T05:31:04.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211353+0000", 2024-09-14T05:31:04.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.761e-06 2024-09-14T05:31:04.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.544 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.544 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211359+0000", 2024-09-14T05:31:04.544 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2430000000000002e-06 2024-09-14T05:31:04.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211364+0000", 2024-09-14T05:31:04.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0239999999999997e-06 2024-09-14T05:31:04.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294059+0000", 2024-09-14T05:31:04.545 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082694834999999994 2024-09-14T05:31:04.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.545 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.545 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294114+0000", 2024-09-14T05:31:04.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4823e-05 2024-09-14T05:31:04.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.546 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.546 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294248+0000", 2024-09-14T05:31:04.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013429800000000001 2024-09-14T05:31:04.546 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.546 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.546 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.546 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27793 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.546 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:31:04.546 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.278443053000004, 2024-09-14T05:31:04.546 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083153362999999994, 2024-09-14T05:31:04.547 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.547 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.547 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.547 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.547 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.547 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.547 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27793 2024-09-14T05:31:04.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.547 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.547 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.547 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.547 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:31:04.547 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.548 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.548 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:31:04.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.548 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.548 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211371+0000", 2024-09-14T05:31:04.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5100000000000001e-07 2024-09-14T05:31:04.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.548 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.548 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211373+0000", 2024-09-14T05:31:04.548 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.001e-06 2024-09-14T05:31:04.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211379+0000", 2024-09-14T05:31:04.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.4559999999999997e-06 2024-09-14T05:31:04.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211384+0000", 2024-09-14T05:31:04.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9749999999999998e-06 2024-09-14T05:31:04.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.549 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.549 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294279+0000", 2024-09-14T05:31:04.549 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.082894282999999999 2024-09-14T05:31:04.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294327+0000", 2024-09-14T05:31:04.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8208000000000003e-05 2024-09-14T05:31:04.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.550 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.550 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294524+0000", 2024-09-14T05:31:04.550 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019708900000000001 2024-09-14T05:31:04.550 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.550 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.550 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.551 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27794 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.551 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:31:04.551 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.278422728999999, 2024-09-14T05:31:04.551 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588590000000004, 2024-09-14T05:31:04.551 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.551 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.551 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.551 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.551 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.551 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.551 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27794 2024-09-14T05:31:04.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.551 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:31:04.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:31:04.552 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.552 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.552 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211391+0000", 2024-09-14T05:31:04.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.4499999999999997e-07 2024-09-14T05:31:04.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211393+0000", 2024-09-14T05:31:04.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.815e-06 2024-09-14T05:31:04.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.553 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.553 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211400+0000", 2024-09-14T05:31:04.553 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.2899999999999999e-06 2024-09-14T05:31:04.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211404+0000", 2024-09-14T05:31:04.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:31:04.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.554 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294560+0000", 2024-09-14T05:31:04.554 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083155534000000003 2024-09-14T05:31:04.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.554 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294631+0000", 2024-09-14T05:31:04.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1378999999999995e-05 2024-09-14T05:31:04.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.555 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.555 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.294980+0000", 2024-09-14T05:31:04.555 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00034844900000000002 2024-09-14T05:31:04.555 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.555 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.555 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.555 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27795 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.556 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:31:04.556 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.278403370999996, 2024-09-14T05:31:04.556 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083970367000000004, 2024-09-14T05:31:04.556 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.556 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.556 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.556 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.556 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.556 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.557 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27795 2024-09-14T05:31:04.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.557 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:31:04.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.557 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.557 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211410+0000", 2024-09-14T05:31:04.557 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211411+0000", 2024-09-14T05:31:04.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.5499999999999999e-07 2024-09-14T05:31:04.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211413+0000", 2024-09-14T05:31:04.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:31:04.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.558 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.558 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211419+0000", 2024-09-14T05:31:04.558 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1720000000000004e-06 2024-09-14T05:31:04.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.559 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.559 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211424+0000", 2024-09-14T05:31:04.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.0690000000000001e-06 2024-09-14T05:31:04.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.559 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.559 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295013+0000", 2024-09-14T05:31:04.559 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.083588898999999994 2024-09-14T05:31:04.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.559 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295102+0000", 2024-09-14T05:31:04.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.9462000000000005e-05 2024-09-14T05:31:04.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.560 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.560 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295381+0000", 2024-09-14T05:31:04.560 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.000278566 2024-09-14T05:31:04.560 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.560 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.560 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.560 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27796 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.561 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:31:04.561 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.278383641999994, 2024-09-14T05:31:04.561 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084379107999999994, 2024-09-14T05:31:04.561 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.561 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.561 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.561 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.561 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.561 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.561 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27796 2024-09-14T05:31:04.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.561 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:31:04.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:31:04.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.562 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.562 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211430+0000", 2024-09-14T05:31:04.562 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.2800000000000002e-07 2024-09-14T05:31:04.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211432+0000", 2024-09-14T05:31:04.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.844e-06 2024-09-14T05:31:04.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211439+0000", 2024-09-14T05:31:04.563 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.5239999999999997e-06 2024-09-14T05:31:04.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.563 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.563 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211444+0000", 2024-09-14T05:31:04.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.95e-06 2024-09-14T05:31:04.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.564 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295602+0000", 2024-09-14T05:31:04.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084158017000000002 2024-09-14T05:31:04.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.564 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295674+0000", 2024-09-14T05:31:04.564 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.2385999999999994e-05 2024-09-14T05:31:04.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.564 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.565 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295809+0000", 2024-09-14T05:31:04.565 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00013495900000000001 2024-09-14T05:31:04.565 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.565 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.565 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.565 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27797 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 266240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.565 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:31:04.565 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.278363659999997, 2024-09-14T05:31:04.565 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084898597000000006, 2024-09-14T05:31:04.565 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.565 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.565 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.566 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.566 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.566 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.566 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27797 2024-09-14T05:31:04.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.566 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.566 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:31:04.566 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.566 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.566 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:31:04.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211450+0000", 2024-09-14T05:31:04.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.39e-07 2024-09-14T05:31:04.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211452+0000", 2024-09-14T05:31:04.567 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.708e-06 2024-09-14T05:31:04.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.567 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.567 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211458+0000", 2024-09-14T05:31:04.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.3489999999999997e-06 2024-09-14T05:31:04.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211464+0000", 2024-09-14T05:31:04.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3820000000000003e-06 2024-09-14T05:31:04.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.568 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295839+0000", 2024-09-14T05:31:04.568 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084375055000000004 2024-09-14T05:31:04.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.568 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.295918+0000", 2024-09-14T05:31:04.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.9253999999999997e-05 2024-09-14T05:31:04.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.569 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.569 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296349+0000", 2024-09-14T05:31:04.569 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00043051 2024-09-14T05:31:04.569 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.569 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.569 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.569 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27798 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 286720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:31:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.278343945000003, 2024-09-14T05:31:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085300275999999994, 2024-09-14T05:31:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27798 2024-09-14T05:31:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:31:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:31:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211470+0000", 2024-09-14T05:31:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:31:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211472+0000", 2024-09-14T05:31:04.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.7579999999999999e-06 2024-09-14T05:31:04.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211478+0000", 2024-09-14T05:31:04.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.0310000000000004e-06 2024-09-14T05:31:04.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.572 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.572 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211483+0000", 2024-09-14T05:31:04.572 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.8110000000000002e-06 2024-09-14T05:31:04.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296469+0000", 2024-09-14T05:31:04.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084986575999999994 2024-09-14T05:31:04.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296545+0000", 2024-09-14T05:31:04.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.5749999999999998e-05 2024-09-14T05:31:04.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.573 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.573 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296770+0000", 2024-09-14T05:31:04.573 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00022498900000000001 2024-09-14T05:31:04.573 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.574 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.574 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.574 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27799 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 307200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.574 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:31:04.574 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.278325031999998, 2024-09-14T05:31:04.574 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085718461999999995, 2024-09-14T05:31:04.574 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.574 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.574 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.574 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.574 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.574 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.574 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27799 2024-09-14T05:31:04.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.575 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.575 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.575 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:31:04.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.575 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.575 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:31:04.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.575 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.575 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211489+0000", 2024-09-14T05:31:04.575 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8000000000000001e-07 2024-09-14T05:31:04.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.576 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211491+0000", 2024-09-14T05:31:04.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.75e-06 2024-09-14T05:31:04.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.576 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211497+0000", 2024-09-14T05:31:04.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.857e-06 2024-09-14T05:31:04.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.576 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.576 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.211502+0000", 2024-09-14T05:31:04.576 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 4.9790000000000001e-06 2024-09-14T05:31:04.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296800+0000", 2024-09-14T05:31:04.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.085298099000000002 2024-09-14T05:31:04.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.577 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.296871+0000", 2024-09-14T05:31:04.577 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.0879999999999999e-05 2024-09-14T05:31:04.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.577 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.578 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297207+0000", 2024-09-14T05:31:04.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00033651699999999999 2024-09-14T05:31:04.578 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.578 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.578 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.578 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27826 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.578 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:31:04.578 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.276609910999994, 2024-09-14T05:31:04.578 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084306729999999996, 2024-09-14T05:31:04.578 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.578 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.579 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.579 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.579 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.579 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.579 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27826 2024-09-14T05:31:04.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.579 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.579 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.579 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:31:04.579 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.580 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.580 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213204+0000", 2024-09-14T05:31:04.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.580 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.580 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213205+0000", 2024-09-14T05:31:04.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.277e-06 2024-09-14T05:31:04.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.580 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.580 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213210+0000", 2024-09-14T05:31:04.580 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.3120000000000001e-06 2024-09-14T05:31:04.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.581 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213226+0000", 2024-09-14T05:31:04.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 1.5279000000000001e-05 2024-09-14T05:31:04.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.581 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.581 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213234+0000", 2024-09-14T05:31:04.581 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 8.7020000000000007e-06 2024-09-14T05:31:04.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.581 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297239+0000", 2024-09-14T05:31:04.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084004876000000006 2024-09-14T05:31:04.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297311+0000", 2024-09-14T05:31:04.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 7.1336999999999998e-05 2024-09-14T05:31:04.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.582 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.582 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297511+0000", 2024-09-14T05:31:04.582 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019994699999999999 2024-09-14T05:31:04.582 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.582 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.582 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.583 INFO:teuthology.orchestra.run.smithi071.stdout: "description": "osd_op(client.4397.0:27827 3.1d 3:bceb2812:::rbd_data.112d641bd628.0000000000000006:10 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T05:31:04.583 INFO:teuthology.orchestra.run.smithi071.stdout: "initiated_at": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:31:04.583 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 78.276564723999996, 2024-09-14T05:31:04.583 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084520537000000007, 2024-09-14T05:31:04.583 INFO:teuthology.orchestra.run.smithi071.stdout: "continuous": false, 2024-09-14T05:31:04.583 INFO:teuthology.orchestra.run.smithi071.stdout: "type_data": { 2024-09-14T05:31:04.583 INFO:teuthology.orchestra.run.smithi071.stdout: "flag_point": "started", 2024-09-14T05:31:04.583 INFO:teuthology.orchestra.run.smithi071.stdout: "client_info": { 2024-09-14T05:31:04.583 INFO:teuthology.orchestra.run.smithi071.stdout: "client": "client.4397", 2024-09-14T05:31:04.583 INFO:teuthology.orchestra.run.smithi071.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.583 INFO:teuthology.orchestra.run.smithi071.stdout: "tid": 27827 2024-09-14T05:31:04.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.584 INFO:teuthology.orchestra.run.smithi071.stdout: "events": [ 2024-09-14T05:31:04.584 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.584 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "initiated", 2024-09-14T05:31:04.584 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:31:04.584 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.584 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.584 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "header_read", 2024-09-14T05:31:04.584 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213249+0000", 2024-09-14T05:31:04.584 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0 2024-09-14T05:31:04.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.584 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.584 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "throttled", 2024-09-14T05:31:04.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213250+0000", 2024-09-14T05:31:04.585 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.1099999999999995e-07 2024-09-14T05:31:04.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.585 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "all_read", 2024-09-14T05:31:04.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213252+0000", 2024-09-14T05:31:04.585 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 2.6699999999999998e-06 2024-09-14T05:31:04.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.585 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "dispatched", 2024-09-14T05:31:04.585 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213259+0000", 2024-09-14T05:31:04.585 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 6.8469999999999998e-06 2024-09-14T05:31:04.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.586 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.586 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.213265+0000", 2024-09-14T05:31:04.586 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 5.4940000000000003e-06 2024-09-14T05:31:04.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.586 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "reached_pg", 2024-09-14T05:31:04.586 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297541+0000", 2024-09-14T05:31:04.586 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.084276731999999993 2024-09-14T05:31:04.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.586 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "started", 2024-09-14T05:31:04.586 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297579+0000", 2024-09-14T05:31:04.586 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 3.8108999999999997e-05 2024-09-14T05:31:04.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-14T05:31:04.587 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-14T05:31:04.587 INFO:teuthology.orchestra.run.smithi071.stdout: "event": "done", 2024-09-14T05:31:04.587 INFO:teuthology.orchestra.run.smithi071.stdout: "time": "2024-09-14T05:29:46.297770+0000", 2024-09-14T05:31:04.587 INFO:teuthology.orchestra.run.smithi071.stdout: "duration": 0.00019007400000000001 2024-09-14T05:31:04.587 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.587 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.587 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.587 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-14T05:31:04.587 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-14T05:31:04.587 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:04.589 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T05:31:04.705 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:31:04.706 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:31:04.706 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:31:04.706 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:31:04.715 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T05:31:04.817 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:31:04.817 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:31:04.817 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:31:04.817 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:31:04.817 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:31:04.825 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T05:31:04.929 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:31:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:31:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:31:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:31:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3523 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6062080~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:31:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.413562271000004, 2024-09-14T05:31:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096911030999999995, 2024-09-14T05:31:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3523 2024-09-14T05:31:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:31:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514358+0000", 2024-09-14T05:31:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514359+0000", 2024-09-14T05:31:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9299999999999999e-07 2024-09-14T05:31:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514361+0000", 2024-09-14T05:31:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:31:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514366+0000", 2024-09-14T05:31:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1660000000000002e-06 2024-09-14T05:31:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514371+0000", 2024-09-14T05:31:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.648e-06 2024-09-14T05:31:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.610954+0000", 2024-09-14T05:31:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096582170999999994 2024-09-14T05:31:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611002+0000", 2024-09-14T05:31:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7923999999999997e-05 2024-09-14T05:31:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611269+0000", 2024-09-14T05:31:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026789499999999998 2024-09-14T05:31:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3524 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6553600~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:31:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.413543078999993, 2024-09-14T05:31:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097263347999999999, 2024-09-14T05:31:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3524 2024-09-14T05:31:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:31:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:31:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514378+0000", 2024-09-14T05:31:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:31:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514380+0000", 2024-09-14T05:31:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6500000000000001e-06 2024-09-14T05:31:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514386+0000", 2024-09-14T05:31:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7360000000000004e-06 2024-09-14T05:31:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514391+0000", 2024-09-14T05:31:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2830000000000003e-06 2024-09-14T05:31:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611295+0000", 2024-09-14T05:31:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096904814000000006 2024-09-14T05:31:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611343+0000", 2024-09-14T05:31:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7506999999999998e-05 2024-09-14T05:31:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611641+0000", 2024-09-14T05:31:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029798800000000001 2024-09-14T05:31:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3525 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7045120~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:31:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.413521192000005, 2024-09-14T05:31:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097644422999999994, 2024-09-14T05:31:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3525 2024-09-14T05:31:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:31:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514399+0000", 2024-09-14T05:31:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514400+0000", 2024-09-14T05:31:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.77e-07 2024-09-14T05:31:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514402+0000", 2024-09-14T05:31:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6530000000000001e-06 2024-09-14T05:31:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514411+0000", 2024-09-14T05:31:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7429999999999997e-06 2024-09-14T05:31:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514417+0000", 2024-09-14T05:31:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2249999999999997e-06 2024-09-14T05:31:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611668+0000", 2024-09-14T05:31:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097250681000000005 2024-09-14T05:31:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.611716+0000", 2024-09-14T05:31:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7336000000000002e-05 2024-09-14T05:31:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612044+0000", 2024-09-14T05:31:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032840799999999998 2024-09-14T05:31:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3526 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7536640~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:31:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.413496707999997, 2024-09-14T05:31:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097943884999999994, 2024-09-14T05:31:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3526 2024-09-14T05:31:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:31:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:04.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:31:04.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:04.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514424+0000", 2024-09-14T05:31:04.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0400000000000002e-07 2024-09-14T05:31:04.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:04.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514426+0000", 2024-09-14T05:31:04.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-14T05:31:04.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:04.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514432+0000", 2024-09-14T05:31:04.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.105e-06 2024-09-14T05:31:04.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514438+0000", 2024-09-14T05:31:04.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5149999999999997e-06 2024-09-14T05:31:04.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:04.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612072+0000", 2024-09-14T05:31:04.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097633947999999998 2024-09-14T05:31:04.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:04.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612120+0000", 2024-09-14T05:31:04.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8458999999999998e-05 2024-09-14T05:31:04.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:04.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612368+0000", 2024-09-14T05:31:04.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024761999999999999 2024-09-14T05:31:04.950 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.950 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:04.951 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.951 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3527 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 8028160~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:04.951 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:31:04.951 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.413475994999999, 2024-09-14T05:31:04.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098244955999999994, 2024-09-14T05:31:04.951 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:04.951 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:04.951 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:04.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:04.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:04.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.951 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3527 2024-09-14T05:31:04.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.952 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:04.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:04.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:31:04.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:04.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.514445+0000", 2024-09-14T05:31:04.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:04.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519842+0000", 2024-09-14T05:31:04.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053975230000000004 2024-09-14T05:31:04.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:04.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519851+0000", 2024-09-14T05:31:04.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5599999999999994e-06 2024-09-14T05:31:04.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:04.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519858+0000", 2024-09-14T05:31:04.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2409999999999998e-06 2024-09-14T05:31:04.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519863+0000", 2024-09-14T05:31:04.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-14T05:31:04.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:04.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612395+0000", 2024-09-14T05:31:04.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092532552000000004 2024-09-14T05:31:04.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:04.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612442+0000", 2024-09-14T05:31:04.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6486000000000002e-05 2024-09-14T05:31:04.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:04.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.612690+0000", 2024-09-14T05:31:04.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024791599999999999 2024-09-14T05:31:04.955 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.955 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:04.955 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.955 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3603 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5177344~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:04.955 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:31:04.956 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.407923550000007, 2024-09-14T05:31:04.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096796436, 2024-09-14T05:31:04.956 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:04.956 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:04.956 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:04.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:04.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:04.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.956 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3603 2024-09-14T05:31:04.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.956 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:04.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:04.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:31:04.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:04.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:31:04.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:04.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519997+0000", 2024-09-14T05:31:04.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1899999999999999e-07 2024-09-14T05:31:04.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:04.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.519998+0000", 2024-09-14T05:31:04.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5000000000000001e-07 2024-09-14T05:31:04.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:04.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520002+0000", 2024-09-14T05:31:04.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7299999999999999e-06 2024-09-14T05:31:04.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520004+0000", 2024-09-14T05:31:04.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.429e-06 2024-09-14T05:31:04.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:04.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616493+0000", 2024-09-14T05:31:04.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096488345000000003 2024-09-14T05:31:04.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:04.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616544+0000", 2024-09-14T05:31:04.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1724000000000001e-05 2024-09-14T05:31:04.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:04.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616794+0000", 2024-09-14T05:31:04.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000249139 2024-09-14T05:31:04.959 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.960 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:04.960 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.960 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3604 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5668864~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:04.960 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:31:04.960 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.407908911999996, 2024-09-14T05:31:04.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108222999999994, 2024-09-14T05:31:04.960 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:04.960 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:04.960 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:04.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:04.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:04.961 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.961 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3604 2024-09-14T05:31:04.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.961 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:04.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:04.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:31:04.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:04.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:31:04.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:04.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520012+0000", 2024-09-14T05:31:04.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7000000000000001e-07 2024-09-14T05:31:04.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:04.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520013+0000", 2024-09-14T05:31:04.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.389e-06 2024-09-14T05:31:04.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:04.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520018+0000", 2024-09-14T05:31:04.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3019999999999996e-06 2024-09-14T05:31:04.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520026+0000", 2024-09-14T05:31:04.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6170000000000007e-06 2024-09-14T05:31:04.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:04.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616816+0000", 2024-09-14T05:31:04.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096789766999999999 2024-09-14T05:31:04.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:04.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.616871+0000", 2024-09-14T05:31:04.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4811000000000001e-05 2024-09-14T05:31:04.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:04.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617120+0000", 2024-09-14T05:31:04.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024906700000000001 2024-09-14T05:31:04.964 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.964 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:04.964 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.965 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3605 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6160384~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:04.965 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:31:04.965 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.407890452000004, 2024-09-14T05:31:04.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097423945999999997, 2024-09-14T05:31:04.965 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:04.965 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:04.965 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:04.965 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:04.965 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:04.965 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.965 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3605 2024-09-14T05:31:04.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.965 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:04.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:04.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:31:04.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:04.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:31:04.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:04.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520030+0000", 2024-09-14T05:31:04.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3999999999999998e-07 2024-09-14T05:31:04.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:04.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520032+0000", 2024-09-14T05:31:04.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.083e-06 2024-09-14T05:31:04.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:04.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520036+0000", 2024-09-14T05:31:04.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.019e-06 2024-09-14T05:31:04.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520038+0000", 2024-09-14T05:31:04.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4849999999999999e-06 2024-09-14T05:31:04.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:04.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617147+0000", 2024-09-14T05:31:04.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097108665999999996 2024-09-14T05:31:04.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:04.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617199+0000", 2024-09-14T05:31:04.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2558000000000001e-05 2024-09-14T05:31:04.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:04.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617454+0000", 2024-09-14T05:31:04.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025489499999999999 2024-09-14T05:31:04.969 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.969 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:04.969 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.969 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3606 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6651904~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:04.969 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:31:04.969 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.407879730000005, 2024-09-14T05:31:04.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097745705000000002, 2024-09-14T05:31:04.970 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:04.970 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:04.970 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:04.970 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:04.970 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:04.970 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.970 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3606 2024-09-14T05:31:04.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.970 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:04.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:04.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:31:04.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:04.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:31:04.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:04.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520041+0000", 2024-09-14T05:31:04.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-14T05:31:04.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:04.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520042+0000", 2024-09-14T05:31:04.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4799999999999997e-07 2024-09-14T05:31:04.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:04.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520046+0000", 2024-09-14T05:31:04.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8149999999999999e-06 2024-09-14T05:31:04.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520049+0000", 2024-09-14T05:31:04.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.576e-06 2024-09-14T05:31:04.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:04.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617476+0000", 2024-09-14T05:31:04.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097427558999999997 2024-09-14T05:31:04.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:04.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617522+0000", 2024-09-14T05:31:04.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5927999999999998e-05 2024-09-14T05:31:04.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:04.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617787+0000", 2024-09-14T05:31:04.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000264663 2024-09-14T05:31:04.973 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.974 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:04.974 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.974 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3607 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7143424~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:04.974 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:31:04.974 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.407869203000004, 2024-09-14T05:31:04.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097871125000000003, 2024-09-14T05:31:04.974 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:04.974 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:04.974 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:04.974 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:04.974 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:04.974 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.975 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3607 2024-09-14T05:31:04.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.975 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:04.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:04.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:31:04.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:04.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520051+0000", 2024-09-14T05:31:04.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:04.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520052+0000", 2024-09-14T05:31:04.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.03e-07 2024-09-14T05:31:04.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:04.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520053+0000", 2024-09-14T05:31:04.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.85e-07 2024-09-14T05:31:04.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:04.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520056+0000", 2024-09-14T05:31:04.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9140000000000001e-06 2024-09-14T05:31:04.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520059+0000", 2024-09-14T05:31:04.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4600000000000002e-06 2024-09-14T05:31:04.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:04.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617809+0000", 2024-09-14T05:31:04.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097750216000000001 2024-09-14T05:31:04.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:04.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617864+0000", 2024-09-14T05:31:04.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5211999999999999e-05 2024-09-14T05:31:04.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:04.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617923+0000", 2024-09-14T05:31:04.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8235000000000001e-05 2024-09-14T05:31:04.978 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.978 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:04.978 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.978 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3608 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 7634944~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:04.978 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:31:04.978 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.407858895000004, 2024-09-14T05:31:04.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.098199212999999994, 2024-09-14T05:31:04.979 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:04.979 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:04.979 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:04.979 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:04.979 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:04.979 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.979 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3608 2024-09-14T05:31:04.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.979 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:04.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:04.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:31:04.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.520062+0000", 2024-09-14T05:31:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525953+0000", 2024-09-14T05:31:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058911850000000002 2024-09-14T05:31:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525957+0000", 2024-09-14T05:31:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2379999999999999e-06 2024-09-14T05:31:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525967+0000", 2024-09-14T05:31:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5389999999999996e-06 2024-09-14T05:31:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.525972+0000", 2024-09-14T05:31:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.541e-06 2024-09-14T05:31:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617945+0000", 2024-09-14T05:31:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091972597000000003 2024-09-14T05:31:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.617992+0000", 2024-09-14T05:31:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7397e-05 2024-09-14T05:31:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.618261+0000", 2024-09-14T05:31:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026871600000000001 2024-09-14T05:31:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3640 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 6586368~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:31:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.401751052999998, 2024-09-14T05:31:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097068580000000002, 2024-09-14T05:31:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3640 2024-09-14T05:31:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:31:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:31:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526170+0000", 2024-09-14T05:31:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.23e-07 2024-09-14T05:31:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526171+0000", 2024-09-14T05:31:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-14T05:31:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526175+0000", 2024-09-14T05:31:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8369999999999999e-06 2024-09-14T05:31:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526177+0000", 2024-09-14T05:31:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4959999999999999e-06 2024-09-14T05:31:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622847+0000", 2024-09-14T05:31:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096669790000000005 2024-09-14T05:31:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.622902+0000", 2024-09-14T05:31:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5157000000000001e-05 2024-09-14T05:31:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623238+0000", 2024-09-14T05:31:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033603399999999997 2024-09-14T05:31:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3641 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7077888~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:31:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.401740543000003, 2024-09-14T05:31:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097371141999999994, 2024-09-14T05:31:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3641 2024-09-14T05:31:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:31:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.526180+0000", 2024-09-14T05:31:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531851+0000", 2024-09-14T05:31:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005671265 2024-09-14T05:31:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531855+0000", 2024-09-14T05:31:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0849999999999999e-06 2024-09-14T05:31:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531867+0000", 2024-09-14T05:31:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1028e-05 2024-09-14T05:31:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.531872+0000", 2024-09-14T05:31:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000003e-06 2024-09-14T05:31:04.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:04.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623251+0000", 2024-09-14T05:31:04.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091378980999999998 2024-09-14T05:31:04.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:04.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623295+0000", 2024-09-14T05:31:04.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3881999999999998e-05 2024-09-14T05:31:04.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:04.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.623551+0000", 2024-09-14T05:31:04.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025636099999999998 2024-09-14T05:31:04.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.992 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:04.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.992 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3721 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:04.992 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:31:04.992 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.395882267999994, 2024-09-14T05:31:04.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096806472000000005, 2024-09-14T05:31:04.992 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:04.992 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:04.992 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:04.992 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:04.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:04.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.993 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3721 2024-09-14T05:31:04.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.993 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:04.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:04.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:31:04.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:04.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532038+0000", 2024-09-14T05:31:04.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:04.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532039+0000", 2024-09-14T05:31:04.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.05e-07 2024-09-14T05:31:04.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:04.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532040+0000", 2024-09-14T05:31:04.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9099999999999991e-07 2024-09-14T05:31:04.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:04.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532043+0000", 2024-09-14T05:31:04.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6610000000000002e-06 2024-09-14T05:31:04.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532046+0000", 2024-09-14T05:31:04.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4080000000000001e-06 2024-09-14T05:31:04.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:04.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628545+0000", 2024-09-14T05:31:04.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096499615999999996 2024-09-14T05:31:04.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:04.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628592+0000", 2024-09-14T05:31:04.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6907000000000003e-05 2024-09-14T05:31:04.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:04.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628845+0000", 2024-09-14T05:31:04.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025268399999999998 2024-09-14T05:31:04.996 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.996 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:04.996 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:04.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.996 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3722 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6193152~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:04.997 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:31:04.997 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.395872187999998, 2024-09-14T05:31:04.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097146587000000006, 2024-09-14T05:31:04.997 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:04.997 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:04.997 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:04.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:04.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:04.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:04.997 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3722 2024-09-14T05:31:04.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.997 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:04.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:04.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:31:04.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:04.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.532048+0000", 2024-09-14T05:31:04.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:04.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:04.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538191+0000", 2024-09-14T05:31:04.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0061429969999999999 2024-09-14T05:31:04.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:04.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538200+0000", 2024-09-14T05:31:04.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7069999999999998e-06 2024-09-14T05:31:04.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:04.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538212+0000", 2024-09-14T05:31:04.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1504e-05 2024-09-14T05:31:04.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:04.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:04.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:04.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538219+0000", 2024-09-14T05:31:04.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7500000000000003e-06 2024-09-14T05:31:05.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628869+0000", 2024-09-14T05:31:05.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090649224 2024-09-14T05:31:05.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.628919+0000", 2024-09-14T05:31:05.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0144e-05 2024-09-14T05:31:05.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.629195+0000", 2024-09-14T05:31:05.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027626099999999998 2024-09-14T05:31:05.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.001 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.001 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3758 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7110656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.001 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:31:05.001 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.389277972000002, 2024-09-14T05:31:05.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096977544999999998, 2024-09-14T05:31:05.001 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.001 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.001 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.002 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3758 2024-09-14T05:31:05.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.002 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:31:05.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:31:05.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538643+0000", 2024-09-14T05:31:05.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-14T05:31:05.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538645+0000", 2024-09-14T05:31:05.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.843e-06 2024-09-14T05:31:05.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538651+0000", 2024-09-14T05:31:05.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6169999999999998e-06 2024-09-14T05:31:05.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538655+0000", 2024-09-14T05:31:05.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5399999999999997e-06 2024-09-14T05:31:05.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635225+0000", 2024-09-14T05:31:05.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096570149999999993 2024-09-14T05:31:05.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635272+0000", 2024-09-14T05:31:05.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7199e-05 2024-09-14T05:31:05.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635620+0000", 2024-09-14T05:31:05.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034782600000000003 2024-09-14T05:31:05.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.005 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.006 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3759 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 7602176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.006 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:31:05.006 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.389260061000002, 2024-09-14T05:31:05.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097335331999999997, 2024-09-14T05:31:05.006 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.006 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.006 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.006 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3759 2024-09-14T05:31:05.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.007 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:31:05.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.538661+0000", 2024-09-14T05:31:05.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544013+0000", 2024-09-14T05:31:05.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0053520829999999997 2024-09-14T05:31:05.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544016+0000", 2024-09-14T05:31:05.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6660000000000001e-06 2024-09-14T05:31:05.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544026+0000", 2024-09-14T05:31:05.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7089999999999997e-06 2024-09-14T05:31:05.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544043+0000", 2024-09-14T05:31:05.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833999999999998e-05 2024-09-14T05:31:05.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635646+0000", 2024-09-14T05:31:05.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091603544999999995 2024-09-14T05:31:05.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635696+0000", 2024-09-14T05:31:05.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9307000000000001e-05 2024-09-14T05:31:05.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.635996+0000", 2024-09-14T05:31:05.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030018800000000001 2024-09-14T05:31:05.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.010 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.010 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3838 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6225920~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.010 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:31:05.010 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.383598380999999, 2024-09-14T05:31:05.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096894235999999995, 2024-09-14T05:31:05.011 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.011 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.011 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.011 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3838 2024-09-14T05:31:05.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.011 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:31:05.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544322+0000", 2024-09-14T05:31:05.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544323+0000", 2024-09-14T05:31:05.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-14T05:31:05.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544325+0000", 2024-09-14T05:31:05.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.75e-06 2024-09-14T05:31:05.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544331+0000", 2024-09-14T05:31:05.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5019999999999998e-06 2024-09-14T05:31:05.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544335+0000", 2024-09-14T05:31:05.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2989999999999998e-06 2024-09-14T05:31:05.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640869+0000", 2024-09-14T05:31:05.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096533351000000003 2024-09-14T05:31:05.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.640919+0000", 2024-09-14T05:31:05.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999999e-05 2024-09-14T05:31:05.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641217+0000", 2024-09-14T05:31:05.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029717600000000002 2024-09-14T05:31:05.015 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.015 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.015 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.015 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3839 3.1b 3:dec66935:::rbd_data.112d1f36d097.0000000000000001:6 [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.015 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:31:05.015 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.383579443000002, 2024-09-14T05:31:05.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.097240339999999995, 2024-09-14T05:31:05.015 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.015 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.015 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.016 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.016 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.016 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3839 2024-09-14T05:31:05.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.016 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:31:05.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.544341+0000", 2024-09-14T05:31:05.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549834+0000", 2024-09-14T05:31:05.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054927819999999999 2024-09-14T05:31:05.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549838+0000", 2024-09-14T05:31:05.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8689999999999997e-06 2024-09-14T05:31:05.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549848+0000", 2024-09-14T05:31:05.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0154e-05 2024-09-14T05:31:05.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.549853+0000", 2024-09-14T05:31:05.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9740000000000001e-06 2024-09-14T05:31:05.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641245+0000", 2024-09-14T05:31:05.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091392245999999996 2024-09-14T05:31:05.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641295+0000", 2024-09-14T05:31:05.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0018999999999997e-05 2024-09-14T05:31:05.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.641582+0000", 2024-09-14T05:31:05.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028629600000000001 2024-09-14T05:31:05.019 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.019 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.019 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.019 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3868 3.1b 3:dfbb9029:::rbd_data.112d1f36d097.0000000000000003:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.020 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:31:05.020 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.377916865000003, 2024-09-14T05:31:05.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096859555999999999, 2024-09-14T05:31:05.020 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.020 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.020 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.020 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.020 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.020 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.020 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3868 2024-09-14T05:31:05.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.020 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:31:05.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.550004+0000", 2024-09-14T05:31:05.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556173+0000", 2024-09-14T05:31:05.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006169038 2024-09-14T05:31:05.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556177+0000", 2024-09-14T05:31:05.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0010000000000003e-06 2024-09-14T05:31:05.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556188+0000", 2024-09-14T05:31:05.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0675e-05 2024-09-14T05:31:05.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.556199+0000", 2024-09-14T05:31:05.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.155e-05 2024-09-14T05:31:05.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646511+0000", 2024-09-14T05:31:05.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090312187000000002 2024-09-14T05:31:05.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646563+0000", 2024-09-14T05:31:05.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2219000000000003e-05 2024-09-14T05:31:05.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.646863+0000", 2024-09-14T05:31:05.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029988600000000002 2024-09-14T05:31:05.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.024 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.024 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.024 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:31:05.024 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-14T05:31:05.025 INFO:teuthology.orchestra.run.smithi071.stderr:noscrub is unset 2024-09-14T05:31:05.026 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T05:31:05.040 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T05:31:05.131 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:31:05.132 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:31:05.132 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:31:05.132 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:31:05.140 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T05:31:05.141 INFO:teuthology.orchestra.run.smithi071.stdout:mon.a: {} 2024-09-14T05:31:05.141 INFO:teuthology.orchestra.run.smithi071.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T05:31:05.150 INFO:teuthology.orchestra.run.smithi071.stdout:mon.b: {} 2024-09-14T05:31:05.150 INFO:teuthology.orchestra.run.smithi071.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T05:31:05.159 INFO:teuthology.orchestra.run.smithi071.stdout:mon.c: {} 2024-09-14T05:31:05.159 INFO:teuthology.orchestra.run.smithi071.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T05:31:05.170 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T05:31:05.245 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:31:05.245 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:31:05.245 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:31:05.245 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:31:05.245 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:31:05.253 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T05:31:05.358 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:31:05.358 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:31:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:31:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:31:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1989 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 16326080~54496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:31:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:31:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.795548593000007, 2024-09-14T05:31:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.096664322999999996, 2024-09-14T05:31:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1989 2024-09-14T05:31:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:31:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561454+0000", 2024-09-14T05:31:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.561455+0000", 2024-09-14T05:31:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1699999999999997e-07 2024-09-14T05:31:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590703+0000", 2024-09-14T05:31:05.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029248225999999999 2024-09-14T05:31:05.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590732+0000", 2024-09-14T05:31:05.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8679000000000001e-05 2024-09-14T05:31:05.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590759+0000", 2024-09-14T05:31:05.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7319999999999999e-05 2024-09-14T05:31:05.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590835+0000", 2024-09-14T05:31:05.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5543999999999994e-05 2024-09-14T05:31:05.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.590956+0000", 2024-09-14T05:31:05.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012168 2024-09-14T05:31:05.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 1", 2024-09-14T05:31:05.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.591261+0000", 2024-09-14T05:31:05.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030484799999999998 2024-09-14T05:31:05.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:31:05.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.637682+0000", 2024-09-14T05:31:05.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046420550999999997 2024-09-14T05:31:05.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:31:05.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.657912+0000", 2024-09-14T05:31:05.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020230011999999999 2024-09-14T05:31:05.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658007+0000", 2024-09-14T05:31:05.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5814000000000001e-05 2024-09-14T05:31:05.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.658118+0000", 2024-09-14T05:31:05.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110832 2024-09-14T05:31:05.365 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.365 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.366 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.366 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2033 3.1e 3:7e47f6eb:::rbd_data.112d1f36d097.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30853661~63531] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:31:05.366 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:31:05.366 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.684273927000007, 2024-09-14T05:31:05.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065167630000000004, 2024-09-14T05:31:05.366 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.366 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.366 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.366 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2033 2024-09-14T05:31:05.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.367 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:31:05.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672729+0000", 2024-09-14T05:31:05.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.672730+0000", 2024-09-14T05:31:05.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.393e-06 2024-09-14T05:31:05.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691289+0000", 2024-09-14T05:31:05.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018558991 2024-09-14T05:31:05.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691316+0000", 2024-09-14T05:31:05.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7498999999999999e-05 2024-09-14T05:31:05.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691331+0000", 2024-09-14T05:31:05.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4652e-05 2024-09-14T05:31:05.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691397+0000", 2024-09-14T05:31:05.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6112000000000007e-05 2024-09-14T05:31:05.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.691509+0000", 2024-09-14T05:31:05.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111599 2024-09-14T05:31:05.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:31:05.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.692253+0000", 2024-09-14T05:31:05.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074456200000000004 2024-09-14T05:31:05.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:31:05.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.719140+0000", 2024-09-14T05:31:05.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026886542999999999 2024-09-14T05:31:05.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:31:05.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737789+0000", 2024-09-14T05:31:05.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018649203999999999 2024-09-14T05:31:05.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737846+0000", 2024-09-14T05:31:05.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6501e-05 2024-09-14T05:31:05.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.737896+0000", 2024-09-14T05:31:05.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0574000000000001e-05 2024-09-14T05:31:05.371 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.371 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.371 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.372 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3008 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13729792~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.372 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:31:05.372 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.053350057000003, 2024-09-14T05:31:05.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061007651000000003, 2024-09-14T05:31:05.372 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.372 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.372 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.372 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3008 2024-09-14T05:31:05.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.372 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:31:05.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303652+0000", 2024-09-14T05:31:05.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303653+0000", 2024-09-14T05:31:05.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0699999999999999e-07 2024-09-14T05:31:05.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303654+0000", 2024-09-14T05:31:05.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-07 2024-09-14T05:31:05.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303657+0000", 2024-09-14T05:31:05.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2890000000000002e-06 2024-09-14T05:31:05.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303661+0000", 2024-09-14T05:31:05.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3290000000000004e-06 2024-09-14T05:31:05.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364233+0000", 2024-09-14T05:31:05.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060572134 2024-09-14T05:31:05.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364315+0000", 2024-09-14T05:31:05.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1549000000000001e-05 2024-09-14T05:31:05.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364660+0000", 2024-09-14T05:31:05.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034525300000000001 2024-09-14T05:31:05.376 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.376 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.376 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.376 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3009 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14221312~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.376 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:31:05.376 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.053335966999995, 2024-09-14T05:31:05.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061445063000000001, 2024-09-14T05:31:05.376 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.376 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.376 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.377 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3009 2024-09-14T05:31:05.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.377 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:31:05.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:31:05.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303667+0000", 2024-09-14T05:31:05.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-14T05:31:05.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303669+0000", 2024-09-14T05:31:05.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7150000000000001e-06 2024-09-14T05:31:05.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303675+0000", 2024-09-14T05:31:05.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5339999999999996e-06 2024-09-14T05:31:05.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303681+0000", 2024-09-14T05:31:05.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3639999999999998e-06 2024-09-14T05:31:05.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364705+0000", 2024-09-14T05:31:05.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061024123 2024-09-14T05:31:05.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.364786+0000", 2024-09-14T05:31:05.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1378999999999995e-05 2024-09-14T05:31:05.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365112+0000", 2024-09-14T05:31:05.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032558700000000001 2024-09-14T05:31:05.380 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.380 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.380 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.381 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3010 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14712832~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.381 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:31:05.381 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.053315103000003, 2024-09-14T05:31:05.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061834630000000002, 2024-09-14T05:31:05.381 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.381 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.381 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.381 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3010 2024-09-14T05:31:05.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.381 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:31:05.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303687+0000", 2024-09-14T05:31:05.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303688+0000", 2024-09-14T05:31:05.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:31:05.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303690+0000", 2024-09-14T05:31:05.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9460000000000001e-06 2024-09-14T05:31:05.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303696+0000", 2024-09-14T05:31:05.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2230000000000004e-06 2024-09-14T05:31:05.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.303700+0000", 2024-09-14T05:31:05.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7290000000000002e-06 2024-09-14T05:31:05.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365163+0000", 2024-09-14T05:31:05.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061463093000000003 2024-09-14T05:31:05.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365214+0000", 2024-09-14T05:31:05.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0977999999999999e-05 2024-09-14T05:31:05.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.365522+0000", 2024-09-14T05:31:05.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000308227 2024-09-14T05:31:05.385 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.385 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.385 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.385 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3241 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.385 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:31:05.385 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.027640680999994, 2024-09-14T05:31:05.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061041892, 2024-09-14T05:31:05.386 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.386 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.386 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.386 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.386 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.386 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.386 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3241 2024-09-14T05:31:05.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.386 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:31:05.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329362+0000", 2024-09-14T05:31:05.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329496+0000", 2024-09-14T05:31:05.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013421799999999999 2024-09-14T05:31:05.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329499+0000", 2024-09-14T05:31:05.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8459999999999999e-06 2024-09-14T05:31:05.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329505+0000", 2024-09-14T05:31:05.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4320000000000004e-06 2024-09-14T05:31:05.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329509+0000", 2024-09-14T05:31:05.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0339999999999998e-06 2024-09-14T05:31:05.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.389947+0000", 2024-09-14T05:31:05.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060438093999999998 2024-09-14T05:31:05.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390038+0000", 2024-09-14T05:31:05.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0253999999999993e-05 2024-09-14T05:31:05.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390404+0000", 2024-09-14T05:31:05.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036601400000000001 2024-09-14T05:31:05.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.390 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.390 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3242 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.390 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:31:05.390 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.027485772999995, 2024-09-14T05:31:05.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061282426000000001, 2024-09-14T05:31:05.390 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.390 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.390 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.390 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.390 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.390 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.391 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3242 2024-09-14T05:31:05.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.391 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:31:05.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:31:05.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329517+0000", 2024-09-14T05:31:05.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9900000000000001e-07 2024-09-14T05:31:05.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329518+0000", 2024-09-14T05:31:05.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.328e-06 2024-09-14T05:31:05.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329522+0000", 2024-09-14T05:31:05.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.563e-06 2024-09-14T05:31:05.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329525+0000", 2024-09-14T05:31:05.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7379999999999999e-06 2024-09-14T05:31:05.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390437+0000", 2024-09-14T05:31:05.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060912463 2024-09-14T05:31:05.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390498+0000", 2024-09-14T05:31:05.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0615999999999997e-05 2024-09-14T05:31:05.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390799+0000", 2024-09-14T05:31:05.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030131899999999998 2024-09-14T05:31:05.394 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.394 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.394 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.394 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3243 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.394 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:31:05.394 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.027473986999993, 2024-09-14T05:31:05.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061728980000000003, 2024-09-14T05:31:05.395 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.395 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.395 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.395 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3243 2024-09-14T05:31:05.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.395 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:31:05.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329529+0000", 2024-09-14T05:31:05.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329692+0000", 2024-09-14T05:31:05.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000163948 2024-09-14T05:31:05.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329696+0000", 2024-09-14T05:31:05.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5540000000000002e-06 2024-09-14T05:31:05.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329706+0000", 2024-09-14T05:31:05.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7960000000000007e-06 2024-09-14T05:31:05.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329712+0000", 2024-09-14T05:31:05.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6109999999999999e-06 2024-09-14T05:31:05.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390861+0000", 2024-09-14T05:31:05.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061148767 2024-09-14T05:31:05.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.390942+0000", 2024-09-14T05:31:05.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1266000000000004e-05 2024-09-14T05:31:05.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391258+0000", 2024-09-14T05:31:05.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031503800000000002 2024-09-14T05:31:05.399 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.399 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.399 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.399 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3244 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14778368~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.399 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:31:05.399 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.027278938999999, 2024-09-14T05:31:05.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062050594000000001, 2024-09-14T05:31:05.399 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.399 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.400 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.400 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3244 2024-09-14T05:31:05.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.400 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:31:05.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329724+0000", 2024-09-14T05:31:05.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329916+0000", 2024-09-14T05:31:05.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019290799999999999 2024-09-14T05:31:05.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329919+0000", 2024-09-14T05:31:05.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9019999999999999e-06 2024-09-14T05:31:05.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329926+0000", 2024-09-14T05:31:05.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9e-06 2024-09-14T05:31:05.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.329932+0000", 2024-09-14T05:31:05.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.088e-06 2024-09-14T05:31:05.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391299+0000", 2024-09-14T05:31:05.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061366136000000002 2024-09-14T05:31:05.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391391+0000", 2024-09-14T05:31:05.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2204e-05 2024-09-14T05:31:05.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391774+0000", 2024-09-14T05:31:05.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038345600000000001 2024-09-14T05:31:05.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.403 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.404 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3921 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 5308416~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.404 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:31:05.404 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.955735129000004, 2024-09-14T05:31:05.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060842694000000003, 2024-09-14T05:31:05.404 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.404 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.404 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.404 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3921 2024-09-14T05:31:05.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.405 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:31:05.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401267+0000", 2024-09-14T05:31:05.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401705+0000", 2024-09-14T05:31:05.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768499999999999 2024-09-14T05:31:05.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401709+0000", 2024-09-14T05:31:05.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0319999999999997e-06 2024-09-14T05:31:05.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401720+0000", 2024-09-14T05:31:05.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1005000000000001e-05 2024-09-14T05:31:05.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401727+0000", 2024-09-14T05:31:05.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1879999999999996e-06 2024-09-14T05:31:05.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461748+0000", 2024-09-14T05:31:05.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060021199999999997 2024-09-14T05:31:05.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.461793+0000", 2024-09-14T05:31:05.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4530999999999998e-05 2024-09-14T05:31:05.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462110+0000", 2024-09-14T05:31:05.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031705299999999997 2024-09-14T05:31:05.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.408 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.408 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3923 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 6291456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.408 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:31:05.408 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.955089100999999, 2024-09-14T05:31:05.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060891273000000003, 2024-09-14T05:31:05.409 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.409 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.409 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.409 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3923 2024-09-14T05:31:05.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.409 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:31:05.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401913+0000", 2024-09-14T05:31:05.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407054+0000", 2024-09-14T05:31:05.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005140787 2024-09-14T05:31:05.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407059+0000", 2024-09-14T05:31:05.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.04e-06 2024-09-14T05:31:05.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407069+0000", 2024-09-14T05:31:05.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4429999999999992e-06 2024-09-14T05:31:05.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407076+0000", 2024-09-14T05:31:05.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1679999999999998e-06 2024-09-14T05:31:05.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462514+0000", 2024-09-14T05:31:05.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055438551000000003 2024-09-14T05:31:05.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462562+0000", 2024-09-14T05:31:05.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7649999999999999e-05 2024-09-14T05:31:05.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.462805+0000", 2024-09-14T05:31:05.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024263400000000001 2024-09-14T05:31:05.412 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.413 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.413 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4052 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 12550144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.413 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:31:05.413 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.939517092000003, 2024-09-14T05:31:05.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060872904999999998, 2024-09-14T05:31:05.413 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.413 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.413 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.414 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4052 2024-09-14T05:31:05.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.414 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:31:05.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417485+0000", 2024-09-14T05:31:05.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417486+0000", 2024-09-14T05:31:05.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-14T05:31:05.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417487+0000", 2024-09-14T05:31:05.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4499999999999995e-07 2024-09-14T05:31:05.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417489+0000", 2024-09-14T05:31:05.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8899999999999999e-06 2024-09-14T05:31:05.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417492+0000", 2024-09-14T05:31:05.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4389999999999999e-06 2024-09-14T05:31:05.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478027+0000", 2024-09-14T05:31:05.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060534959999999999 2024-09-14T05:31:05.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478079+0000", 2024-09-14T05:31:05.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2104999999999997e-05 2024-09-14T05:31:05.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478358+0000", 2024-09-14T05:31:05.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000279351 2024-09-14T05:31:05.417 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.417 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.417 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.417 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4053 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13041664~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.417 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:31:05.418 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.939507707999994, 2024-09-14T05:31:05.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061258866000000002, 2024-09-14T05:31:05.418 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.418 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.418 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.418 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4053 2024-09-14T05:31:05.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.418 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:31:05.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:31:05.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417495+0000", 2024-09-14T05:31:05.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.17e-07 2024-09-14T05:31:05.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417496+0000", 2024-09-14T05:31:05.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0890000000000001e-06 2024-09-14T05:31:05.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417499+0000", 2024-09-14T05:31:05.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8629999999999999e-06 2024-09-14T05:31:05.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417502+0000", 2024-09-14T05:31:05.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6290000000000002e-06 2024-09-14T05:31:05.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478387+0000", 2024-09-14T05:31:05.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060885433000000003 2024-09-14T05:31:05.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478435+0000", 2024-09-14T05:31:05.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8167999999999997e-05 2024-09-14T05:31:05.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478754+0000", 2024-09-14T05:31:05.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031846700000000001 2024-09-14T05:31:05.422 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.422 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.422 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.422 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4054 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 13533184~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.422 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:31:05.422 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.939497824, 2024-09-14T05:31:05.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061604694000000002, 2024-09-14T05:31:05.422 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.422 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.422 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.423 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.423 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4054 2024-09-14T05:31:05.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.423 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:31:05.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:31:05.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417505+0000", 2024-09-14T05:31:05.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.04e-07 2024-09-14T05:31:05.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417506+0000", 2024-09-14T05:31:05.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8299999999999995e-07 2024-09-14T05:31:05.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417509+0000", 2024-09-14T05:31:05.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.88e-06 2024-09-14T05:31:05.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417511+0000", 2024-09-14T05:31:05.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5950000000000001e-06 2024-09-14T05:31:05.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478784+0000", 2024-09-14T05:31:05.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061272971000000002 2024-09-14T05:31:05.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.478841+0000", 2024-09-14T05:31:05.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-14T05:31:05.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479109+0000", 2024-09-14T05:31:05.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026794600000000003 2024-09-14T05:31:05.426 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.426 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.426 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.426 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4055 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14024704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.426 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:31:05.426 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.939488122, 2024-09-14T05:31:05.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061932299000000003, 2024-09-14T05:31:05.427 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.427 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.427 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.427 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4055 2024-09-14T05:31:05.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.427 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:31:05.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417514+0000", 2024-09-14T05:31:05.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417515+0000", 2024-09-14T05:31:05.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-14T05:31:05.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417516+0000", 2024-09-14T05:31:05.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7199999999999997e-07 2024-09-14T05:31:05.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417519+0000", 2024-09-14T05:31:05.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.937e-06 2024-09-14T05:31:05.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417522+0000", 2024-09-14T05:31:05.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1420000000000002e-06 2024-09-14T05:31:05.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479140+0000", 2024-09-14T05:31:05.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061617933999999999 2024-09-14T05:31:05.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479188+0000", 2024-09-14T05:31:05.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8717000000000001e-05 2024-09-14T05:31:05.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479447+0000", 2024-09-14T05:31:05.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025838800000000002 2024-09-14T05:31:05.430 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.431 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.431 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.431 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4056 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14516224~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.431 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:31:05.431 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.939477948999993, 2024-09-14T05:31:05.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062263185999999998, 2024-09-14T05:31:05.431 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.431 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.431 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.431 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.431 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.432 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4056 2024-09-14T05:31:05.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.432 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:31:05.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:31:05.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417525+0000", 2024-09-14T05:31:05.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8700000000000002e-07 2024-09-14T05:31:05.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417528+0000", 2024-09-14T05:31:05.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.579e-06 2024-09-14T05:31:05.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417533+0000", 2024-09-14T05:31:05.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2400000000000001e-06 2024-09-14T05:31:05.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417536+0000", 2024-09-14T05:31:05.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0309999999999999e-06 2024-09-14T05:31:05.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479477+0000", 2024-09-14T05:31:05.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061940967 2024-09-14T05:31:05.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479523+0000", 2024-09-14T05:31:05.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6335000000000003e-05 2024-09-14T05:31:05.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479788+0000", 2024-09-14T05:31:05.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026474700000000002 2024-09-14T05:31:05.435 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.435 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.435 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.435 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4057 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15007744~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.435 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:31:05.435 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.939463488000001, 2024-09-14T05:31:05.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062589928000000003, 2024-09-14T05:31:05.436 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.436 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.436 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.436 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4057 2024-09-14T05:31:05.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.436 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:31:05.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.417539+0000", 2024-09-14T05:31:05.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422838+0000", 2024-09-14T05:31:05.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052990349999999997 2024-09-14T05:31:05.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422854+0000", 2024-09-14T05:31:05.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5523999999999999e-05 2024-09-14T05:31:05.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422864+0000", 2024-09-14T05:31:05.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0084e-05 2024-09-14T05:31:05.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.422872+0000", 2024-09-14T05:31:05.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-14T05:31:05.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479813+0000", 2024-09-14T05:31:05.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056941389000000002 2024-09-14T05:31:05.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.479868+0000", 2024-09-14T05:31:05.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4973999999999999e-05 2024-09-14T05:31:05.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.480129+0000", 2024-09-14T05:31:05.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026084200000000001 2024-09-14T05:31:05.439 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.439 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.440 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.440 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4171 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14057472~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.440 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:31:05.440 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.927465928000004, 2024-09-14T05:31:05.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061184929999999998, 2024-09-14T05:31:05.440 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.440 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.440 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.440 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.440 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.440 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.440 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4171 2024-09-14T05:31:05.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.441 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:31:05.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:31:05.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429537+0000", 2024-09-14T05:31:05.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.39e-07 2024-09-14T05:31:05.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429541+0000", 2024-09-14T05:31:05.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6040000000000001e-06 2024-09-14T05:31:05.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429547+0000", 2024-09-14T05:31:05.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3280000000000002e-06 2024-09-14T05:31:05.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429552+0000", 2024-09-14T05:31:05.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4859999999999997e-06 2024-09-14T05:31:05.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490379+0000", 2024-09-14T05:31:05.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060826681 2024-09-14T05:31:05.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490434+0000", 2024-09-14T05:31:05.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4577999999999998e-05 2024-09-14T05:31:05.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490722+0000", 2024-09-14T05:31:05.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028781400000000001 2024-09-14T05:31:05.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.444 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.444 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4172 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 14548992~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.444 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:31:05.445 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.927443920000002, 2024-09-14T05:31:05.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061498256000000001, 2024-09-14T05:31:05.445 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.445 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.445 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.445 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4172 2024-09-14T05:31:05.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.445 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:31:05.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:31:05.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429559+0000", 2024-09-14T05:31:05.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.72e-07 2024-09-14T05:31:05.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429561+0000", 2024-09-14T05:31:05.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.717e-06 2024-09-14T05:31:05.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429566+0000", 2024-09-14T05:31:05.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6529999999999999e-06 2024-09-14T05:31:05.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429571+0000", 2024-09-14T05:31:05.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5979999999999999e-06 2024-09-14T05:31:05.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490751+0000", 2024-09-14T05:31:05.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061180336000000002 2024-09-14T05:31:05.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.490798+0000", 2024-09-14T05:31:05.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6875000000000001e-05 2024-09-14T05:31:05.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491057+0000", 2024-09-14T05:31:05.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025870500000000002 2024-09-14T05:31:05.449 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.449 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.449 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.449 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4173 3.1f 3:f8a8c641:::rbd_data.112d1f36d097.0000000000000000:6 [read 15040512~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.449 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:31:05.449 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.927426294, 2024-09-14T05:31:05.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061894820000000003, 2024-09-14T05:31:05.449 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.449 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.449 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.450 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.450 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.450 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.450 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4173 2024-09-14T05:31:05.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.450 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:31:05.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429576+0000", 2024-09-14T05:31:05.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429577+0000", 2024-09-14T05:31:05.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-14T05:31:05.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429578+0000", 2024-09-14T05:31:05.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6980000000000001e-06 2024-09-14T05:31:05.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429585+0000", 2024-09-14T05:31:05.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.725e-06 2024-09-14T05:31:05.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.429590+0000", 2024-09-14T05:31:05.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.938e-06 2024-09-14T05:31:05.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491091+0000", 2024-09-14T05:31:05.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061500773000000002 2024-09-14T05:31:05.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491136+0000", 2024-09-14T05:31:05.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5284000000000002e-05 2024-09-14T05:31:05.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.491471+0000", 2024-09-14T05:31:05.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033485699999999998 2024-09-14T05:31:05.453 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.453 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.453 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.453 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.453 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.453 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:31:05.455 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T05:31:05.559 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:31:05.559 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:31:05.559 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:31:05.559 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:31:05.568 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T05:31:05.672 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:31:05.672 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:31:05.672 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:31:05.672 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:31:05.672 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:31:05.680 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T05:31:05.785 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:31:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:31:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:31:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:31:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1205 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:31:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:31:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 95.586367155000005, 2024-09-14T05:31:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0086364279999999998, 2024-09-14T05:31:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1205 2024-09-14T05:31:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:31:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197890+0000", 2024-09-14T05:31:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197891+0000", 2024-09-14T05:31:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.364e-06 2024-09-14T05:31:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197896+0000", 2024-09-14T05:31:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3000000000000001e-06 2024-09-14T05:31:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197910+0000", 2024-09-14T05:31:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3339000000000001e-05 2024-09-14T05:31:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197919+0000", 2024-09-14T05:31:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8869999999999998e-06 2024-09-14T05:31:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.197974+0000", 2024-09-14T05:31:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5720000000000002e-05 2024-09-14T05:31:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.198094+0000", 2024-09-14T05:31:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119739 2024-09-14T05:31:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206526+0000", 2024-09-14T05:31:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084320790000000003 2024-09-14T05:31:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1220 3.1 3:840a4455:::rbd_data.112dd25fb4c.0000000000000088:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:31:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:31:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 95.577610790999998, 2024-09-14T05:31:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012822337, 2024-09-14T05:31:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1220 2024-09-14T05:31:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:31:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206646+0000", 2024-09-14T05:31:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206647+0000", 2024-09-14T05:31:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4600000000000003e-07 2024-09-14T05:31:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206652+0000", 2024-09-14T05:31:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6619999999999996e-06 2024-09-14T05:31:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206662+0000", 2024-09-14T05:31:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.995e-06 2024-09-14T05:31:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206667+0000", 2024-09-14T05:31:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-14T05:31:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206706+0000", 2024-09-14T05:31:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9523000000000001e-05 2024-09-14T05:31:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206755+0000", 2024-09-14T05:31:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000002e-05 2024-09-14T05:31:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:31:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.206877+0000", 2024-09-14T05:31:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121938 2024-09-14T05:31:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:31:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.207310+0000", 2024-09-14T05:31:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000432452 2024-09-14T05:31:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:31:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219364+0000", 2024-09-14T05:31:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012054481000000001 2024-09-14T05:31:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219425+0000", 2024-09-14T05:31:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0618000000000001e-05 2024-09-14T05:31:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219468+0000", 2024-09-14T05:31:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3739000000000003e-05 2024-09-14T05:31:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1243 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:31:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:31:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 95.564645564000003, 2024-09-14T05:31:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078660580000000004, 2024-09-14T05:31:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1243 2024-09-14T05:31:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:31:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219611+0000", 2024-09-14T05:31:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219612+0000", 2024-09-14T05:31:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.09e-07 2024-09-14T05:31:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219616+0000", 2024-09-14T05:31:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.216e-06 2024-09-14T05:31:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219630+0000", 2024-09-14T05:31:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3064999999999999e-05 2024-09-14T05:31:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219637+0000", 2024-09-14T05:31:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7409999999999992e-06 2024-09-14T05:31:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219697+0000", 2024-09-14T05:31:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.02e-05 2024-09-14T05:31:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.219833+0000", 2024-09-14T05:31:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013595600000000001 2024-09-14T05:31:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227477+0000", 2024-09-14T05:31:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076439710000000003 2024-09-14T05:31:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1264 3.12 3:4bc7da7f:::rbd_data.112dd25fb4c.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:31:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:31:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 95.556634771000006, 2024-09-14T05:31:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080867990000000004, 2024-09-14T05:31:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1264 2024-09-14T05:31:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:31:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227622+0000", 2024-09-14T05:31:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227623+0000", 2024-09-14T05:31:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7000000000000003e-07 2024-09-14T05:31:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227627+0000", 2024-09-14T05:31:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.194e-06 2024-09-14T05:31:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227639+0000", 2024-09-14T05:31:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1296e-05 2024-09-14T05:31:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227648+0000", 2024-09-14T05:31:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2280000000000004e-06 2024-09-14T05:31:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227689+0000", 2024-09-14T05:31:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1329999999999999e-05 2024-09-14T05:31:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227741+0000", 2024-09-14T05:31:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1641000000000002e-05 2024-09-14T05:31:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-14T05:31:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.227914+0000", 2024-09-14T05:31:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000173215 2024-09-14T05:31:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:31:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.228474+0000", 2024-09-14T05:31:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055980199999999998 2024-09-14T05:31:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:31:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235608+0000", 2024-09-14T05:31:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0071343830000000002 2024-09-14T05:31:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235669+0000", 2024-09-14T05:31:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1068999999999994e-05 2024-09-14T05:31:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.235709+0000", 2024-09-14T05:31:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9671e-05 2024-09-14T05:31:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1526 3.4 3:2695cbe4:::rbd_data.112dd25fb4c.0000000000000119:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:31:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:31:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 95.458094208000006, 2024-09-14T05:31:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0079537359999999994, 2024-09-14T05:31:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1526 2024-09-14T05:31:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:31:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:31:05.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326163+0000", 2024-09-14T05:31:05.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3600000000000003e-07 2024-09-14T05:31:05.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326168+0000", 2024-09-14T05:31:05.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3460000000000004e-06 2024-09-14T05:31:05.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326178+0000", 2024-09-14T05:31:05.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0291e-05 2024-09-14T05:31:05.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326183+0000", 2024-09-14T05:31:05.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-14T05:31:05.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326230+0000", 2024-09-14T05:31:05.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6650999999999998e-05 2024-09-14T05:31:05.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326295+0000", 2024-09-14T05:31:05.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5158000000000002e-05 2024-09-14T05:31:05.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:31:05.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.326408+0000", 2024-09-14T05:31:05.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113039 2024-09-14T05:31:05.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:31:05.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.331648+0000", 2024-09-14T05:31:05.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052404929999999997 2024-09-14T05:31:05.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:31:05.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334011+0000", 2024-09-14T05:31:05.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002362669 2024-09-14T05:31:05.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334064+0000", 2024-09-14T05:31:05.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3198999999999997e-05 2024-09-14T05:31:05.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334116+0000", 2024-09-14T05:31:05.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2133999999999999e-05 2024-09-14T05:31:05.811 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.812 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.812 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.812 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:1531 3.4 3:2472763d:::rbd_data.112dd25fb4c.000000000000011b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:31:05.812 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:31:05.812 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 95.455276041999994, 2024-09-14T05:31:05.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082114309999999999, 2024-09-14T05:31:05.812 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.812 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.812 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.812 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.812 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.812 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.813 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1531 2024-09-14T05:31:05.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.813 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:31:05.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328981+0000", 2024-09-14T05:31:05.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328982+0000", 2024-09-14T05:31:05.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7700000000000004e-07 2024-09-14T05:31:05.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328985+0000", 2024-09-14T05:31:05.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7900000000000001e-06 2024-09-14T05:31:05.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.328996+0000", 2024-09-14T05:31:05.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.028e-05 2024-09-14T05:31:05.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329001+0000", 2024-09-14T05:31:05.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0610000000000003e-06 2024-09-14T05:31:05.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329035+0000", 2024-09-14T05:31:05.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4366999999999998e-05 2024-09-14T05:31:05.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329082+0000", 2024-09-14T05:31:05.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6646e-05 2024-09-14T05:31:05.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:31:05.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.329195+0000", 2024-09-14T05:31:05.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113279 2024-09-14T05:31:05.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:31:05.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.334293+0000", 2024-09-14T05:31:05.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050980399999999999 2024-09-14T05:31:05.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:31:05.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337101+0000", 2024-09-14T05:31:05.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0028076170000000001 2024-09-14T05:31:05.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337154+0000", 2024-09-14T05:31:05.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3641999999999999e-05 2024-09-14T05:31:05.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.337192+0000", 2024-09-14T05:31:05.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7932000000000002e-05 2024-09-14T05:31:05.817 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.817 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.817 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.817 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:2131 3.b e30/24 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head v 30'36, mlcod=30'34)", 2024-09-14T05:31:05.818 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:31:05.818 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.991928686999998, 2024-09-14T05:31:05.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031871709999999998, 2024-09-14T05:31:05.818 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.818 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.818 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.818 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:31:05.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792328+0000", 2024-09-14T05:31:05.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.792331+0000", 2024-09-14T05:31:05.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6900000000000001e-06 2024-09-14T05:31:05.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797319+0000", 2024-09-14T05:31:05.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049876349999999998 2024-09-14T05:31:05.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797349+0000", 2024-09-14T05:31:05.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0046e-05 2024-09-14T05:31:05.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797359+0000", 2024-09-14T05:31:05.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0037999999999999e-05 2024-09-14T05:31:05.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797445+0000", 2024-09-14T05:31:05.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6132999999999997e-05 2024-09-14T05:31:05.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.797472+0000", 2024-09-14T05:31:05.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7478999999999999e-05 2024-09-14T05:31:05.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824161+0000", 2024-09-14T05:31:05.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026688714999999998 2024-09-14T05:31:05.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824200+0000", 2024-09-14T05:31:05.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8974000000000003e-05 2024-09-14T05:31:05.821 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.822 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.822 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.822 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4259 3.4 3:25f21dc9:::rbd_data.112d3d611e54.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 294912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.822 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:31:05.822 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.392756073000001, 2024-09-14T05:31:05.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010043553, 2024-09-14T05:31:05.822 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.822 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.822 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.823 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4259 2024-09-14T05:31:05.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.823 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:31:05.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391501+0000", 2024-09-14T05:31:05.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391503+0000", 2024-09-14T05:31:05.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9099999999999999e-06 2024-09-14T05:31:05.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391821+0000", 2024-09-14T05:31:05.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031871800000000001 2024-09-14T05:31:05.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391844+0000", 2024-09-14T05:31:05.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2045e-05 2024-09-14T05:31:05.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391853+0000", 2024-09-14T05:31:05.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7249999999999992e-06 2024-09-14T05:31:05.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.391917+0000", 2024-09-14T05:31:05.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3458000000000001e-05 2024-09-14T05:31:05.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392037+0000", 2024-09-14T05:31:05.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120449 2024-09-14T05:31:05.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:31:05.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.392305+0000", 2024-09-14T05:31:05.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026739299999999999 2024-09-14T05:31:05.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:31:05.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.393609+0000", 2024-09-14T05:31:05.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013042450000000001 2024-09-14T05:31:05.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:31:05.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401430+0000", 2024-09-14T05:31:05.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078211670000000004 2024-09-14T05:31:05.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401499+0000", 2024-09-14T05:31:05.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9374000000000004e-05 2024-09-14T05:31:05.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.401544+0000", 2024-09-14T05:31:05.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5068999999999998e-05 2024-09-14T05:31:05.827 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.827 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.827 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.827 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4554 3.1 3:8288b446:::rbd_object_map.112d3d611e54:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:05.827 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:31:05.827 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.799855261999994, 2024-09-14T05:31:05.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069998999999999999, 2024-09-14T05:31:05.828 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.828 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.828 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.828 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.828 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.828 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.828 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4554 2024-09-14T05:31:05.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.828 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:31:05.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:31:05.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984402+0000", 2024-09-14T05:31:05.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9299999999999999e-07 2024-09-14T05:31:05.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984404+0000", 2024-09-14T05:31:05.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.855e-06 2024-09-14T05:31:05.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984407+0000", 2024-09-14T05:31:05.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-14T05:31:05.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.984410+0000", 2024-09-14T05:31:05.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8159999999999998e-06 2024-09-14T05:31:05.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985511+0000", 2024-09-14T05:31:05.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00110053 2024-09-14T05:31:05.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985561+0000", 2024-09-14T05:31:05.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0389000000000002e-05 2024-09-14T05:31:05.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-14T05:31:05.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.985772+0000", 2024-09-14T05:31:05.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000210571 2024-09-14T05:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.987075+0000", 2024-09-14T05:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001303128 2024-09-14T05:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991303+0000", 2024-09-14T05:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042283310000000001 2024-09-14T05:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991359+0000", 2024-09-14T05:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6335999999999997e-05 2024-09-14T05:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.991402+0000", 2024-09-14T05:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2074999999999999e-05 2024-09-14T05:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4711 3.4 3:226df663:::rbd_data.112d3d611e54.000000000000002d:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T05:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.617184105999996, 2024-09-14T05:31:05.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011304193000000001, 2024-09-14T05:31:05.834 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.834 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.834 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:05.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:05.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:05.834 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4711 2024-09-14T05:31:05.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.834 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:31:05.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167073+0000", 2024-09-14T05:31:05.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167074+0000", 2024-09-14T05:31:05.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0999999999999997e-07 2024-09-14T05:31:05.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167078+0000", 2024-09-14T05:31:05.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.878e-06 2024-09-14T05:31:05.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167087+0000", 2024-09-14T05:31:05.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6940000000000007e-06 2024-09-14T05:31:05.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167093+0000", 2024-09-14T05:31:05.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7740000000000001e-06 2024-09-14T05:31:05.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167128+0000", 2024-09-14T05:31:05.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5231999999999997e-05 2024-09-14T05:31:05.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.167207+0000", 2024-09-14T05:31:05.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9036999999999996e-05 2024-09-14T05:31:05.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-14T05:31:05.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.169140+0000", 2024-09-14T05:31:05.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001932897 2024-09-14T05:31:05.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:31:05.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178258+0000", 2024-09-14T05:31:05.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091180819999999996 2024-09-14T05:31:05.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:31:05.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178314+0000", 2024-09-14T05:31:05.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5571e-05 2024-09-14T05:31:05.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178348+0000", 2024-09-14T05:31:05.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3747000000000001e-05 2024-09-14T05:31:05.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:32.178377+0000", 2024-09-14T05:31:05.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9471e-05 2024-09-14T05:31:05.839 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.839 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.839 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.839 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4397.0:72582 3.b e47/24 3:d5072567:::rbd_data.112d4a90444c.0000000000000119:head v 47'386, mlcod=46'383)", 2024-09-14T05:31:05.839 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:31:05.839 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.259230252999998, 2024-09-14T05:31:05.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0067720940000000002, 2024-09-14T05:31:05.839 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.839 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.839 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.839 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:31:05.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:31:05.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525027+0000", 2024-09-14T05:31:05.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1100000000000002e-07 2024-09-14T05:31:05.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525031+0000", 2024-09-14T05:31:05.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0740000000000003e-06 2024-09-14T05:31:05.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525040+0000", 2024-09-14T05:31:05.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5329999999999994e-06 2024-09-14T05:31:05.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525045+0000", 2024-09-14T05:31:05.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6879999999999996e-06 2024-09-14T05:31:05.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525090+0000", 2024-09-14T05:31:05.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4743000000000002e-05 2024-09-14T05:31:05.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.525102+0000", 2024-09-14T05:31:05.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2067e-05 2024-09-14T05:31:05.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531757+0000", 2024-09-14T05:31:05.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066550380000000003 2024-09-14T05:31:05.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:49.531799+0000", 2024-09-14T05:31:05.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1640000000000001e-05 2024-09-14T05:31:05.843 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.843 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.843 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.843 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.7.0:509 3.b e100/24 3:d1b3949e:::rbd_data.112d46b1ca94.000000000000010c:11 v 100'523, mlcod=99'518)", 2024-09-14T05:31:05.843 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.009409+0000", 2024-09-14T05:31:05.843 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.774848133999999, 2024-09-14T05:31:05.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075659999999999998, 2024-09-14T05:31:05.844 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.844 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.844 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.844 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009409+0000", 2024-09-14T05:31:05.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009409+0000", 2024-09-14T05:31:05.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009409+0000", 2024-09-14T05:31:05.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7599999999999997e-07 2024-09-14T05:31:05.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009421+0000", 2024-09-14T05:31:05.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1705999999999999e-05 2024-09-14T05:31:05.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009432+0000", 2024-09-14T05:31:05.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0769e-05 2024-09-14T05:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009439+0000", 2024-09-14T05:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5969999999999997e-06 2024-09-14T05:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009505+0000", 2024-09-14T05:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6310999999999996e-05 2024-09-14T05:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.009521+0000", 2024-09-14T05:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.571e-05 2024-09-14T05:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.016931+0000", 2024-09-14T05:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0074101619999999997 2024-09-14T05:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.016975+0000", 2024-09-14T05:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4069000000000001e-05 2024-09-14T05:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.2.0:1434 3.5 e100/24 3:a529dc6d:::rbd_data.112d46b1ca94.00000000000000d4:11 v 100'327, mlcod=100'321)", 2024-09-14T05:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.013998+0000", 2024-09-14T05:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.770258717000001, 2024-09-14T05:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0083544610000000005, 2024-09-14T05:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.013998+0000", 2024-09-14T05:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.013998+0000", 2024-09-14T05:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.013999+0000", 2024-09-14T05:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-14T05:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014003+0000", 2024-09-14T05:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.51e-06 2024-09-14T05:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014013+0000", 2024-09-14T05:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3379999999999994e-06 2024-09-14T05:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014018+0000", 2024-09-14T05:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5269999999999998e-06 2024-09-14T05:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014843+0000", 2024-09-14T05:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00082516000000000002 2024-09-14T05:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014871+0000", 2024-09-14T05:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7628e-05 2024-09-14T05:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.022313+0000", 2024-09-14T05:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.007441563 2024-09-14T05:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.022353+0000", 2024-09-14T05:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0055000000000002e-05 2024-09-14T05:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.0.0:1713 3.15 e100/24 3:a916a3d3:::rbd_data.112d46b1ca94.00000000000000d3:11 v 100'427, mlcod=99'422)", 2024-09-14T05:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.014451+0000", 2024-09-14T05:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.769805432999998, 2024-09-14T05:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078576710000000001, 2024-09-14T05:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014451+0000", 2024-09-14T05:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014451+0000", 2024-09-14T05:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014452+0000", 2024-09-14T05:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9599999999999998e-07 2024-09-14T05:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014458+0000", 2024-09-14T05:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8699999999999997e-06 2024-09-14T05:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014469+0000", 2024-09-14T05:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0627e-05 2024-09-14T05:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014476+0000", 2024-09-14T05:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.046e-06 2024-09-14T05:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.014995+0000", 2024-09-14T05:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051870599999999996 2024-09-14T05:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.015014+0000", 2024-09-14T05:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9768000000000001e-05 2024-09-14T05:31:05.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.022266+0000", 2024-09-14T05:31:05.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0072513430000000004 2024-09-14T05:31:05.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.022309+0000", 2024-09-14T05:31:05.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3514999999999999e-05 2024-09-14T05:31:05.856 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.856 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.856 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.857 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.7.0:510 3.b e100/24 3:d5c4519c:::rbd_data.112d46b1ca94.0000000000000024:11 v 100'525, mlcod=100'521)", 2024-09-14T05:31:05.857 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.017655+0000", 2024-09-14T05:31:05.857 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.766601973, 2024-09-14T05:31:05.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0073483380000000003, 2024-09-14T05:31:05.857 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.857 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.857 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.857 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017655+0000", 2024-09-14T05:31:05.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017655+0000", 2024-09-14T05:31:05.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017656+0000", 2024-09-14T05:31:05.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8800000000000002e-07 2024-09-14T05:31:05.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017660+0000", 2024-09-14T05:31:05.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2370000000000003e-06 2024-09-14T05:31:05.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017669+0000", 2024-09-14T05:31:05.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7749999999999998e-06 2024-09-14T05:31:05.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017675+0000", 2024-09-14T05:31:05.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1990000000000003e-06 2024-09-14T05:31:05.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017727+0000", 2024-09-14T05:31:05.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1835000000000001e-05 2024-09-14T05:31:05.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.017745+0000", 2024-09-14T05:31:05.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8233999999999999e-05 2024-09-14T05:31:05.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.024955+0000", 2024-09-14T05:31:05.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0072104279999999996 2024-09-14T05:31:05.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.025003+0000", 2024-09-14T05:31:05.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7942000000000003e-05 2024-09-14T05:31:05.861 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.861 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.861 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.861 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.5.0:2812 3.4 e100/39 3:2237b773:::rbd_data.112d46b1ca94.00000000000001f5:11 v 100'550, mlcod=100'546)", 2024-09-14T05:31:05.861 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.018029+0000", 2024-09-14T05:31:05.861 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.766227948000001, 2024-09-14T05:31:05.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0071886470000000003, 2024-09-14T05:31:05.861 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.861 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.861 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.861 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018029+0000", 2024-09-14T05:31:05.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018029+0000", 2024-09-14T05:31:05.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018030+0000", 2024-09-14T05:31:05.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3900000000000004e-07 2024-09-14T05:31:05.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018036+0000", 2024-09-14T05:31:05.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9669999999999998e-06 2024-09-14T05:31:05.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018049+0000", 2024-09-14T05:31:05.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2897e-05 2024-09-14T05:31:05.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018055+0000", 2024-09-14T05:31:05.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8109999999999997e-06 2024-09-14T05:31:05.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018104+0000", 2024-09-14T05:31:05.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8408999999999997e-05 2024-09-14T05:31:05.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018125+0000", 2024-09-14T05:31:05.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0992000000000002e-05 2024-09-14T05:31:05.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.025174+0000", 2024-09-14T05:31:05.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0070491850000000003 2024-09-14T05:31:05.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.025218+0000", 2024-09-14T05:31:05.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3646999999999997e-05 2024-09-14T05:31:05.865 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.865 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.865 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.865 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.7.0:511 3.b e100/24 3:d76f30d4:::rbd_data.112d46b1ca94.0000000000000048:11 v 100'527, mlcod=100'521)", 2024-09-14T05:31:05.865 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.018634+0000", 2024-09-14T05:31:05.865 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.765623172000002, 2024-09-14T05:31:05.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0090907680000000008, 2024-09-14T05:31:05.865 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.865 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.866 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.866 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018634+0000", 2024-09-14T05:31:05.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018634+0000", 2024-09-14T05:31:05.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018634+0000", 2024-09-14T05:31:05.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5899999999999996e-07 2024-09-14T05:31:05.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018639+0000", 2024-09-14T05:31:05.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5229999999999997e-06 2024-09-14T05:31:05.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018649+0000", 2024-09-14T05:31:05.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5999999999999996e-06 2024-09-14T05:31:05.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.018656+0000", 2024-09-14T05:31:05.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5789999999999998e-06 2024-09-14T05:31:05.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.020500+0000", 2024-09-14T05:31:05.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0018434840000000001 2024-09-14T05:31:05.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.020522+0000", 2024-09-14T05:31:05.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2892999999999999e-05 2024-09-14T05:31:05.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.027685+0000", 2024-09-14T05:31:05.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.007162758 2024-09-14T05:31:05.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.027725+0000", 2024-09-14T05:31:05.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9271999999999999e-05 2024-09-14T05:31:05.869 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.869 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.869 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.869 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.2.0:1440 3.5 e100/24 3:a70fccf8:::rbd_data.112d46b1ca94.00000000000000ec:11 v 100'329, mlcod=100'325)", 2024-09-14T05:31:05.869 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.023047+0000", 2024-09-14T05:31:05.870 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.761209635, 2024-09-14T05:31:05.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.007561229, 2024-09-14T05:31:05.870 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.870 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.870 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.870 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023047+0000", 2024-09-14T05:31:05.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023047+0000", 2024-09-14T05:31:05.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023048+0000", 2024-09-14T05:31:05.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0999999999999997e-07 2024-09-14T05:31:05.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023054+0000", 2024-09-14T05:31:05.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9510000000000003e-06 2024-09-14T05:31:05.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023067+0000", 2024-09-14T05:31:05.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2959000000000001e-05 2024-09-14T05:31:05.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023074+0000", 2024-09-14T05:31:05.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0450000000000003e-06 2024-09-14T05:31:05.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023155+0000", 2024-09-14T05:31:05.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1422999999999996e-05 2024-09-14T05:31:05.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023179+0000", 2024-09-14T05:31:05.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.376e-05 2024-09-14T05:31:05.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.030542+0000", 2024-09-14T05:31:05.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0073630859999999996 2024-09-14T05:31:05.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.030609+0000", 2024-09-14T05:31:05.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6194999999999999e-05 2024-09-14T05:31:05.873 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.874 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.874 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.874 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.0.0:1718 3.15 e100/24 3:add139e1:::rbd_data.112d46b1ca94.0000000000000007:11 v 100'429, mlcod=100'425)", 2024-09-14T05:31:05.874 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.023071+0000", 2024-09-14T05:31:05.874 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.761185823000002, 2024-09-14T05:31:05.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076044830000000004, 2024-09-14T05:31:05.874 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.874 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.874 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.874 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023071+0000", 2024-09-14T05:31:05.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023071+0000", 2024-09-14T05:31:05.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023072+0000", 2024-09-14T05:31:05.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2099999999999996e-07 2024-09-14T05:31:05.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023077+0000", 2024-09-14T05:31:05.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3759999999999999e-06 2024-09-14T05:31:05.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023085+0000", 2024-09-14T05:31:05.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2009999999999999e-06 2024-09-14T05:31:05.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023091+0000", 2024-09-14T05:31:05.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3720000000000004e-06 2024-09-14T05:31:05.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023241+0000", 2024-09-14T05:31:05.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015081799999999999 2024-09-14T05:31:05.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023261+0000", 2024-09-14T05:31:05.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9076000000000002e-05 2024-09-14T05:31:05.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.030614+0000", 2024-09-14T05:31:05.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00735347 2024-09-14T05:31:05.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.030676+0000", 2024-09-14T05:31:05.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1549000000000003e-05 2024-09-14T05:31:05.878 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.878 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.878 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.878 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(osd.0.0:1719 3.15 e100/24 3:adb89761:::rbd_data.112d46b1ca94.00000000000003a3:11 v 100'431, mlcod=100'425)", 2024-09-14T05:31:05.878 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:30:49.023954+0000", 2024-09-14T05:31:05.878 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.760302886000002, 2024-09-14T05:31:05.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094961170000000001, 2024-09-14T05:31:05.878 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:05.878 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:05.878 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:05.879 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:05.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:05.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023954+0000", 2024-09-14T05:31:05.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:05.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023954+0000", 2024-09-14T05:31:05.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:05.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:05.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023955+0000", 2024-09-14T05:31:05.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7400000000000003e-07 2024-09-14T05:31:05.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:05.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023960+0000", 2024-09-14T05:31:05.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1460000000000004e-06 2024-09-14T05:31:05.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:05.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023970+0000", 2024-09-14T05:31:05.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0455000000000001e-05 2024-09-14T05:31:05.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:05.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.023976+0000", 2024-09-14T05:31:05.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1659999999999999e-06 2024-09-14T05:31:05.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:05.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.025476+0000", 2024-09-14T05:31:05.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001499088 2024-09-14T05:31:05.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:05.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.025498+0000", 2024-09-14T05:31:05.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2041000000000001e-05 2024-09-14T05:31:05.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:05.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.033380+0000", 2024-09-14T05:31:05.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078826269999999997 2024-09-14T05:31:05.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:05.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:05.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:05.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:30:49.033450+0000", 2024-09-14T05:31:05.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9919999999999995e-05 2024-09-14T05:31:05.882 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.882 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.882 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.882 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:05.882 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:05.882 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:31:05.883 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T05:31:05.986 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:31:05.986 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:31:05.986 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-14T05:31:05.987 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:31:05.995 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T05:31:06.032 INFO:teuthology.orchestra.run.smithi071.stderr:nodeep-scrub is unset 2024-09-14T05:31:06.033 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T05:31:06.049 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-14T05:31:06.099 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:31:06.100 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-14T05:31:06.100 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-14T05:31:06.100 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-14T05:31:06.100 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:31:06.109 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T05:31:06.214 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-14T05:31:06.214 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-14T05:31:06.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-14T05:31:06.214 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-14T05:31:06.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.214 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:2131 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T05:31:06.215 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:31:06.215 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 95.437697670000006, 2024-09-14T05:31:06.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049526593000000001, 2024-09-14T05:31:06.215 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.215 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.215 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T05:31:06.215 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.215 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.215 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.215 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 2131 2024-09-14T05:31:06.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.215 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:31:06.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774865+0000", 2024-09-14T05:31:06.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.774867+0000", 2024-09-14T05:31:06.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4430000000000002e-06 2024-09-14T05:31:06.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790357+0000", 2024-09-14T05:31:06.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015489466 2024-09-14T05:31:06.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790383+0000", 2024-09-14T05:31:06.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6247000000000001e-05 2024-09-14T05:31:06.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790394+0000", 2024-09-14T05:31:06.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0728e-05 2024-09-14T05:31:06.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790471+0000", 2024-09-14T05:31:06.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7539000000000004e-05 2024-09-14T05:31:06.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790639+0000", 2024-09-14T05:31:06.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016787700000000001 2024-09-14T05:31:06.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-14T05:31:06.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.790998+0000", 2024-09-14T05:31:06.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035931299999999999 2024-09-14T05:31:06.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-14T05:31:06.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.823529+0000", 2024-09-14T05:31:06.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032530798 2024-09-14T05:31:06.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-14T05:31:06.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824324+0000", 2024-09-14T05:31:06.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00079442399999999995 2024-09-14T05:31:06.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-14T05:31:06.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824362+0000", 2024-09-14T05:31:06.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8398000000000001e-05 2024-09-14T05:31:06.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:30.824391+0000", 2024-09-14T05:31:06.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.936e-05 2024-09-14T05:31:06.220 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.220 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.220 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.221 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3892 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7602176~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.221 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:31:06.221 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.856119672000005, 2024-09-14T05:31:06.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047082951999999997, 2024-09-14T05:31:06.221 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.221 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.221 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.221 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.221 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.221 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.221 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3892 2024-09-14T05:31:06.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.222 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:31:06.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:31:06.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356443+0000", 2024-09-14T05:31:06.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0199999999999998e-07 2024-09-14T05:31:06.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356454+0000", 2024-09-14T05:31:06.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0414999999999999e-05 2024-09-14T05:31:06.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356463+0000", 2024-09-14T05:31:06.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6739999999999992e-06 2024-09-14T05:31:06.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356468+0000", 2024-09-14T05:31:06.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5680000000000001e-06 2024-09-14T05:31:06.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403143+0000", 2024-09-14T05:31:06.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.046675051000000002 2024-09-14T05:31:06.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403206+0000", 2024-09-14T05:31:06.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3127999999999996e-05 2024-09-14T05:31:06.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403526+0000", 2024-09-14T05:31:06.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031981400000000002 2024-09-14T05:31:06.225 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.225 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.225 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.225 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3893 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8093696~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.225 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:31:06.225 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.856074835000001, 2024-09-14T05:31:06.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047516305000000002, 2024-09-14T05:31:06.225 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.225 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.225 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.225 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.226 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.226 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.226 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3893 2024-09-14T05:31:06.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.226 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:31:06.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:31:06.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356488+0000", 2024-09-14T05:31:06.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6899999999999997e-07 2024-09-14T05:31:06.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356491+0000", 2024-09-14T05:31:06.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8109999999999999e-06 2024-09-14T05:31:06.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356498+0000", 2024-09-14T05:31:06.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5320000000000003e-06 2024-09-14T05:31:06.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356502+0000", 2024-09-14T05:31:06.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000002e-06 2024-09-14T05:31:06.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403579+0000", 2024-09-14T05:31:06.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047076706000000003 2024-09-14T05:31:06.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.403664+0000", 2024-09-14T05:31:06.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5240000000000004e-05 2024-09-14T05:31:06.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404004+0000", 2024-09-14T05:31:06.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033974500000000002 2024-09-14T05:31:06.229 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.229 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.229 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.230 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3993 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 262144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.230 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:31:06.230 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.856052964, 2024-09-14T05:31:06.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048088476999999998, 2024-09-14T05:31:06.230 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.230 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.230 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.230 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.230 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.230 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.230 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3993 2024-09-14T05:31:06.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.230 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:31:06.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:31:06.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356510+0000", 2024-09-14T05:31:06.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.46e-07 2024-09-14T05:31:06.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356512+0000", 2024-09-14T05:31:06.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.827e-06 2024-09-14T05:31:06.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356518+0000", 2024-09-14T05:31:06.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7780000000000004e-06 2024-09-14T05:31:06.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356522+0000", 2024-09-14T05:31:06.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.865e-06 2024-09-14T05:31:06.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404036+0000", 2024-09-14T05:31:06.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.047513725999999999 2024-09-14T05:31:06.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404135+0000", 2024-09-14T05:31:06.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9352999999999996e-05 2024-09-14T05:31:06.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404598+0000", 2024-09-14T05:31:06.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046258200000000001 2024-09-14T05:31:06.234 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.234 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.234 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.234 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3994 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.234 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:31:06.234 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.856010952000005, 2024-09-14T05:31:06.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048274625000000002, 2024-09-14T05:31:06.234 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.234 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.234 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.235 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.235 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.235 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.235 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3994 2024-09-14T05:31:06.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.235 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:31:06.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:31:06.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356552+0000", 2024-09-14T05:31:06.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3799999999999997e-07 2024-09-14T05:31:06.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356554+0000", 2024-09-14T05:31:06.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2259999999999999e-06 2024-09-14T05:31:06.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356560+0000", 2024-09-14T05:31:06.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-14T05:31:06.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356566+0000", 2024-09-14T05:31:06.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3909999999999997e-06 2024-09-14T05:31:06.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404632+0000", 2024-09-14T05:31:06.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048066154 2024-09-14T05:31:06.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404719+0000", 2024-09-14T05:31:06.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7440000000000003e-05 2024-09-14T05:31:06.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404826+0000", 2024-09-14T05:31:06.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106813 2024-09-14T05:31:06.238 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.238 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.238 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.238 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3995 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.238 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:31:06.238 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.855985813000004, 2024-09-14T05:31:06.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048828209999999997, 2024-09-14T05:31:06.239 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.239 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.239 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.239 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.239 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.239 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.239 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3995 2024-09-14T05:31:06.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.239 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:31:06.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:31:06.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356577+0000", 2024-09-14T05:31:06.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6599999999999996e-07 2024-09-14T05:31:06.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356579+0000", 2024-09-14T05:31:06.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0719999999999998e-06 2024-09-14T05:31:06.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356585+0000", 2024-09-14T05:31:06.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.959e-06 2024-09-14T05:31:06.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356590+0000", 2024-09-14T05:31:06.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.464e-06 2024-09-14T05:31:06.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404875+0000", 2024-09-14T05:31:06.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048285649999999999 2024-09-14T05:31:06.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.404985+0000", 2024-09-14T05:31:06.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109417 2024-09-14T05:31:06.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405405+0000", 2024-09-14T05:31:06.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042008200000000001 2024-09-14T05:31:06.242 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.243 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.243 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.243 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3996 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.243 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:31:06.243 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.855676930000001, 2024-09-14T05:31:06.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049061595, 2024-09-14T05:31:06.243 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.243 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.243 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.243 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.243 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.243 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.244 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3996 2024-09-14T05:31:06.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.244 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:31:06.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:31:06.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356886+0000", 2024-09-14T05:31:06.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.61e-07 2024-09-14T05:31:06.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356892+0000", 2024-09-14T05:31:06.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9050000000000002e-06 2024-09-14T05:31:06.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356901+0000", 2024-09-14T05:31:06.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4149999999999999e-06 2024-09-14T05:31:06.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.356907+0000", 2024-09-14T05:31:06.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0140000000000004e-06 2024-09-14T05:31:06.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405448+0000", 2024-09-14T05:31:06.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048541647 2024-09-14T05:31:06.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405542+0000", 2024-09-14T05:31:06.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3639000000000003e-05 2024-09-14T05:31:06.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405947+0000", 2024-09-14T05:31:06.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040521399999999999 2024-09-14T05:31:06.247 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.247 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.247 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.247 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3997 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2228224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.247 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:31:06.247 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.855462947999996, 2024-09-14T05:31:06.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049308774999999999, 2024-09-14T05:31:06.248 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.248 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.248 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.248 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.248 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.248 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.248 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3997 2024-09-14T05:31:06.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.248 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:31:06.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:31:06.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357100+0000", 2024-09-14T05:31:06.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6499999999999999e-07 2024-09-14T05:31:06.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357104+0000", 2024-09-14T05:31:06.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4989999999999999e-06 2024-09-14T05:31:06.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357112+0000", 2024-09-14T05:31:06.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4279999999999996e-06 2024-09-14T05:31:06.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357117+0000", 2024-09-14T05:31:06.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6190000000000002e-06 2024-09-14T05:31:06.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.405989+0000", 2024-09-14T05:31:06.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.048872080999999998 2024-09-14T05:31:06.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406074+0000", 2024-09-14T05:31:06.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4676e-05 2024-09-14T05:31:06.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406408+0000", 2024-09-14T05:31:06.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033480700000000001 2024-09-14T05:31:06.252 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.252 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.252 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.253 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3998 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.253 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:31:06.253 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.855141519, 2024-09-14T05:31:06.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049462314, 2024-09-14T05:31:06.253 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.253 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.253 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.253 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.253 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.253 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.253 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3998 2024-09-14T05:31:06.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.253 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:31:06.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357421+0000", 2024-09-14T05:31:06.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357422+0000", 2024-09-14T05:31:06.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3099999999999996e-07 2024-09-14T05:31:06.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357429+0000", 2024-09-14T05:31:06.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2880000000000004e-06 2024-09-14T05:31:06.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357439+0000", 2024-09-14T05:31:06.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.302e-06 2024-09-14T05:31:06.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357445+0000", 2024-09-14T05:31:06.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4269999999999996e-06 2024-09-14T05:31:06.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406448+0000", 2024-09-14T05:31:06.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049003096000000003 2024-09-14T05:31:06.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406537+0000", 2024-09-14T05:31:06.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9135999999999995e-05 2024-09-14T05:31:06.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406883+0000", 2024-09-14T05:31:06.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000346134 2024-09-14T05:31:06.257 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.257 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.257 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.257 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:3999 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.257 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:31:06.257 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.855071162000002, 2024-09-14T05:31:06.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049797556, 2024-09-14T05:31:06.257 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.257 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.258 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.258 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.258 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.258 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.258 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3999 2024-09-14T05:31:06.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.258 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.258 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.258 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:31:06.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357491+0000", 2024-09-14T05:31:06.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357492+0000", 2024-09-14T05:31:06.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8299999999999996e-07 2024-09-14T05:31:06.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357495+0000", 2024-09-14T05:31:06.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0450000000000001e-06 2024-09-14T05:31:06.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357503+0000", 2024-09-14T05:31:06.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9689999999999994e-06 2024-09-14T05:31:06.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357510+0000", 2024-09-14T05:31:06.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-14T05:31:06.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406914+0000", 2024-09-14T05:31:06.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049404293000000002 2024-09-14T05:31:06.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.406965+0000", 2024-09-14T05:31:06.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0470999999999998e-05 2024-09-14T05:31:06.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407289+0000", 2024-09-14T05:31:06.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032434899999999998 2024-09-14T05:31:06.261 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.261 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.261 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.262 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4000 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 3702784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.262 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:31:06.262 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.854724817000005, 2024-09-14T05:31:06.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049936728999999999, 2024-09-14T05:31:06.262 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.262 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.262 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.262 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.262 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.262 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.262 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4000 2024-09-14T05:31:06.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.262 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:31:06.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357838+0000", 2024-09-14T05:31:06.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357839+0000", 2024-09-14T05:31:06.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-14T05:31:06.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357842+0000", 2024-09-14T05:31:06.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4850000000000001e-06 2024-09-14T05:31:06.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357851+0000", 2024-09-14T05:31:06.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1819999999999995e-06 2024-09-14T05:31:06.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357859+0000", 2024-09-14T05:31:06.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8129999999999996e-06 2024-09-14T05:31:06.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407337+0000", 2024-09-14T05:31:06.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049478084999999998 2024-09-14T05:31:06.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407442+0000", 2024-09-14T05:31:06.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000105302 2024-09-14T05:31:06.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407774+0000", 2024-09-14T05:31:06.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033200100000000001 2024-09-14T05:31:06.265 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.265 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.266 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.266 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4001 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4194304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.266 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:31:06.266 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.854629462000005, 2024-09-14T05:31:06.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050304205999999997, 2024-09-14T05:31:06.266 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.266 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.266 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.266 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.266 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.266 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.266 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4001 2024-09-14T05:31:06.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.267 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:31:06.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357933+0000", 2024-09-14T05:31:06.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357934+0000", 2024-09-14T05:31:06.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000004e-07 2024-09-14T05:31:06.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357937+0000", 2024-09-14T05:31:06.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0869999999999998e-06 2024-09-14T05:31:06.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357946+0000", 2024-09-14T05:31:06.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1439999999999992e-06 2024-09-14T05:31:06.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.357952+0000", 2024-09-14T05:31:06.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.609e-06 2024-09-14T05:31:06.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407820+0000", 2024-09-14T05:31:06.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049868580000000003 2024-09-14T05:31:06.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.407909+0000", 2024-09-14T05:31:06.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8887000000000005e-05 2024-09-14T05:31:06.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408237+0000", 2024-09-14T05:31:06.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032824599999999997 2024-09-14T05:31:06.270 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.270 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.270 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.270 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4002 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.270 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:31:06.270 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.854412866999994, 2024-09-14T05:31:06.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050539529999999999, 2024-09-14T05:31:06.270 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.270 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.270 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.271 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.271 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.271 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.271 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4002 2024-09-14T05:31:06.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.271 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:31:06.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:31:06.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358150+0000", 2024-09-14T05:31:06.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5099999999999999e-07 2024-09-14T05:31:06.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358154+0000", 2024-09-14T05:31:06.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6799999999999999e-06 2024-09-14T05:31:06.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358165+0000", 2024-09-14T05:31:06.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0899e-05 2024-09-14T05:31:06.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358172+0000", 2024-09-14T05:31:06.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8689999999999998e-06 2024-09-14T05:31:06.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408281+0000", 2024-09-14T05:31:06.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050108699 2024-09-14T05:31:06.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408369+0000", 2024-09-14T05:31:06.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8542999999999996e-05 2024-09-14T05:31:06.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408689+0000", 2024-09-14T05:31:06.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032008900000000002 2024-09-14T05:31:06.274 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.274 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.274 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.274 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4003 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5177344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.274 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:31:06.274 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.854373031999998, 2024-09-14T05:31:06.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050868759, 2024-09-14T05:31:06.275 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.275 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.275 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.275 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.275 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.275 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.275 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4003 2024-09-14T05:31:06.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.275 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:31:06.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:31:06.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358190+0000", 2024-09-14T05:31:06.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-07 2024-09-14T05:31:06.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358193+0000", 2024-09-14T05:31:06.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.588e-06 2024-09-14T05:31:06.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358204+0000", 2024-09-14T05:31:06.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1525999999999999e-05 2024-09-14T05:31:06.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358213+0000", 2024-09-14T05:31:06.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5010000000000004e-06 2024-09-14T05:31:06.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408719+0000", 2024-09-14T05:31:06.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050506708999999997 2024-09-14T05:31:06.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.408766+0000", 2024-09-14T05:31:06.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6579999999999998e-05 2024-09-14T05:31:06.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409058+0000", 2024-09-14T05:31:06.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000292365 2024-09-14T05:31:06.278 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.278 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.278 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.279 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4004 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 5668864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.279 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:31:06.279 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.853950189000003, 2024-09-14T05:31:06.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050839848, 2024-09-14T05:31:06.279 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.279 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.279 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.279 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.279 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.279 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.279 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4004 2024-09-14T05:31:06.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.279 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:31:06.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358612+0000", 2024-09-14T05:31:06.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358613+0000", 2024-09-14T05:31:06.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4499999999999996e-07 2024-09-14T05:31:06.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358618+0000", 2024-09-14T05:31:06.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6009999999999997e-06 2024-09-14T05:31:06.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358630+0000", 2024-09-14T05:31:06.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2122e-05 2024-09-14T05:31:06.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358638+0000", 2024-09-14T05:31:06.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3969999999999994e-06 2024-09-14T05:31:06.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409088+0000", 2024-09-14T05:31:06.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050449565000000002 2024-09-14T05:31:06.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409153+0000", 2024-09-14T05:31:06.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4728999999999996e-05 2024-09-14T05:31:06.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409452+0000", 2024-09-14T05:31:06.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029958900000000001 2024-09-14T05:31:06.283 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.283 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.283 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.283 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4005 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6160384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.283 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:31:06.283 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.853733254999995, 2024-09-14T05:31:06.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051054752000000002, 2024-09-14T05:31:06.283 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.283 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.283 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.283 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.283 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.284 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.284 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4005 2024-09-14T05:31:06.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.284 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:31:06.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358829+0000", 2024-09-14T05:31:06.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358830+0000", 2024-09-14T05:31:06.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-14T05:31:06.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358833+0000", 2024-09-14T05:31:06.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9529999999999999e-06 2024-09-14T05:31:06.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358843+0000", 2024-09-14T05:31:06.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8069999999999994e-06 2024-09-14T05:31:06.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358849+0000", 2024-09-14T05:31:06.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1040000000000003e-06 2024-09-14T05:31:06.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409487+0000", 2024-09-14T05:31:06.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050638626999999999 2024-09-14T05:31:06.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409535+0000", 2024-09-14T05:31:06.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7525000000000003e-05 2024-09-14T05:31:06.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409884+0000", 2024-09-14T05:31:06.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034914800000000003 2024-09-14T05:31:06.287 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.287 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.287 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.287 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4006 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 6651904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.287 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:31:06.287 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.853696142999993, 2024-09-14T05:31:06.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051425323000000002, 2024-09-14T05:31:06.288 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.288 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.288 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.288 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.288 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.288 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.288 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4006 2024-09-14T05:31:06.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.288 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:31:06.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358866+0000", 2024-09-14T05:31:06.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358867+0000", 2024-09-14T05:31:06.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2900000000000004e-07 2024-09-14T05:31:06.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358869+0000", 2024-09-14T05:31:06.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4210000000000002e-06 2024-09-14T05:31:06.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358876+0000", 2024-09-14T05:31:06.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4980000000000003e-06 2024-09-14T05:31:06.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358881+0000", 2024-09-14T05:31:06.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3090000000000003e-06 2024-09-14T05:31:06.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409915+0000", 2024-09-14T05:31:06.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051033348999999999 2024-09-14T05:31:06.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.409964+0000", 2024-09-14T05:31:06.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9858000000000003e-05 2024-09-14T05:31:06.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410292+0000", 2024-09-14T05:31:06.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032735899999999999 2024-09-14T05:31:06.291 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.291 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.291 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.292 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4007 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7143424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.292 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:31:06.292 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.853673549999996, 2024-09-14T05:31:06.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05184076, 2024-09-14T05:31:06.292 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.292 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.292 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.292 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4007 2024-09-14T05:31:06.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.293 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:31:06.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:31:06.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358889+0000", 2024-09-14T05:31:06.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.34e-07 2024-09-14T05:31:06.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358891+0000", 2024-09-14T05:31:06.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9819999999999998e-06 2024-09-14T05:31:06.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358897+0000", 2024-09-14T05:31:06.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9390000000000002e-06 2024-09-14T05:31:06.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358902+0000", 2024-09-14T05:31:06.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6770000000000003e-06 2024-09-14T05:31:06.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410335+0000", 2024-09-14T05:31:06.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051432991999999997 2024-09-14T05:31:06.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410416+0000", 2024-09-14T05:31:06.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1316999999999994e-05 2024-09-14T05:31:06.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410730+0000", 2024-09-14T05:31:06.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000313419 2024-09-14T05:31:06.296 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.296 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.296 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.296 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4008 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 7634944~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.296 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:31:06.296 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.853649188999995, 2024-09-14T05:31:06.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052208082000000003, 2024-09-14T05:31:06.296 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.297 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.297 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.297 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.297 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.297 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.297 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4008 2024-09-14T05:31:06.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.297 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:31:06.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358913+0000", 2024-09-14T05:31:06.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358914+0000", 2024-09-14T05:31:06.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6100000000000001e-07 2024-09-14T05:31:06.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358916+0000", 2024-09-14T05:31:06.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.902e-06 2024-09-14T05:31:06.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358923+0000", 2024-09-14T05:31:06.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4780000000000002e-06 2024-09-14T05:31:06.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.358929+0000", 2024-09-14T05:31:06.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0970000000000002e-06 2024-09-14T05:31:06.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410761+0000", 2024-09-14T05:31:06.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051831436000000002 2024-09-14T05:31:06.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.410829+0000", 2024-09-14T05:31:06.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7874999999999996e-05 2024-09-14T05:31:06.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411121+0000", 2024-09-14T05:31:06.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029283299999999999 2024-09-14T05:31:06.300 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.300 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.300 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.301 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4397.0:4009 3.b 3:d7dbd0f4:::rbd_data.112d1f36d097.0000000000000004:6 [read 8126464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-14T05:31:06.301 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:31:06.301 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.853501964000003, 2024-09-14T05:31:06.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052518169000000003, 2024-09-14T05:31:06.301 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-14T05:31:06.301 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-14T05:31:06.301 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-14T05:31:06.301 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-14T05:31:06.301 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4397", 2024-09-14T05:31:06.301 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.71:0/1357400920", 2024-09-14T05:31:06.301 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4009 2024-09-14T05:31:06.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.302 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-14T05:31:06.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-14T05:31:06.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:31:06.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-14T05:31:06.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:31:06.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-14T05:31:06.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-14T05:31:06.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359061+0000", 2024-09-14T05:31:06.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0200000000000002e-07 2024-09-14T05:31:06.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-14T05:31:06.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359066+0000", 2024-09-14T05:31:06.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7899999999999999e-06 2024-09-14T05:31:06.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-14T05:31:06.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359074+0000", 2024-09-14T05:31:06.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2579999999999995e-06 2024-09-14T05:31:06.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-14T05:31:06.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.359082+0000", 2024-09-14T05:31:06.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.464e-06 2024-09-14T05:31:06.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-14T05:31:06.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411160+0000", 2024-09-14T05:31:06.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052078078 2024-09-14T05:31:06.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-14T05:31:06.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411244+0000", 2024-09-14T05:31:06.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4041000000000004e-05 2024-09-14T05:31:06.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-14T05:31:06.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-14T05:31:06.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-14T05:31:06.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-14T05:29:31.411579+0000", 2024-09-14T05:31:06.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033493599999999998 2024-09-14T05:31:06.305 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.305 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.305 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.305 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-14T05:31:06.305 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-14T05:31:06.305 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-14T05:31:07.034 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-14T05:31:07.051 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T05:31:08.045 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T05:31:08.061 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-14T05:31:09.005 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-14T05:31:09.020 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T05:31:10.055 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T05:31:10.071 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-14T05:31:11.061 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-14T05:31:11.076 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T05:31:12.067 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T05:31:12.082 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-14T05:31:13.070 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-14T05:31:13.086 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T05:31:14.010 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T05:31:14.026 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-14T05:31:15.079 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-14T05:31:15.095 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T05:31:16.081 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T05:31:16.098 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-14T05:31:17.097 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-14T05:31:17.114 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T05:31:18.104 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T05:31:18.120 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-14T05:31:19.119 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-14T05:31:19.134 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T05:31:20.120 INFO:teuthology.orchestra.run.smithi071.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T05:31:20.137 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-14T05:31:21.126 INFO:teuthology.orchestra.run.smithi071.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-14T05:31:21.143 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-14T05:31:21.144 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-14T05:31:21.144 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-14T05:31:21.144 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-14T05:31:21.144 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-14T05:31:21.144 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T05:31:21.433 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:31:21.434 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":132,"fsid":"1127be44-5f14-4e7d-ad23-f608afe154f5","created":"2024-09-14T05:28:54.203072+0000","modified":"2024-09-14T05:31:21.124952+0000","last_up_change":"2024-09-14T05:29:06.755531+0000","last_in_change":"2024-09-14T05:28:56.535048+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T05:29:06.452744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T05:29:12.380843+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T05:29:22.335419+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":40,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7c7817f-4a65-4f66-9af5-d31d0a92cf30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6819","nonce":2926109885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6821","nonce":2926109885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6825","nonce":2926109885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6823","nonce":2926109885}]},"public_addr":"172.21.15.71:6819/2926109885","cluster_addr":"172.21.15.71:6821/2926109885","heartbeat_back_addr":"172.21.15.71:6825/2926109885","heartbeat_front_addr":"172.21.15.71:6823/2926109885","state":["exists","up"]},{"osd":1,"uuid":"e184604e-b973-4b7a-bba2-6f471753a0fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6827","nonce":345649648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6829","nonce":345649648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6833","nonce":345649648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6831","nonce":345649648}]},"public_addr":"172.21.15.71:6827/345649648","cluster_addr":"172.21.15.71:6829/345649648","heartbeat_back_addr":"172.21.15.71:6833/345649648","heartbeat_front_addr":"172.21.15.71:6831/345649648","state":["exists","up"]},{"osd":2,"uuid":"d8b70f09-5afb-4296-b8c9-719c172e0e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6801","nonce":2721860691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6803","nonce":2721860691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6807","nonce":2721860691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6805","nonce":2721860691}]},"public_addr":"172.21.15.71:6801/2721860691","cluster_addr":"172.21.15.71:6803/2721860691","heartbeat_back_addr":"172.21.15.71:6807/2721860691","heartbeat_front_addr":"172.21.15.71:6805/2721860691","state":["exists","up"]},{"osd":3,"uuid":"26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6809","nonce":494607463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6811","nonce":494607463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6815","nonce":494607463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6813","nonce":494607463}]},"public_addr":"172.21.15.71:6809/494607463","cluster_addr":"172.21.15.71:6811/494607463","heartbeat_back_addr":"172.21.15.71:6815/494607463","heartbeat_front_addr":"172.21.15.71:6813/494607463","state":["exists","up"]},{"osd":4,"uuid":"515cb7d6-16eb-4aff-973a-7f7ed49325ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6817","nonce":2240133417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6819","nonce":2240133417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6823","nonce":2240133417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6821","nonce":2240133417}]},"public_addr":"172.21.15.150:6817/2240133417","cluster_addr":"172.21.15.150:6819/2240133417","heartbeat_back_addr":"172.21.15.150:6823/2240133417","heartbeat_front_addr":"172.21.15.150:6821/2240133417","state":["exists","up"]},{"osd":5,"uuid":"c434b53a-04be-45b9-b89a-193654133950","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6809","nonce":2393825223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6811","nonce":2393825223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6815","nonce":2393825223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6813","nonce":2393825223}]},"public_addr":"172.21.15.150:6809/2393825223","cluster_addr":"172.21.15.150:6811/2393825223","heartbeat_back_addr":"172.21.15.150:6815/2393825223","heartbeat_front_addr":"172.21.15.150:6813/2393825223","state":["exists","up"]},{"osd":6,"uuid":"3d2f8405-9286-4b40-8acd-036f67373ac6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6801","nonce":3235729544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6803","nonce":3235729544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6807","nonce":3235729544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6805","nonce":3235729544}]},"public_addr":"172.21.15.150:6801/3235729544","cluster_addr":"172.21.15.150:6803/3235729544","heartbeat_back_addr":"172.21.15.150:6807/3235729544","heartbeat_front_addr":"172.21.15.150:6805/3235729544","state":["exists","up"]},{"osd":7,"uuid":"6724f080-4dc3-465a-939f-b023985bf998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6825","nonce":2381001355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6827","nonce":2381001355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6831","nonce":2381001355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6829","nonce":2381001355}]},"public_addr":"172.21.15.150:6825/2381001355","cluster_addr":"172.21.15.150:6827/2381001355","heartbeat_back_addr":"172.21.15.150:6831/2381001355","heartbeat_front_addr":"172.21.15.150:6829/2381001355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.548803+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.075023+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.117036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.332113+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.330371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:02.753946+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:02.071186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.378781+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.7","osds":[2,4]},{"pgid":"3.12","osds":[7,0]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T05:31:21.447 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-14T05:31:21.447 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T05:31:21.732 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:31:21.733 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":132,"fsid":"1127be44-5f14-4e7d-ad23-f608afe154f5","created":"2024-09-14T05:28:54.203072+0000","modified":"2024-09-14T05:31:21.124952+0000","last_up_change":"2024-09-14T05:29:06.755531+0000","last_in_change":"2024-09-14T05:28:56.535048+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T05:29:06.452744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T05:29:12.380843+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T05:29:22.335419+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":40,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7c7817f-4a65-4f66-9af5-d31d0a92cf30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6819","nonce":2926109885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6821","nonce":2926109885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6825","nonce":2926109885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6823","nonce":2926109885}]},"public_addr":"172.21.15.71:6819/2926109885","cluster_addr":"172.21.15.71:6821/2926109885","heartbeat_back_addr":"172.21.15.71:6825/2926109885","heartbeat_front_addr":"172.21.15.71:6823/2926109885","state":["exists","up"]},{"osd":1,"uuid":"e184604e-b973-4b7a-bba2-6f471753a0fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6827","nonce":345649648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6829","nonce":345649648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6833","nonce":345649648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6831","nonce":345649648}]},"public_addr":"172.21.15.71:6827/345649648","cluster_addr":"172.21.15.71:6829/345649648","heartbeat_back_addr":"172.21.15.71:6833/345649648","heartbeat_front_addr":"172.21.15.71:6831/345649648","state":["exists","up"]},{"osd":2,"uuid":"d8b70f09-5afb-4296-b8c9-719c172e0e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6801","nonce":2721860691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6803","nonce":2721860691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6807","nonce":2721860691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6805","nonce":2721860691}]},"public_addr":"172.21.15.71:6801/2721860691","cluster_addr":"172.21.15.71:6803/2721860691","heartbeat_back_addr":"172.21.15.71:6807/2721860691","heartbeat_front_addr":"172.21.15.71:6805/2721860691","state":["exists","up"]},{"osd":3,"uuid":"26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6809","nonce":494607463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6811","nonce":494607463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6815","nonce":494607463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6813","nonce":494607463}]},"public_addr":"172.21.15.71:6809/494607463","cluster_addr":"172.21.15.71:6811/494607463","heartbeat_back_addr":"172.21.15.71:6815/494607463","heartbeat_front_addr":"172.21.15.71:6813/494607463","state":["exists","up"]},{"osd":4,"uuid":"515cb7d6-16eb-4aff-973a-7f7ed49325ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6817","nonce":2240133417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6819","nonce":2240133417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6823","nonce":2240133417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6821","nonce":2240133417}]},"public_addr":"172.21.15.150:6817/2240133417","cluster_addr":"172.21.15.150:6819/2240133417","heartbeat_back_addr":"172.21.15.150:6823/2240133417","heartbeat_front_addr":"172.21.15.150:6821/2240133417","state":["exists","up"]},{"osd":5,"uuid":"c434b53a-04be-45b9-b89a-193654133950","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6809","nonce":2393825223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6811","nonce":2393825223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6815","nonce":2393825223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6813","nonce":2393825223}]},"public_addr":"172.21.15.150:6809/2393825223","cluster_addr":"172.21.15.150:6811/2393825223","heartbeat_back_addr":"172.21.15.150:6815/2393825223","heartbeat_front_addr":"172.21.15.150:6813/2393825223","state":["exists","up"]},{"osd":6,"uuid":"3d2f8405-9286-4b40-8acd-036f67373ac6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6801","nonce":3235729544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6803","nonce":3235729544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6807","nonce":3235729544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6805","nonce":3235729544}]},"public_addr":"172.21.15.150:6801/3235729544","cluster_addr":"172.21.15.150:6803/3235729544","heartbeat_back_addr":"172.21.15.150:6807/3235729544","heartbeat_front_addr":"172.21.15.150:6805/3235729544","state":["exists","up"]},{"osd":7,"uuid":"6724f080-4dc3-465a-939f-b023985bf998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6825","nonce":2381001355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6827","nonce":2381001355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6831","nonce":2381001355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6829","nonce":2381001355}]},"public_addr":"172.21.15.150:6825/2381001355","cluster_addr":"172.21.15.150:6827/2381001355","heartbeat_back_addr":"172.21.15.150:6831/2381001355","heartbeat_front_addr":"172.21.15.150:6829/2381001355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.548803+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.075023+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.117036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.332113+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.330371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:02.753946+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:02.071186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.378781+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.7","osds":[2,4]},{"pgid":"3.12","osds":[7,0]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T05:31:21.746 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-14T05:31:21.747 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-14T05:31:21.747 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-14T05:31:21.747 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-14T05:31:21.748 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-14T05:31:21.748 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-14T05:31:21.749 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-14T05:31:21.749 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-14T05:31:21.982 INFO:teuthology.orchestra.run.smithi071.stdout:68719476767 2024-09-14T05:31:21.982 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T05:31:21.987 INFO:teuthology.orchestra.run.smithi071.stdout:64424509472 2024-09-14T05:31:21.987 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T05:31:21.990 INFO:teuthology.orchestra.run.smithi071.stdout:64424509471 2024-09-14T05:31:21.990 INFO:teuthology.orchestra.run.smithi071.stdout:64424509471 2024-09-14T05:31:21.991 INFO:teuthology.orchestra.run.smithi071.stdout:64424509470 2024-09-14T05:31:21.991 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T05:31:21.992 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T05:31:21.992 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T05:31:21.993 INFO:teuthology.orchestra.run.smithi071.stdout:64424509470 2024-09-14T05:31:21.994 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T05:31:21.994 INFO:teuthology.orchestra.run.smithi071.stdout:64424509473 2024-09-14T05:31:21.995 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T05:31:21.999 INFO:teuthology.orchestra.run.smithi071.stdout:64424509471 2024-09-14T05:31:21.999 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T05:31:22.547 INFO:teuthology.orchestra.run.smithi071.stdout:68719476766 2024-09-14T05:31:22.561 INFO:teuthology.orchestra.run.smithi071.stdout:64424509470 2024-09-14T05:31:22.638 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476767 got 68719476766 for osd.0 2024-09-14T05:31:22.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509472 got 64424509470 for osd.6 2024-09-14T05:31:22.640 INFO:teuthology.orchestra.run.smithi071.stdout:64424509469 2024-09-14T05:31:22.640 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509471 got 64424509469 for osd.4 2024-09-14T05:31:22.641 INFO:teuthology.orchestra.run.smithi071.stdout:64424509468 2024-09-14T05:31:22.641 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509470 got 64424509468 for osd.5 2024-09-14T05:31:22.641 INFO:teuthology.orchestra.run.smithi071.stdout:64424509469 2024-09-14T05:31:22.652 INFO:teuthology.orchestra.run.smithi071.stdout:64424509468 2024-09-14T05:31:22.653 INFO:teuthology.orchestra.run.smithi071.stdout:64424509469 2024-09-14T05:31:22.653 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509471 got 64424509469 for osd.3 2024-09-14T05:31:22.654 INFO:teuthology.orchestra.run.smithi071.stdout:64424509471 2024-09-14T05:31:22.668 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509471 got 64424509469 for osd.7 2024-09-14T05:31:22.668 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509470 got 64424509468 for osd.2 2024-09-14T05:31:22.669 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509473 got 64424509471 for osd.1 2024-09-14T05:31:23.639 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T05:31:23.640 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T05:31:23.640 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T05:31:23.642 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T05:31:23.655 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T05:31:23.668 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T05:31:23.669 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T05:31:23.670 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T05:31:24.231 INFO:teuthology.orchestra.run.smithi071.stdout:68719476767 2024-09-14T05:31:24.249 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476767 got 68719476767 for osd.0 2024-09-14T05:31:24.250 DEBUG:teuthology.parallel:result is None 2024-09-14T05:31:24.251 INFO:teuthology.orchestra.run.smithi071.stdout:64424509471 2024-09-14T05:31:24.256 INFO:teuthology.orchestra.run.smithi071.stdout:64424509472 2024-09-14T05:31:24.272 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509471 got 64424509471 for osd.4 2024-09-14T05:31:24.273 DEBUG:teuthology.parallel:result is None 2024-09-14T05:31:24.274 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509472 got 64424509472 for osd.6 2024-09-14T05:31:24.274 DEBUG:teuthology.parallel:result is None 2024-09-14T05:31:24.280 INFO:teuthology.orchestra.run.smithi071.stdout:64424509471 2024-09-14T05:31:24.293 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509471 got 64424509471 for osd.7 2024-09-14T05:31:24.293 DEBUG:teuthology.parallel:result is None 2024-09-14T05:31:24.293 INFO:teuthology.orchestra.run.smithi071.stdout:64424509470 2024-09-14T05:31:24.297 INFO:teuthology.orchestra.run.smithi071.stdout:64424509471 2024-09-14T05:31:24.305 INFO:teuthology.orchestra.run.smithi071.stdout:64424509473 2024-09-14T05:31:24.311 INFO:teuthology.orchestra.run.smithi071.stdout:64424509470 2024-09-14T05:31:24.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509470 got 64424509470 for osd.2 2024-09-14T05:31:24.313 DEBUG:teuthology.parallel:result is None 2024-09-14T05:31:24.316 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509471 got 64424509471 for osd.3 2024-09-14T05:31:24.316 DEBUG:teuthology.parallel:result is None 2024-09-14T05:31:24.321 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509473 got 64424509473 for osd.1 2024-09-14T05:31:24.322 DEBUG:teuthology.parallel:result is None 2024-09-14T05:31:24.326 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509470 got 64424509470 for osd.5 2024-09-14T05:31:24.326 DEBUG:teuthology.parallel:result is None 2024-09-14T05:31:24.326 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T05:31:24.326 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:31:24.649 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:31:24.649 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:31:24.661 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-09-14T05:31:24.458624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":598,"num_object_clones":0,"num_object_copies":1196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":598,"num_whiteouts":590,"num_read":182806,"num_read_kb":565819,"num_write":20912,"num_write_kb":282451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19323,"ondisk_log_size":19323,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":339320,"kb_used_data":13428,"kb_used_omap":12,"kb_used_meta":325683,"kb_avail":754635400,"statfs":{"total":773094113280,"available":772746649600,"internally_reserved":0,"allocated":13750272,"data_stored":7146439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":333499952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000951"},"pg_stats":[{"pgid":"3.1f","version":"114'767","reported_seq":10977,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121322+0000","last_change":"2024-09-14T05:31:03.027720+0000","last_active":"2024-09-14T05:31:20.121322+0000","last_peered":"2024-09-14T05:31:20.121322+0000","last_clean":"2024-09-14T05:31:20.121322+0000","last_became_active":"2024-09-14T05:29:24.018892+0000","last_became_peered":"2024-09-14T05:29:24.018892+0000","last_unstale":"2024-09-14T05:31:20.121322+0000","last_undegraded":"2024-09-14T05:31:20.121322+0000","last_fullsized":"2024-09-14T05:31:20.121322+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:00:33.452652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013315950000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":12275,"num_read_kb":37400,"num_write":832,"num_write_kb":31155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"114'418","reported_seq":3310,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121298+0000","last_change":"2024-09-14T05:31:03.027683+0000","last_active":"2024-09-14T05:31:20.121298+0000","last_peered":"2024-09-14T05:31:20.121298+0000","last_clean":"2024-09-14T05:31:20.121298+0000","last_became_active":"2024-09-14T05:29:24.014468+0000","last_became_peered":"2024-09-14T05:29:24.014468+0000","last_unstale":"2024-09-14T05:31:20.121298+0000","last_undegraded":"2024-09-14T05:31:20.121298+0000","last_fullsized":"2024-09-14T05:31:20.121298+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:58.301444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001315826,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2805,"num_read_kb":32498,"num_write":409,"num_write_kb":21098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"113'1047","reported_seq":9884,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128908+0000","last_change":"2024-09-14T05:31:03.020460+0000","last_active":"2024-09-14T05:31:21.128908+0000","last_peered":"2024-09-14T05:31:21.128908+0000","last_clean":"2024-09-14T05:31:21.128908+0000","last_became_active":"2024-09-14T05:29:24.014251+0000","last_became_peered":"2024-09-14T05:29:24.014251+0000","last_unstale":"2024-09-14T05:31:21.128908+0000","last_undegraded":"2024-09-14T05:31:21.128908+0000","last_fullsized":"2024-09-14T05:31:21.128908+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1047,"log_dups_size":0,"ondisk_log_size":1047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:50:10.775143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012210070000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12253,"num_read_kb":23814,"num_write":1553,"num_write_kb":7077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"114'602","reported_seq":10588,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.273068+0000","last_change":"2024-09-14T05:31:03.035040+0000","last_active":"2024-09-14T05:31:20.273068+0000","last_peered":"2024-09-14T05:31:20.273068+0000","last_clean":"2024-09-14T05:31:20.273068+0000","last_became_active":"2024-09-14T05:29:24.012427+0000","last_became_peered":"2024-09-14T05:29:24.012427+0000","last_unstale":"2024-09-14T05:31:20.273068+0000","last_undegraded":"2024-09-14T05:31:20.273068+0000","last_fullsized":"2024-09-14T05:31:20.273068+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:11:08.020997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073664900000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":10982,"num_read_kb":35239,"num_write":569,"num_write_kb":4638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"112'587","reported_seq":3666,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130947+0000","last_change":"2024-09-14T05:31:03.020201+0000","last_active":"2024-09-14T05:31:21.130947+0000","last_peered":"2024-09-14T05:31:21.130947+0000","last_clean":"2024-09-14T05:31:21.130947+0000","last_became_active":"2024-09-14T05:29:24.012079+0000","last_became_peered":"2024-09-14T05:29:24.012079+0000","last_unstale":"2024-09-14T05:31:21.130947+0000","last_undegraded":"2024-09-14T05:31:21.130947+0000","last_fullsized":"2024-09-14T05:31:21.130947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:30:17.989902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010699310000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4414,"num_read_kb":26220,"num_write":576,"num_write_kb":21867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"110'1005","reported_seq":6499,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.273103+0000","last_change":"2024-09-14T05:31:03.035277+0000","last_active":"2024-09-14T05:31:20.273103+0000","last_peered":"2024-09-14T05:31:20.273103+0000","last_clean":"2024-09-14T05:31:20.273103+0000","last_became_active":"2024-09-14T05:29:24.012461+0000","last_became_peered":"2024-09-14T05:29:24.012461+0000","last_unstale":"2024-09-14T05:31:20.273103+0000","last_undegraded":"2024-09-14T05:31:20.273103+0000","last_fullsized":"2024-09-14T05:31:20.273103+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:23:06.828927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090417700000000004,"stat_sum":{"num_bytes":19,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":10524,"num_read_kb":34488,"num_write":1274,"num_write_kb":8459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"112'581","reported_seq":2237,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130927+0000","last_change":"2024-09-14T05:31:03.019768+0000","last_active":"2024-09-14T05:31:21.130927+0000","last_peered":"2024-09-14T05:31:21.130927+0000","last_clean":"2024-09-14T05:31:21.130927+0000","last_became_active":"2024-09-14T05:29:24.018753+0000","last_became_peered":"2024-09-14T05:29:24.018753+0000","last_unstale":"2024-09-14T05:31:21.130927+0000","last_undegraded":"2024-09-14T05:31:21.130927+0000","last_fullsized":"2024-09-14T05:31:21.130927+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:38:28.243765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074058200000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2590,"num_read_kb":13584,"num_write":688,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"113'699","reported_seq":2937,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121320+0000","last_change":"2024-09-14T05:31:03.027199+0000","last_active":"2024-09-14T05:31:20.121320+0000","last_peered":"2024-09-14T05:31:20.121320+0000","last_clean":"2024-09-14T05:31:20.121320+0000","last_became_active":"2024-09-14T05:29:24.011176+0000","last_became_peered":"2024-09-14T05:29:24.011176+0000","last_unstale":"2024-09-14T05:31:20.121320+0000","last_undegraded":"2024-09-14T05:31:20.121320+0000","last_fullsized":"2024-09-14T05:31:20.121320+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:27:54.861994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091515699999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4708,"num_read_kb":7495,"num_write":700,"num_write_kb":3426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"112'357","reported_seq":3176,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121325+0000","last_change":"2024-09-14T05:31:03.028059+0000","last_active":"2024-09-14T05:31:20.121325+0000","last_peered":"2024-09-14T05:31:20.121325+0000","last_clean":"2024-09-14T05:31:20.121325+0000","last_became_active":"2024-09-14T05:29:24.014005+0000","last_became_peered":"2024-09-14T05:29:24.014005+0000","last_unstale":"2024-09-14T05:31:20.121325+0000","last_undegraded":"2024-09-14T05:31:20.121325+0000","last_fullsized":"2024-09-14T05:31:20.121325+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:18:52.725886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001656189,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3149,"num_read_kb":11342,"num_write":316,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"113'340","reported_seq":9918,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521406+0000","last_change":"2024-09-14T05:31:03.023758+0000","last_active":"2024-09-14T05:31:21.521406+0000","last_peered":"2024-09-14T05:31:21.521406+0000","last_clean":"2024-09-14T05:31:21.521406+0000","last_became_active":"2024-09-14T05:29:24.013434+0000","last_became_peered":"2024-09-14T05:29:24.013434+0000","last_unstale":"2024-09-14T05:31:21.521406+0000","last_undegraded":"2024-09-14T05:31:21.521406+0000","last_fullsized":"2024-09-14T05:31:21.521406+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:44.069349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418475,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":9226,"num_read_kb":22920,"num_write":296,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"113'440","reported_seq":2749,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.188832+0000","last_change":"2024-09-14T05:31:03.068934+0000","last_active":"2024-09-14T05:31:22.188832+0000","last_peered":"2024-09-14T05:31:22.188832+0000","last_clean":"2024-09-14T05:31:22.188832+0000","last_became_active":"2024-09-14T05:29:24.014457+0000","last_became_peered":"2024-09-14T05:29:24.014457+0000","last_unstale":"2024-09-14T05:31:22.188832+0000","last_undegraded":"2024-09-14T05:31:22.188832+0000","last_fullsized":"2024-09-14T05:31:22.188832+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:01:09.992476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011032850000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2612,"num_read_kb":9349,"num_write":416,"num_write_kb":12875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"109'843","reported_seq":2591,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128823+0000","last_change":"2024-09-14T05:31:03.020119+0000","last_active":"2024-09-14T05:31:21.128823+0000","last_peered":"2024-09-14T05:31:21.128823+0000","last_clean":"2024-09-14T05:31:21.128823+0000","last_became_active":"2024-09-14T05:29:24.013266+0000","last_became_peered":"2024-09-14T05:29:24.013266+0000","last_unstale":"2024-09-14T05:31:21.128823+0000","last_undegraded":"2024-09-14T05:31:21.128823+0000","last_fullsized":"2024-09-14T05:31:21.128823+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:34:03.137226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087100800000000005,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":4368,"num_read_kb":5632,"num_write":1075,"num_write_kb":4026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":240,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128815+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:31:21.128815+0000","last_peered":"2024-09-14T05:31:21.128815+0000","last_clean":"2024-09-14T05:31:21.128815+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:31:21.128815+0000","last_undegraded":"2024-09-14T05:31:21.128815+0000","last_fullsized":"2024-09-14T05:31:21.128815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"114'375","reported_seq":2319,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.131016+0000","last_change":"2024-09-14T05:31:03.020581+0000","last_active":"2024-09-14T05:31:21.131016+0000","last_peered":"2024-09-14T05:31:21.131016+0000","last_clean":"2024-09-14T05:31:21.131016+0000","last_became_active":"2024-09-14T05:29:24.012099+0000","last_became_peered":"2024-09-14T05:29:24.012099+0000","last_unstale":"2024-09-14T05:31:21.131016+0000","last_undegraded":"2024-09-14T05:31:21.131016+0000","last_fullsized":"2024-09-14T05:31:21.131016+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:44:38.747225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013882689999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2144,"num_read_kb":3725,"num_write":284,"num_write_kb":5458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":235,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.131025+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:31:21.131025+0000","last_peered":"2024-09-14T05:31:21.131025+0000","last_clean":"2024-09-14T05:31:21.131025+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:31:21.131025+0000","last_undegraded":"2024-09-14T05:31:21.131025+0000","last_fullsized":"2024-09-14T05:31:21.131025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"114'951","reported_seq":4046,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521397+0000","last_change":"2024-09-14T05:31:03.023762+0000","last_active":"2024-09-14T05:31:21.521397+0000","last_peered":"2024-09-14T05:31:21.521397+0000","last_clean":"2024-09-14T05:31:21.521397+0000","last_became_active":"2024-09-14T05:30:31.398064+0000","last_became_peered":"2024-09-14T05:30:31.398064+0000","last_unstale":"2024-09-14T05:31:21.521397+0000","last_undegraded":"2024-09-14T05:31:21.521397+0000","last_fullsized":"2024-09-14T05:31:21.521397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:15:44.039992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014417360000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6829,"num_read_kb":8465,"num_write":1096,"num_write_kb":5167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":648894,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128980+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:31:21.128980+0000","last_peered":"2024-09-14T05:31:21.128980+0000","last_clean":"2024-09-14T05:31:21.128980+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:31:21.128980+0000","last_undegraded":"2024-09-14T05:31:21.128980+0000","last_fullsized":"2024-09-14T05:31:21.128980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"112'569","reported_seq":2718,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130208+0000","last_change":"2024-09-14T05:31:20.123367+0000","last_active":"2024-09-14T05:31:21.130208+0000","last_peered":"2024-09-14T05:31:21.130208+0000","last_clean":"2024-09-14T05:31:21.130208+0000","last_became_active":"2024-09-14T05:31:20.122139+0000","last_became_peered":"2024-09-14T05:31:20.122139+0000","last_unstale":"2024-09-14T05:31:21.130208+0000","last_undegraded":"2024-09-14T05:31:21.130208+0000","last_fullsized":"2024-09-14T05:31:21.130208+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:03:12.206743+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0050460000000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3487,"num_read_kb":12454,"num_write":614,"num_write_kb":11389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":234,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121553+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:31:20.121553+0000","last_peered":"2024-09-14T05:31:20.121553+0000","last_clean":"2024-09-14T05:31:20.121553+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:31:20.121553+0000","last_undegraded":"2024-09-14T05:31:20.121553+0000","last_fullsized":"2024-09-14T05:31:20.121553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"114'351","reported_seq":1951,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521367+0000","last_change":"2024-09-14T05:31:03.024292+0000","last_active":"2024-09-14T05:31:21.521367+0000","last_peered":"2024-09-14T05:31:21.521367+0000","last_clean":"2024-09-14T05:31:21.521367+0000","last_became_active":"2024-09-14T05:29:24.016940+0000","last_became_peered":"2024-09-14T05:29:24.016940+0000","last_unstale":"2024-09-14T05:31:21.521367+0000","last_undegraded":"2024-09-14T05:31:21.521367+0000","last_fullsized":"2024-09-14T05:31:21.521367+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:42:00.348036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001928208,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1597,"num_read_kb":4408,"num_write":335,"num_write_kb":4001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"114'642","reported_seq":3022,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189410+0000","last_change":"2024-09-14T05:31:03.069465+0000","last_active":"2024-09-14T05:31:22.189410+0000","last_peered":"2024-09-14T05:31:22.189410+0000","last_clean":"2024-09-14T05:31:22.189410+0000","last_became_active":"2024-09-14T05:29:24.015905+0000","last_became_peered":"2024-09-14T05:29:24.015905+0000","last_unstale":"2024-09-14T05:31:22.189410+0000","last_undegraded":"2024-09-14T05:31:22.189410+0000","last_fullsized":"2024-09-14T05:31:22.189410+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:31:28.697477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00154122,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":2532,"num_read_kb":9709,"num_write":637,"num_write_kb":7772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":243,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128727+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:31:21.128727+0000","last_peered":"2024-09-14T05:31:21.128727+0000","last_clean":"2024-09-14T05:31:21.128727+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:31:21.128727+0000","last_undegraded":"2024-09-14T05:31:21.128727+0000","last_fullsized":"2024-09-14T05:31:21.128727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"113'485","reported_seq":10963,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121177+0000","last_change":"2024-09-14T05:31:03.026777+0000","last_active":"2024-09-14T05:31:20.121177+0000","last_peered":"2024-09-14T05:31:20.121177+0000","last_clean":"2024-09-14T05:31:20.121177+0000","last_became_active":"2024-09-14T05:29:24.011764+0000","last_became_peered":"2024-09-14T05:29:24.011764+0000","last_unstale":"2024-09-14T05:31:20.121177+0000","last_undegraded":"2024-09-14T05:31:20.121177+0000","last_fullsized":"2024-09-14T05:31:20.121177+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:07.795205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057339999999999995,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":11805,"num_read_kb":38991,"num_write":413,"num_write_kb":4959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":237,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.270563+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:31:20.270563+0000","last_peered":"2024-09-14T05:31:20.270563+0000","last_clean":"2024-09-14T05:31:20.270563+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:31:20.270563+0000","last_undegraded":"2024-09-14T05:31:20.270563+0000","last_fullsized":"2024-09-14T05:31:20.270563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"110'701","reported_seq":1977,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130208+0000","last_change":"2024-09-14T05:31:20.123603+0000","last_active":"2024-09-14T05:31:21.130208+0000","last_peered":"2024-09-14T05:31:21.130208+0000","last_clean":"2024-09-14T05:31:21.130208+0000","last_became_active":"2024-09-14T05:31:20.122523+0000","last_became_peered":"2024-09-14T05:31:20.122523+0000","last_unstale":"2024-09-14T05:31:21.130208+0000","last_undegraded":"2024-09-14T05:31:21.130208+0000","last_fullsized":"2024-09-14T05:31:21.130208+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:44:47.461984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022421440000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3140,"num_read_kb":7266,"num_write":925,"num_write_kb":10032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":241,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128547+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:31:21.128547+0000","last_peered":"2024-09-14T05:31:21.128547+0000","last_clean":"2024-09-14T05:31:21.128547+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:31:21.128547+0000","last_undegraded":"2024-09-14T05:31:21.128547+0000","last_fullsized":"2024-09-14T05:31:21.128547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"113'585","reported_seq":2767,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130931+0000","last_change":"2024-09-14T05:31:03.020204+0000","last_active":"2024-09-14T05:31:21.130931+0000","last_peered":"2024-09-14T05:31:21.130931+0000","last_clean":"2024-09-14T05:31:21.130931+0000","last_became_active":"2024-09-14T05:29:24.015792+0000","last_became_peered":"2024-09-14T05:29:24.015792+0000","last_unstale":"2024-09-14T05:31:21.130931+0000","last_undegraded":"2024-09-14T05:31:21.130931+0000","last_fullsized":"2024-09-14T05:31:21.130931+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:12:45.477846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011628109999999999,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":3532,"num_read_kb":9462,"num_write":620,"num_write_kb":5631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":242,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128830+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:31:21.128830+0000","last_peered":"2024-09-14T05:31:21.128830+0000","last_clean":"2024-09-14T05:31:21.128830+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:31:21.128830+0000","last_undegraded":"2024-09-14T05:31:21.128830+0000","last_fullsized":"2024-09-14T05:31:21.128830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":327,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.272917+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:31:20.272917+0000","last_peered":"2024-09-14T05:31:20.272917+0000","last_clean":"2024-09-14T05:31:20.272917+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:31:20.272917+0000","last_undegraded":"2024-09-14T05:31:20.272917+0000","last_fullsized":"2024-09-14T05:31:20.272917+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"113'395","reported_seq":2290,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130890+0000","last_change":"2024-09-14T05:31:03.019738+0000","last_active":"2024-09-14T05:31:21.130890+0000","last_peered":"2024-09-14T05:31:21.130890+0000","last_clean":"2024-09-14T05:31:21.130890+0000","last_became_active":"2024-09-14T05:29:24.012222+0000","last_became_peered":"2024-09-14T05:29:24.012222+0000","last_unstale":"2024-09-14T05:31:21.130890+0000","last_undegraded":"2024-09-14T05:31:21.130890+0000","last_fullsized":"2024-09-14T05:31:21.130890+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:52:56.721362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072525400000000005,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2833,"num_read_kb":6029,"num_write":327,"num_write_kb":3878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"113'608","reported_seq":2440,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128564+0000","last_change":"2024-09-14T05:31:03.020704+0000","last_active":"2024-09-14T05:31:21.128564+0000","last_peered":"2024-09-14T05:31:21.128564+0000","last_clean":"2024-09-14T05:31:21.128564+0000","last_became_active":"2024-09-14T05:29:24.012041+0000","last_became_peered":"2024-09-14T05:29:24.012041+0000","last_unstale":"2024-09-14T05:31:21.128564+0000","last_undegraded":"2024-09-14T05:31:21.128564+0000","last_fullsized":"2024-09-14T05:31:21.128564+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:42:01.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001433679,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2914,"num_read_kb":7354,"num_write":748,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"113'655","reported_seq":2691,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128609+0000","last_change":"2024-09-14T05:31:03.021329+0000","last_active":"2024-09-14T05:31:21.128609+0000","last_peered":"2024-09-14T05:31:21.128609+0000","last_clean":"2024-09-14T05:31:21.128609+0000","last_became_active":"2024-09-14T05:29:24.011403+0000","last_became_peered":"2024-09-14T05:29:24.011403+0000","last_unstale":"2024-09-14T05:31:21.128609+0000","last_undegraded":"2024-09-14T05:31:21.128609+0000","last_fullsized":"2024-09-14T05:31:21.128609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:04:32.706037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00168743,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":3956,"num_read_kb":8037,"num_write":714,"num_write_kb":4980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"112'632","reported_seq":10267,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.270508+0000","last_change":"2024-09-14T05:31:03.032462+0000","last_active":"2024-09-14T05:31:20.270508+0000","last_peered":"2024-09-14T05:31:20.270508+0000","last_clean":"2024-09-14T05:31:20.270508+0000","last_became_active":"2024-09-14T05:29:24.006836+0000","last_became_peered":"2024-09-14T05:29:24.006836+0000","last_unstale":"2024-09-14T05:31:20.270508+0000","last_undegraded":"2024-09-14T05:31:20.270508+0000","last_fullsized":"2024-09-14T05:31:20.270508+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:22:08.301740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060523399999999996,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":12123,"num_read_kb":29901,"num_write":689,"num_write_kb":13072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"113'448","reported_seq":3554,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189014+0000","last_change":"2024-09-14T05:31:03.068410+0000","last_active":"2024-09-14T05:31:22.189014+0000","last_peered":"2024-09-14T05:31:22.189014+0000","last_clean":"2024-09-14T05:31:22.189014+0000","last_became_active":"2024-09-14T05:29:24.012574+0000","last_became_peered":"2024-09-14T05:29:24.012574+0000","last_unstale":"2024-09-14T05:31:22.189014+0000","last_undegraded":"2024-09-14T05:31:22.189014+0000","last_fullsized":"2024-09-14T05:31:22.189014+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:58:16.204790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064954900000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2817,"num_read_kb":13317,"num_write":361,"num_write_kb":9799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"112'1131","reported_seq":11394,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128943+0000","last_change":"2024-09-14T05:31:03.021253+0000","last_active":"2024-09-14T05:31:21.128943+0000","last_peered":"2024-09-14T05:31:21.128943+0000","last_clean":"2024-09-14T05:31:21.128943+0000","last_became_active":"2024-09-14T05:29:24.012401+0000","last_became_peered":"2024-09-14T05:29:24.012401+0000","last_unstale":"2024-09-14T05:31:21.128943+0000","last_undegraded":"2024-09-14T05:31:21.128943+0000","last_fullsized":"2024-09-14T05:31:21.128943+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1131,"log_dups_size":0,"ondisk_log_size":1131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:18:41.728775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018223110000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":14909,"num_read_kb":22344,"num_write":1338,"num_write_kb":6495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"113'367","reported_seq":2862,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521474+0000","last_change":"2024-09-14T05:31:03.024269+0000","last_active":"2024-09-14T05:31:21.521474+0000","last_peered":"2024-09-14T05:31:21.521474+0000","last_clean":"2024-09-14T05:31:21.521474+0000","last_became_active":"2024-09-14T05:29:24.015315+0000","last_became_peered":"2024-09-14T05:29:24.015315+0000","last_unstale":"2024-09-14T05:31:21.521474+0000","last_undegraded":"2024-09-14T05:31:21.521474+0000","last_fullsized":"2024-09-14T05:31:21.521474+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:27:50.755671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001918957,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2150,"num_read_kb":8871,"num_write":290,"num_write_kb":3939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"111'532","reported_seq":11844,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189148+0000","last_change":"2024-09-14T05:31:03.068658+0000","last_active":"2024-09-14T05:31:22.189148+0000","last_peered":"2024-09-14T05:31:22.189148+0000","last_clean":"2024-09-14T05:31:22.189148+0000","last_became_active":"2024-09-14T05:29:24.018854+0000","last_became_peered":"2024-09-14T05:29:24.018854+0000","last_unstale":"2024-09-14T05:31:22.189148+0000","last_undegraded":"2024-09-14T05:31:22.189148+0000","last_fullsized":"2024-09-14T05:31:22.189148+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:12:27.055181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000829428,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":12059,"num_read_kb":31916,"num_write":436,"num_write_kb":11843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"113'389","reported_seq":3039,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.272918+0000","last_change":"2024-09-14T05:31:03.034867+0000","last_active":"2024-09-14T05:31:20.272918+0000","last_peered":"2024-09-14T05:31:20.272918+0000","last_clean":"2024-09-14T05:31:20.272918+0000","last_became_active":"2024-09-14T05:29:24.010897+0000","last_became_peered":"2024-09-14T05:29:24.010897+0000","last_unstale":"2024-09-14T05:31:20.272918+0000","last_undegraded":"2024-09-14T05:31:20.272918+0000","last_fullsized":"2024-09-14T05:31:20.272918+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:33:58.717977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051683800000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2374,"num_read_kb":22456,"num_write":338,"num_write_kb":8939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"112'576","reported_seq":4039,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128547+0000","last_change":"2024-09-14T05:31:03.020915+0000","last_active":"2024-09-14T05:31:21.128547+0000","last_peered":"2024-09-14T05:31:21.128547+0000","last_clean":"2024-09-14T05:31:21.128547+0000","last_became_active":"2024-09-14T05:29:24.013792+0000","last_became_peered":"2024-09-14T05:29:24.013792+0000","last_unstale":"2024-09-14T05:31:21.128547+0000","last_undegraded":"2024-09-14T05:31:21.128547+0000","last_fullsized":"2024-09-14T05:31:21.128547+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:09:14.825887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001768403,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5197,"num_read_kb":24484,"num_write":526,"num_write_kb":3336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"112'490","reported_seq":2632,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.270506+0000","last_change":"2024-09-14T05:31:03.032142+0000","last_active":"2024-09-14T05:31:20.270506+0000","last_peered":"2024-09-14T05:31:20.270506+0000","last_clean":"2024-09-14T05:31:20.270506+0000","last_became_active":"2024-09-14T05:30:38.013627+0000","last_became_peered":"2024-09-14T05:30:38.013627+0000","last_unstale":"2024-09-14T05:31:20.270506+0000","last_undegraded":"2024-09-14T05:31:20.270506+0000","last_fullsized":"2024-09-14T05:31:20.270506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:55:09.154072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050464100000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3016,"num_read_kb":16678,"num_write":502,"num_write_kb":10866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2769134,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"113'680","reported_seq":2493,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128721+0000","last_change":"2024-09-14T05:31:03.019963+0000","last_active":"2024-09-14T05:31:21.128721+0000","last_peered":"2024-09-14T05:31:21.128721+0000","last_clean":"2024-09-14T05:31:21.128721+0000","last_became_active":"2024-09-14T05:29:24.014371+0000","last_became_peered":"2024-09-14T05:29:24.014371+0000","last_unstale":"2024-09-14T05:31:21.128721+0000","last_undegraded":"2024-09-14T05:31:21.128721+0000","last_fullsized":"2024-09-14T05:31:21.128721+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:06.352481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080910499999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3420,"num_read_kb":19916,"num_write":912,"num_write_kb":5503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":594,"num_object_clones":0,"num_object_copies":1188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":594,"num_whiteouts":590,"num_read":182740,"num_read_kb":565764,"num_write":20809,"num_write_kb":280821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19248,"ondisk_log_size":19248,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476768,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50776,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321064,"statfs":{"total":96636764160,"available":96584769536,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509471,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41036,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330804,"statfs":{"total":96636764160,"available":96594743296,"internally_reserved":0,"allocated":1560576,"data_stored":742034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28576,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343264,"statfs":{"total":96636764160,"available":96607502336,"internally_reserved":0,"allocated":1576960,"data_stored":747924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2158592,"data_stored":1332421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28568,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343272,"statfs":{"total":96636764160,"available":96607510528,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72988,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298852,"statfs":{"total":96636764160,"available":96562024448,"internally_reserved":0,"allocated":1576960,"data_stored":747924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509473,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51348,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320492,"statfs":{"total":96636764160,"available":96584183808,"internally_reserved":0,"allocated":2158592,"data_stored":1332421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:31:24.663 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:31:24.943 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:31:24.944 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:31:24.955 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-09-14T05:31:24.458624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":598,"num_object_clones":0,"num_object_copies":1196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":598,"num_whiteouts":590,"num_read":182806,"num_read_kb":565819,"num_write":20912,"num_write_kb":282451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19323,"ondisk_log_size":19323,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":339320,"kb_used_data":13428,"kb_used_omap":12,"kb_used_meta":325683,"kb_avail":754635400,"statfs":{"total":773094113280,"available":772746649600,"internally_reserved":0,"allocated":13750272,"data_stored":7146439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":333499952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000951"},"pg_stats":[{"pgid":"3.1f","version":"114'767","reported_seq":10977,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121322+0000","last_change":"2024-09-14T05:31:03.027720+0000","last_active":"2024-09-14T05:31:20.121322+0000","last_peered":"2024-09-14T05:31:20.121322+0000","last_clean":"2024-09-14T05:31:20.121322+0000","last_became_active":"2024-09-14T05:29:24.018892+0000","last_became_peered":"2024-09-14T05:29:24.018892+0000","last_unstale":"2024-09-14T05:31:20.121322+0000","last_undegraded":"2024-09-14T05:31:20.121322+0000","last_fullsized":"2024-09-14T05:31:20.121322+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:00:33.452652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013315950000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":12275,"num_read_kb":37400,"num_write":832,"num_write_kb":31155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"114'418","reported_seq":3310,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121298+0000","last_change":"2024-09-14T05:31:03.027683+0000","last_active":"2024-09-14T05:31:20.121298+0000","last_peered":"2024-09-14T05:31:20.121298+0000","last_clean":"2024-09-14T05:31:20.121298+0000","last_became_active":"2024-09-14T05:29:24.014468+0000","last_became_peered":"2024-09-14T05:29:24.014468+0000","last_unstale":"2024-09-14T05:31:20.121298+0000","last_undegraded":"2024-09-14T05:31:20.121298+0000","last_fullsized":"2024-09-14T05:31:20.121298+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:58.301444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001315826,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2805,"num_read_kb":32498,"num_write":409,"num_write_kb":21098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"113'1047","reported_seq":9884,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128908+0000","last_change":"2024-09-14T05:31:03.020460+0000","last_active":"2024-09-14T05:31:21.128908+0000","last_peered":"2024-09-14T05:31:21.128908+0000","last_clean":"2024-09-14T05:31:21.128908+0000","last_became_active":"2024-09-14T05:29:24.014251+0000","last_became_peered":"2024-09-14T05:29:24.014251+0000","last_unstale":"2024-09-14T05:31:21.128908+0000","last_undegraded":"2024-09-14T05:31:21.128908+0000","last_fullsized":"2024-09-14T05:31:21.128908+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1047,"log_dups_size":0,"ondisk_log_size":1047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:50:10.775143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012210070000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12253,"num_read_kb":23814,"num_write":1553,"num_write_kb":7077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"114'602","reported_seq":10588,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.273068+0000","last_change":"2024-09-14T05:31:03.035040+0000","last_active":"2024-09-14T05:31:20.273068+0000","last_peered":"2024-09-14T05:31:20.273068+0000","last_clean":"2024-09-14T05:31:20.273068+0000","last_became_active":"2024-09-14T05:29:24.012427+0000","last_became_peered":"2024-09-14T05:29:24.012427+0000","last_unstale":"2024-09-14T05:31:20.273068+0000","last_undegraded":"2024-09-14T05:31:20.273068+0000","last_fullsized":"2024-09-14T05:31:20.273068+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:11:08.020997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073664900000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":10982,"num_read_kb":35239,"num_write":569,"num_write_kb":4638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"112'587","reported_seq":3666,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130947+0000","last_change":"2024-09-14T05:31:03.020201+0000","last_active":"2024-09-14T05:31:21.130947+0000","last_peered":"2024-09-14T05:31:21.130947+0000","last_clean":"2024-09-14T05:31:21.130947+0000","last_became_active":"2024-09-14T05:29:24.012079+0000","last_became_peered":"2024-09-14T05:29:24.012079+0000","last_unstale":"2024-09-14T05:31:21.130947+0000","last_undegraded":"2024-09-14T05:31:21.130947+0000","last_fullsized":"2024-09-14T05:31:21.130947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:30:17.989902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010699310000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4414,"num_read_kb":26220,"num_write":576,"num_write_kb":21867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"110'1005","reported_seq":6499,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.273103+0000","last_change":"2024-09-14T05:31:03.035277+0000","last_active":"2024-09-14T05:31:20.273103+0000","last_peered":"2024-09-14T05:31:20.273103+0000","last_clean":"2024-09-14T05:31:20.273103+0000","last_became_active":"2024-09-14T05:29:24.012461+0000","last_became_peered":"2024-09-14T05:29:24.012461+0000","last_unstale":"2024-09-14T05:31:20.273103+0000","last_undegraded":"2024-09-14T05:31:20.273103+0000","last_fullsized":"2024-09-14T05:31:20.273103+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:23:06.828927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090417700000000004,"stat_sum":{"num_bytes":19,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":10524,"num_read_kb":34488,"num_write":1274,"num_write_kb":8459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"112'581","reported_seq":2237,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130927+0000","last_change":"2024-09-14T05:31:03.019768+0000","last_active":"2024-09-14T05:31:21.130927+0000","last_peered":"2024-09-14T05:31:21.130927+0000","last_clean":"2024-09-14T05:31:21.130927+0000","last_became_active":"2024-09-14T05:29:24.018753+0000","last_became_peered":"2024-09-14T05:29:24.018753+0000","last_unstale":"2024-09-14T05:31:21.130927+0000","last_undegraded":"2024-09-14T05:31:21.130927+0000","last_fullsized":"2024-09-14T05:31:21.130927+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:38:28.243765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074058200000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2590,"num_read_kb":13584,"num_write":688,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"113'699","reported_seq":2937,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121320+0000","last_change":"2024-09-14T05:31:03.027199+0000","last_active":"2024-09-14T05:31:20.121320+0000","last_peered":"2024-09-14T05:31:20.121320+0000","last_clean":"2024-09-14T05:31:20.121320+0000","last_became_active":"2024-09-14T05:29:24.011176+0000","last_became_peered":"2024-09-14T05:29:24.011176+0000","last_unstale":"2024-09-14T05:31:20.121320+0000","last_undegraded":"2024-09-14T05:31:20.121320+0000","last_fullsized":"2024-09-14T05:31:20.121320+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:27:54.861994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091515699999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4708,"num_read_kb":7495,"num_write":700,"num_write_kb":3426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"112'357","reported_seq":3176,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121325+0000","last_change":"2024-09-14T05:31:03.028059+0000","last_active":"2024-09-14T05:31:20.121325+0000","last_peered":"2024-09-14T05:31:20.121325+0000","last_clean":"2024-09-14T05:31:20.121325+0000","last_became_active":"2024-09-14T05:29:24.014005+0000","last_became_peered":"2024-09-14T05:29:24.014005+0000","last_unstale":"2024-09-14T05:31:20.121325+0000","last_undegraded":"2024-09-14T05:31:20.121325+0000","last_fullsized":"2024-09-14T05:31:20.121325+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:18:52.725886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001656189,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3149,"num_read_kb":11342,"num_write":316,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"113'340","reported_seq":9918,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521406+0000","last_change":"2024-09-14T05:31:03.023758+0000","last_active":"2024-09-14T05:31:21.521406+0000","last_peered":"2024-09-14T05:31:21.521406+0000","last_clean":"2024-09-14T05:31:21.521406+0000","last_became_active":"2024-09-14T05:29:24.013434+0000","last_became_peered":"2024-09-14T05:29:24.013434+0000","last_unstale":"2024-09-14T05:31:21.521406+0000","last_undegraded":"2024-09-14T05:31:21.521406+0000","last_fullsized":"2024-09-14T05:31:21.521406+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:44.069349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418475,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":9226,"num_read_kb":22920,"num_write":296,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"113'440","reported_seq":2749,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.188832+0000","last_change":"2024-09-14T05:31:03.068934+0000","last_active":"2024-09-14T05:31:22.188832+0000","last_peered":"2024-09-14T05:31:22.188832+0000","last_clean":"2024-09-14T05:31:22.188832+0000","last_became_active":"2024-09-14T05:29:24.014457+0000","last_became_peered":"2024-09-14T05:29:24.014457+0000","last_unstale":"2024-09-14T05:31:22.188832+0000","last_undegraded":"2024-09-14T05:31:22.188832+0000","last_fullsized":"2024-09-14T05:31:22.188832+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:01:09.992476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011032850000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2612,"num_read_kb":9349,"num_write":416,"num_write_kb":12875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"109'843","reported_seq":2591,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128823+0000","last_change":"2024-09-14T05:31:03.020119+0000","last_active":"2024-09-14T05:31:21.128823+0000","last_peered":"2024-09-14T05:31:21.128823+0000","last_clean":"2024-09-14T05:31:21.128823+0000","last_became_active":"2024-09-14T05:29:24.013266+0000","last_became_peered":"2024-09-14T05:29:24.013266+0000","last_unstale":"2024-09-14T05:31:21.128823+0000","last_undegraded":"2024-09-14T05:31:21.128823+0000","last_fullsized":"2024-09-14T05:31:21.128823+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:34:03.137226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087100800000000005,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":4368,"num_read_kb":5632,"num_write":1075,"num_write_kb":4026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":240,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128815+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:31:21.128815+0000","last_peered":"2024-09-14T05:31:21.128815+0000","last_clean":"2024-09-14T05:31:21.128815+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:31:21.128815+0000","last_undegraded":"2024-09-14T05:31:21.128815+0000","last_fullsized":"2024-09-14T05:31:21.128815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"114'375","reported_seq":2319,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.131016+0000","last_change":"2024-09-14T05:31:03.020581+0000","last_active":"2024-09-14T05:31:21.131016+0000","last_peered":"2024-09-14T05:31:21.131016+0000","last_clean":"2024-09-14T05:31:21.131016+0000","last_became_active":"2024-09-14T05:29:24.012099+0000","last_became_peered":"2024-09-14T05:29:24.012099+0000","last_unstale":"2024-09-14T05:31:21.131016+0000","last_undegraded":"2024-09-14T05:31:21.131016+0000","last_fullsized":"2024-09-14T05:31:21.131016+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:44:38.747225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013882689999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2144,"num_read_kb":3725,"num_write":284,"num_write_kb":5458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":235,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.131025+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:31:21.131025+0000","last_peered":"2024-09-14T05:31:21.131025+0000","last_clean":"2024-09-14T05:31:21.131025+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:31:21.131025+0000","last_undegraded":"2024-09-14T05:31:21.131025+0000","last_fullsized":"2024-09-14T05:31:21.131025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"114'951","reported_seq":4046,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521397+0000","last_change":"2024-09-14T05:31:03.023762+0000","last_active":"2024-09-14T05:31:21.521397+0000","last_peered":"2024-09-14T05:31:21.521397+0000","last_clean":"2024-09-14T05:31:21.521397+0000","last_became_active":"2024-09-14T05:30:31.398064+0000","last_became_peered":"2024-09-14T05:30:31.398064+0000","last_unstale":"2024-09-14T05:31:21.521397+0000","last_undegraded":"2024-09-14T05:31:21.521397+0000","last_fullsized":"2024-09-14T05:31:21.521397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:15:44.039992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014417360000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6829,"num_read_kb":8465,"num_write":1096,"num_write_kb":5167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":648894,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128980+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:31:21.128980+0000","last_peered":"2024-09-14T05:31:21.128980+0000","last_clean":"2024-09-14T05:31:21.128980+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:31:21.128980+0000","last_undegraded":"2024-09-14T05:31:21.128980+0000","last_fullsized":"2024-09-14T05:31:21.128980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"112'569","reported_seq":2718,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130208+0000","last_change":"2024-09-14T05:31:20.123367+0000","last_active":"2024-09-14T05:31:21.130208+0000","last_peered":"2024-09-14T05:31:21.130208+0000","last_clean":"2024-09-14T05:31:21.130208+0000","last_became_active":"2024-09-14T05:31:20.122139+0000","last_became_peered":"2024-09-14T05:31:20.122139+0000","last_unstale":"2024-09-14T05:31:21.130208+0000","last_undegraded":"2024-09-14T05:31:21.130208+0000","last_fullsized":"2024-09-14T05:31:21.130208+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:03:12.206743+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0050460000000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3487,"num_read_kb":12454,"num_write":614,"num_write_kb":11389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":234,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121553+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:31:20.121553+0000","last_peered":"2024-09-14T05:31:20.121553+0000","last_clean":"2024-09-14T05:31:20.121553+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:31:20.121553+0000","last_undegraded":"2024-09-14T05:31:20.121553+0000","last_fullsized":"2024-09-14T05:31:20.121553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"114'351","reported_seq":1951,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521367+0000","last_change":"2024-09-14T05:31:03.024292+0000","last_active":"2024-09-14T05:31:21.521367+0000","last_peered":"2024-09-14T05:31:21.521367+0000","last_clean":"2024-09-14T05:31:21.521367+0000","last_became_active":"2024-09-14T05:29:24.016940+0000","last_became_peered":"2024-09-14T05:29:24.016940+0000","last_unstale":"2024-09-14T05:31:21.521367+0000","last_undegraded":"2024-09-14T05:31:21.521367+0000","last_fullsized":"2024-09-14T05:31:21.521367+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:42:00.348036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001928208,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1597,"num_read_kb":4408,"num_write":335,"num_write_kb":4001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"114'642","reported_seq":3022,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189410+0000","last_change":"2024-09-14T05:31:03.069465+0000","last_active":"2024-09-14T05:31:22.189410+0000","last_peered":"2024-09-14T05:31:22.189410+0000","last_clean":"2024-09-14T05:31:22.189410+0000","last_became_active":"2024-09-14T05:29:24.015905+0000","last_became_peered":"2024-09-14T05:29:24.015905+0000","last_unstale":"2024-09-14T05:31:22.189410+0000","last_undegraded":"2024-09-14T05:31:22.189410+0000","last_fullsized":"2024-09-14T05:31:22.189410+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:31:28.697477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00154122,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":2532,"num_read_kb":9709,"num_write":637,"num_write_kb":7772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":243,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128727+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:31:21.128727+0000","last_peered":"2024-09-14T05:31:21.128727+0000","last_clean":"2024-09-14T05:31:21.128727+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:31:21.128727+0000","last_undegraded":"2024-09-14T05:31:21.128727+0000","last_fullsized":"2024-09-14T05:31:21.128727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"113'485","reported_seq":10963,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121177+0000","last_change":"2024-09-14T05:31:03.026777+0000","last_active":"2024-09-14T05:31:20.121177+0000","last_peered":"2024-09-14T05:31:20.121177+0000","last_clean":"2024-09-14T05:31:20.121177+0000","last_became_active":"2024-09-14T05:29:24.011764+0000","last_became_peered":"2024-09-14T05:29:24.011764+0000","last_unstale":"2024-09-14T05:31:20.121177+0000","last_undegraded":"2024-09-14T05:31:20.121177+0000","last_fullsized":"2024-09-14T05:31:20.121177+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:07.795205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057339999999999995,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":11805,"num_read_kb":38991,"num_write":413,"num_write_kb":4959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":237,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.270563+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:31:20.270563+0000","last_peered":"2024-09-14T05:31:20.270563+0000","last_clean":"2024-09-14T05:31:20.270563+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:31:20.270563+0000","last_undegraded":"2024-09-14T05:31:20.270563+0000","last_fullsized":"2024-09-14T05:31:20.270563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"110'701","reported_seq":1977,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130208+0000","last_change":"2024-09-14T05:31:20.123603+0000","last_active":"2024-09-14T05:31:21.130208+0000","last_peered":"2024-09-14T05:31:21.130208+0000","last_clean":"2024-09-14T05:31:21.130208+0000","last_became_active":"2024-09-14T05:31:20.122523+0000","last_became_peered":"2024-09-14T05:31:20.122523+0000","last_unstale":"2024-09-14T05:31:21.130208+0000","last_undegraded":"2024-09-14T05:31:21.130208+0000","last_fullsized":"2024-09-14T05:31:21.130208+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:44:47.461984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022421440000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3140,"num_read_kb":7266,"num_write":925,"num_write_kb":10032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":241,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128547+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:31:21.128547+0000","last_peered":"2024-09-14T05:31:21.128547+0000","last_clean":"2024-09-14T05:31:21.128547+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:31:21.128547+0000","last_undegraded":"2024-09-14T05:31:21.128547+0000","last_fullsized":"2024-09-14T05:31:21.128547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"113'585","reported_seq":2767,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130931+0000","last_change":"2024-09-14T05:31:03.020204+0000","last_active":"2024-09-14T05:31:21.130931+0000","last_peered":"2024-09-14T05:31:21.130931+0000","last_clean":"2024-09-14T05:31:21.130931+0000","last_became_active":"2024-09-14T05:29:24.015792+0000","last_became_peered":"2024-09-14T05:29:24.015792+0000","last_unstale":"2024-09-14T05:31:21.130931+0000","last_undegraded":"2024-09-14T05:31:21.130931+0000","last_fullsized":"2024-09-14T05:31:21.130931+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:12:45.477846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011628109999999999,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":3532,"num_read_kb":9462,"num_write":620,"num_write_kb":5631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":242,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128830+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:31:21.128830+0000","last_peered":"2024-09-14T05:31:21.128830+0000","last_clean":"2024-09-14T05:31:21.128830+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:31:21.128830+0000","last_undegraded":"2024-09-14T05:31:21.128830+0000","last_fullsized":"2024-09-14T05:31:21.128830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":327,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.272917+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:31:20.272917+0000","last_peered":"2024-09-14T05:31:20.272917+0000","last_clean":"2024-09-14T05:31:20.272917+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:31:20.272917+0000","last_undegraded":"2024-09-14T05:31:20.272917+0000","last_fullsized":"2024-09-14T05:31:20.272917+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"113'395","reported_seq":2290,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130890+0000","last_change":"2024-09-14T05:31:03.019738+0000","last_active":"2024-09-14T05:31:21.130890+0000","last_peered":"2024-09-14T05:31:21.130890+0000","last_clean":"2024-09-14T05:31:21.130890+0000","last_became_active":"2024-09-14T05:29:24.012222+0000","last_became_peered":"2024-09-14T05:29:24.012222+0000","last_unstale":"2024-09-14T05:31:21.130890+0000","last_undegraded":"2024-09-14T05:31:21.130890+0000","last_fullsized":"2024-09-14T05:31:21.130890+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:52:56.721362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072525400000000005,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2833,"num_read_kb":6029,"num_write":327,"num_write_kb":3878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"113'608","reported_seq":2440,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128564+0000","last_change":"2024-09-14T05:31:03.020704+0000","last_active":"2024-09-14T05:31:21.128564+0000","last_peered":"2024-09-14T05:31:21.128564+0000","last_clean":"2024-09-14T05:31:21.128564+0000","last_became_active":"2024-09-14T05:29:24.012041+0000","last_became_peered":"2024-09-14T05:29:24.012041+0000","last_unstale":"2024-09-14T05:31:21.128564+0000","last_undegraded":"2024-09-14T05:31:21.128564+0000","last_fullsized":"2024-09-14T05:31:21.128564+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:42:01.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001433679,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2914,"num_read_kb":7354,"num_write":748,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"113'655","reported_seq":2691,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128609+0000","last_change":"2024-09-14T05:31:03.021329+0000","last_active":"2024-09-14T05:31:21.128609+0000","last_peered":"2024-09-14T05:31:21.128609+0000","last_clean":"2024-09-14T05:31:21.128609+0000","last_became_active":"2024-09-14T05:29:24.011403+0000","last_became_peered":"2024-09-14T05:29:24.011403+0000","last_unstale":"2024-09-14T05:31:21.128609+0000","last_undegraded":"2024-09-14T05:31:21.128609+0000","last_fullsized":"2024-09-14T05:31:21.128609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:04:32.706037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00168743,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":3956,"num_read_kb":8037,"num_write":714,"num_write_kb":4980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"112'632","reported_seq":10267,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.270508+0000","last_change":"2024-09-14T05:31:03.032462+0000","last_active":"2024-09-14T05:31:20.270508+0000","last_peered":"2024-09-14T05:31:20.270508+0000","last_clean":"2024-09-14T05:31:20.270508+0000","last_became_active":"2024-09-14T05:29:24.006836+0000","last_became_peered":"2024-09-14T05:29:24.006836+0000","last_unstale":"2024-09-14T05:31:20.270508+0000","last_undegraded":"2024-09-14T05:31:20.270508+0000","last_fullsized":"2024-09-14T05:31:20.270508+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:22:08.301740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060523399999999996,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":12123,"num_read_kb":29901,"num_write":689,"num_write_kb":13072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"113'448","reported_seq":3554,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189014+0000","last_change":"2024-09-14T05:31:03.068410+0000","last_active":"2024-09-14T05:31:22.189014+0000","last_peered":"2024-09-14T05:31:22.189014+0000","last_clean":"2024-09-14T05:31:22.189014+0000","last_became_active":"2024-09-14T05:29:24.012574+0000","last_became_peered":"2024-09-14T05:29:24.012574+0000","last_unstale":"2024-09-14T05:31:22.189014+0000","last_undegraded":"2024-09-14T05:31:22.189014+0000","last_fullsized":"2024-09-14T05:31:22.189014+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:58:16.204790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064954900000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2817,"num_read_kb":13317,"num_write":361,"num_write_kb":9799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"112'1131","reported_seq":11394,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128943+0000","last_change":"2024-09-14T05:31:03.021253+0000","last_active":"2024-09-14T05:31:21.128943+0000","last_peered":"2024-09-14T05:31:21.128943+0000","last_clean":"2024-09-14T05:31:21.128943+0000","last_became_active":"2024-09-14T05:29:24.012401+0000","last_became_peered":"2024-09-14T05:29:24.012401+0000","last_unstale":"2024-09-14T05:31:21.128943+0000","last_undegraded":"2024-09-14T05:31:21.128943+0000","last_fullsized":"2024-09-14T05:31:21.128943+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1131,"log_dups_size":0,"ondisk_log_size":1131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:18:41.728775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018223110000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":14909,"num_read_kb":22344,"num_write":1338,"num_write_kb":6495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"113'367","reported_seq":2862,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521474+0000","last_change":"2024-09-14T05:31:03.024269+0000","last_active":"2024-09-14T05:31:21.521474+0000","last_peered":"2024-09-14T05:31:21.521474+0000","last_clean":"2024-09-14T05:31:21.521474+0000","last_became_active":"2024-09-14T05:29:24.015315+0000","last_became_peered":"2024-09-14T05:29:24.015315+0000","last_unstale":"2024-09-14T05:31:21.521474+0000","last_undegraded":"2024-09-14T05:31:21.521474+0000","last_fullsized":"2024-09-14T05:31:21.521474+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:27:50.755671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001918957,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2150,"num_read_kb":8871,"num_write":290,"num_write_kb":3939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"111'532","reported_seq":11844,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189148+0000","last_change":"2024-09-14T05:31:03.068658+0000","last_active":"2024-09-14T05:31:22.189148+0000","last_peered":"2024-09-14T05:31:22.189148+0000","last_clean":"2024-09-14T05:31:22.189148+0000","last_became_active":"2024-09-14T05:29:24.018854+0000","last_became_peered":"2024-09-14T05:29:24.018854+0000","last_unstale":"2024-09-14T05:31:22.189148+0000","last_undegraded":"2024-09-14T05:31:22.189148+0000","last_fullsized":"2024-09-14T05:31:22.189148+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:12:27.055181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000829428,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":12059,"num_read_kb":31916,"num_write":436,"num_write_kb":11843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"113'389","reported_seq":3039,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.272918+0000","last_change":"2024-09-14T05:31:03.034867+0000","last_active":"2024-09-14T05:31:20.272918+0000","last_peered":"2024-09-14T05:31:20.272918+0000","last_clean":"2024-09-14T05:31:20.272918+0000","last_became_active":"2024-09-14T05:29:24.010897+0000","last_became_peered":"2024-09-14T05:29:24.010897+0000","last_unstale":"2024-09-14T05:31:20.272918+0000","last_undegraded":"2024-09-14T05:31:20.272918+0000","last_fullsized":"2024-09-14T05:31:20.272918+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:33:58.717977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051683800000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2374,"num_read_kb":22456,"num_write":338,"num_write_kb":8939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"112'576","reported_seq":4039,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128547+0000","last_change":"2024-09-14T05:31:03.020915+0000","last_active":"2024-09-14T05:31:21.128547+0000","last_peered":"2024-09-14T05:31:21.128547+0000","last_clean":"2024-09-14T05:31:21.128547+0000","last_became_active":"2024-09-14T05:29:24.013792+0000","last_became_peered":"2024-09-14T05:29:24.013792+0000","last_unstale":"2024-09-14T05:31:21.128547+0000","last_undegraded":"2024-09-14T05:31:21.128547+0000","last_fullsized":"2024-09-14T05:31:21.128547+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:09:14.825887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001768403,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5197,"num_read_kb":24484,"num_write":526,"num_write_kb":3336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"112'490","reported_seq":2632,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.270506+0000","last_change":"2024-09-14T05:31:03.032142+0000","last_active":"2024-09-14T05:31:20.270506+0000","last_peered":"2024-09-14T05:31:20.270506+0000","last_clean":"2024-09-14T05:31:20.270506+0000","last_became_active":"2024-09-14T05:30:38.013627+0000","last_became_peered":"2024-09-14T05:30:38.013627+0000","last_unstale":"2024-09-14T05:31:20.270506+0000","last_undegraded":"2024-09-14T05:31:20.270506+0000","last_fullsized":"2024-09-14T05:31:20.270506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:55:09.154072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050464100000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3016,"num_read_kb":16678,"num_write":502,"num_write_kb":10866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2769134,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"113'680","reported_seq":2493,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128721+0000","last_change":"2024-09-14T05:31:03.019963+0000","last_active":"2024-09-14T05:31:21.128721+0000","last_peered":"2024-09-14T05:31:21.128721+0000","last_clean":"2024-09-14T05:31:21.128721+0000","last_became_active":"2024-09-14T05:29:24.014371+0000","last_became_peered":"2024-09-14T05:29:24.014371+0000","last_unstale":"2024-09-14T05:31:21.128721+0000","last_undegraded":"2024-09-14T05:31:21.128721+0000","last_fullsized":"2024-09-14T05:31:21.128721+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:06.352481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080910499999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3420,"num_read_kb":19916,"num_write":912,"num_write_kb":5503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":594,"num_object_clones":0,"num_object_copies":1188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":594,"num_whiteouts":590,"num_read":182740,"num_read_kb":565764,"num_write":20809,"num_write_kb":280821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19248,"ondisk_log_size":19248,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476768,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50776,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321064,"statfs":{"total":96636764160,"available":96584769536,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509471,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41036,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330804,"statfs":{"total":96636764160,"available":96594743296,"internally_reserved":0,"allocated":1560576,"data_stored":742034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28576,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343264,"statfs":{"total":96636764160,"available":96607502336,"internally_reserved":0,"allocated":1576960,"data_stored":747924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2158592,"data_stored":1332421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28568,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343272,"statfs":{"total":96636764160,"available":96607510528,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72988,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298852,"statfs":{"total":96636764160,"available":96562024448,"internally_reserved":0,"allocated":1576960,"data_stored":747924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509473,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51348,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320492,"statfs":{"total":96636764160,"available":96584183808,"internally_reserved":0,"allocated":2158592,"data_stored":1332421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:31:24.958 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T05:31:24.958 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-14T05:31:25.264 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-14T05:31:25.274 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T05:31:25.274 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:31:25.557 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:31:25.558 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:31:25.569 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-09-14T05:31:24.458624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":598,"num_object_clones":0,"num_object_copies":1196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":598,"num_whiteouts":590,"num_read":182806,"num_read_kb":565819,"num_write":20912,"num_write_kb":282451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19323,"ondisk_log_size":19323,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":339320,"kb_used_data":13428,"kb_used_omap":12,"kb_used_meta":325683,"kb_avail":754635400,"statfs":{"total":773094113280,"available":772746649600,"internally_reserved":0,"allocated":13750272,"data_stored":7146439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":333499952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000951"},"pg_stats":[{"pgid":"3.1f","version":"114'767","reported_seq":10977,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121322+0000","last_change":"2024-09-14T05:31:03.027720+0000","last_active":"2024-09-14T05:31:20.121322+0000","last_peered":"2024-09-14T05:31:20.121322+0000","last_clean":"2024-09-14T05:31:20.121322+0000","last_became_active":"2024-09-14T05:29:24.018892+0000","last_became_peered":"2024-09-14T05:29:24.018892+0000","last_unstale":"2024-09-14T05:31:20.121322+0000","last_undegraded":"2024-09-14T05:31:20.121322+0000","last_fullsized":"2024-09-14T05:31:20.121322+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:00:33.452652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013315950000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":12275,"num_read_kb":37400,"num_write":832,"num_write_kb":31155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"114'418","reported_seq":3310,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121298+0000","last_change":"2024-09-14T05:31:03.027683+0000","last_active":"2024-09-14T05:31:20.121298+0000","last_peered":"2024-09-14T05:31:20.121298+0000","last_clean":"2024-09-14T05:31:20.121298+0000","last_became_active":"2024-09-14T05:29:24.014468+0000","last_became_peered":"2024-09-14T05:29:24.014468+0000","last_unstale":"2024-09-14T05:31:20.121298+0000","last_undegraded":"2024-09-14T05:31:20.121298+0000","last_fullsized":"2024-09-14T05:31:20.121298+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:58.301444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001315826,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2805,"num_read_kb":32498,"num_write":409,"num_write_kb":21098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"113'1047","reported_seq":9884,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128908+0000","last_change":"2024-09-14T05:31:03.020460+0000","last_active":"2024-09-14T05:31:21.128908+0000","last_peered":"2024-09-14T05:31:21.128908+0000","last_clean":"2024-09-14T05:31:21.128908+0000","last_became_active":"2024-09-14T05:29:24.014251+0000","last_became_peered":"2024-09-14T05:29:24.014251+0000","last_unstale":"2024-09-14T05:31:21.128908+0000","last_undegraded":"2024-09-14T05:31:21.128908+0000","last_fullsized":"2024-09-14T05:31:21.128908+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1047,"log_dups_size":0,"ondisk_log_size":1047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:50:10.775143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012210070000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12253,"num_read_kb":23814,"num_write":1553,"num_write_kb":7077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"114'602","reported_seq":10588,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.273068+0000","last_change":"2024-09-14T05:31:03.035040+0000","last_active":"2024-09-14T05:31:20.273068+0000","last_peered":"2024-09-14T05:31:20.273068+0000","last_clean":"2024-09-14T05:31:20.273068+0000","last_became_active":"2024-09-14T05:29:24.012427+0000","last_became_peered":"2024-09-14T05:29:24.012427+0000","last_unstale":"2024-09-14T05:31:20.273068+0000","last_undegraded":"2024-09-14T05:31:20.273068+0000","last_fullsized":"2024-09-14T05:31:20.273068+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:11:08.020997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073664900000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":10982,"num_read_kb":35239,"num_write":569,"num_write_kb":4638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"112'587","reported_seq":3666,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130947+0000","last_change":"2024-09-14T05:31:03.020201+0000","last_active":"2024-09-14T05:31:21.130947+0000","last_peered":"2024-09-14T05:31:21.130947+0000","last_clean":"2024-09-14T05:31:21.130947+0000","last_became_active":"2024-09-14T05:29:24.012079+0000","last_became_peered":"2024-09-14T05:29:24.012079+0000","last_unstale":"2024-09-14T05:31:21.130947+0000","last_undegraded":"2024-09-14T05:31:21.130947+0000","last_fullsized":"2024-09-14T05:31:21.130947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:30:17.989902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010699310000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4414,"num_read_kb":26220,"num_write":576,"num_write_kb":21867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"110'1005","reported_seq":6499,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.273103+0000","last_change":"2024-09-14T05:31:03.035277+0000","last_active":"2024-09-14T05:31:20.273103+0000","last_peered":"2024-09-14T05:31:20.273103+0000","last_clean":"2024-09-14T05:31:20.273103+0000","last_became_active":"2024-09-14T05:29:24.012461+0000","last_became_peered":"2024-09-14T05:29:24.012461+0000","last_unstale":"2024-09-14T05:31:20.273103+0000","last_undegraded":"2024-09-14T05:31:20.273103+0000","last_fullsized":"2024-09-14T05:31:20.273103+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:23:06.828927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090417700000000004,"stat_sum":{"num_bytes":19,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":10524,"num_read_kb":34488,"num_write":1274,"num_write_kb":8459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"112'581","reported_seq":2237,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130927+0000","last_change":"2024-09-14T05:31:03.019768+0000","last_active":"2024-09-14T05:31:21.130927+0000","last_peered":"2024-09-14T05:31:21.130927+0000","last_clean":"2024-09-14T05:31:21.130927+0000","last_became_active":"2024-09-14T05:29:24.018753+0000","last_became_peered":"2024-09-14T05:29:24.018753+0000","last_unstale":"2024-09-14T05:31:21.130927+0000","last_undegraded":"2024-09-14T05:31:21.130927+0000","last_fullsized":"2024-09-14T05:31:21.130927+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:38:28.243765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074058200000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2590,"num_read_kb":13584,"num_write":688,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"113'699","reported_seq":2937,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121320+0000","last_change":"2024-09-14T05:31:03.027199+0000","last_active":"2024-09-14T05:31:20.121320+0000","last_peered":"2024-09-14T05:31:20.121320+0000","last_clean":"2024-09-14T05:31:20.121320+0000","last_became_active":"2024-09-14T05:29:24.011176+0000","last_became_peered":"2024-09-14T05:29:24.011176+0000","last_unstale":"2024-09-14T05:31:20.121320+0000","last_undegraded":"2024-09-14T05:31:20.121320+0000","last_fullsized":"2024-09-14T05:31:20.121320+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:27:54.861994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091515699999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4708,"num_read_kb":7495,"num_write":700,"num_write_kb":3426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"112'357","reported_seq":3176,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121325+0000","last_change":"2024-09-14T05:31:03.028059+0000","last_active":"2024-09-14T05:31:20.121325+0000","last_peered":"2024-09-14T05:31:20.121325+0000","last_clean":"2024-09-14T05:31:20.121325+0000","last_became_active":"2024-09-14T05:29:24.014005+0000","last_became_peered":"2024-09-14T05:29:24.014005+0000","last_unstale":"2024-09-14T05:31:20.121325+0000","last_undegraded":"2024-09-14T05:31:20.121325+0000","last_fullsized":"2024-09-14T05:31:20.121325+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:18:52.725886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001656189,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3149,"num_read_kb":11342,"num_write":316,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"113'340","reported_seq":9918,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521406+0000","last_change":"2024-09-14T05:31:03.023758+0000","last_active":"2024-09-14T05:31:21.521406+0000","last_peered":"2024-09-14T05:31:21.521406+0000","last_clean":"2024-09-14T05:31:21.521406+0000","last_became_active":"2024-09-14T05:29:24.013434+0000","last_became_peered":"2024-09-14T05:29:24.013434+0000","last_unstale":"2024-09-14T05:31:21.521406+0000","last_undegraded":"2024-09-14T05:31:21.521406+0000","last_fullsized":"2024-09-14T05:31:21.521406+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:44.069349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418475,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":9226,"num_read_kb":22920,"num_write":296,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"113'440","reported_seq":2749,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.188832+0000","last_change":"2024-09-14T05:31:03.068934+0000","last_active":"2024-09-14T05:31:22.188832+0000","last_peered":"2024-09-14T05:31:22.188832+0000","last_clean":"2024-09-14T05:31:22.188832+0000","last_became_active":"2024-09-14T05:29:24.014457+0000","last_became_peered":"2024-09-14T05:29:24.014457+0000","last_unstale":"2024-09-14T05:31:22.188832+0000","last_undegraded":"2024-09-14T05:31:22.188832+0000","last_fullsized":"2024-09-14T05:31:22.188832+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:01:09.992476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011032850000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2612,"num_read_kb":9349,"num_write":416,"num_write_kb":12875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"109'843","reported_seq":2591,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128823+0000","last_change":"2024-09-14T05:31:03.020119+0000","last_active":"2024-09-14T05:31:21.128823+0000","last_peered":"2024-09-14T05:31:21.128823+0000","last_clean":"2024-09-14T05:31:21.128823+0000","last_became_active":"2024-09-14T05:29:24.013266+0000","last_became_peered":"2024-09-14T05:29:24.013266+0000","last_unstale":"2024-09-14T05:31:21.128823+0000","last_undegraded":"2024-09-14T05:31:21.128823+0000","last_fullsized":"2024-09-14T05:31:21.128823+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:34:03.137226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087100800000000005,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":4368,"num_read_kb":5632,"num_write":1075,"num_write_kb":4026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":240,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128815+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:31:21.128815+0000","last_peered":"2024-09-14T05:31:21.128815+0000","last_clean":"2024-09-14T05:31:21.128815+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:31:21.128815+0000","last_undegraded":"2024-09-14T05:31:21.128815+0000","last_fullsized":"2024-09-14T05:31:21.128815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"114'375","reported_seq":2319,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.131016+0000","last_change":"2024-09-14T05:31:03.020581+0000","last_active":"2024-09-14T05:31:21.131016+0000","last_peered":"2024-09-14T05:31:21.131016+0000","last_clean":"2024-09-14T05:31:21.131016+0000","last_became_active":"2024-09-14T05:29:24.012099+0000","last_became_peered":"2024-09-14T05:29:24.012099+0000","last_unstale":"2024-09-14T05:31:21.131016+0000","last_undegraded":"2024-09-14T05:31:21.131016+0000","last_fullsized":"2024-09-14T05:31:21.131016+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:44:38.747225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013882689999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2144,"num_read_kb":3725,"num_write":284,"num_write_kb":5458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":235,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.131025+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:31:21.131025+0000","last_peered":"2024-09-14T05:31:21.131025+0000","last_clean":"2024-09-14T05:31:21.131025+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:31:21.131025+0000","last_undegraded":"2024-09-14T05:31:21.131025+0000","last_fullsized":"2024-09-14T05:31:21.131025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"114'951","reported_seq":4046,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521397+0000","last_change":"2024-09-14T05:31:03.023762+0000","last_active":"2024-09-14T05:31:21.521397+0000","last_peered":"2024-09-14T05:31:21.521397+0000","last_clean":"2024-09-14T05:31:21.521397+0000","last_became_active":"2024-09-14T05:30:31.398064+0000","last_became_peered":"2024-09-14T05:30:31.398064+0000","last_unstale":"2024-09-14T05:31:21.521397+0000","last_undegraded":"2024-09-14T05:31:21.521397+0000","last_fullsized":"2024-09-14T05:31:21.521397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:15:44.039992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014417360000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6829,"num_read_kb":8465,"num_write":1096,"num_write_kb":5167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":648894,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128980+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:31:21.128980+0000","last_peered":"2024-09-14T05:31:21.128980+0000","last_clean":"2024-09-14T05:31:21.128980+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:31:21.128980+0000","last_undegraded":"2024-09-14T05:31:21.128980+0000","last_fullsized":"2024-09-14T05:31:21.128980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"112'569","reported_seq":2718,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130208+0000","last_change":"2024-09-14T05:31:20.123367+0000","last_active":"2024-09-14T05:31:21.130208+0000","last_peered":"2024-09-14T05:31:21.130208+0000","last_clean":"2024-09-14T05:31:21.130208+0000","last_became_active":"2024-09-14T05:31:20.122139+0000","last_became_peered":"2024-09-14T05:31:20.122139+0000","last_unstale":"2024-09-14T05:31:21.130208+0000","last_undegraded":"2024-09-14T05:31:21.130208+0000","last_fullsized":"2024-09-14T05:31:21.130208+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:03:12.206743+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0050460000000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3487,"num_read_kb":12454,"num_write":614,"num_write_kb":11389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":234,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121553+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:31:20.121553+0000","last_peered":"2024-09-14T05:31:20.121553+0000","last_clean":"2024-09-14T05:31:20.121553+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:31:20.121553+0000","last_undegraded":"2024-09-14T05:31:20.121553+0000","last_fullsized":"2024-09-14T05:31:20.121553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"114'351","reported_seq":1951,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521367+0000","last_change":"2024-09-14T05:31:03.024292+0000","last_active":"2024-09-14T05:31:21.521367+0000","last_peered":"2024-09-14T05:31:21.521367+0000","last_clean":"2024-09-14T05:31:21.521367+0000","last_became_active":"2024-09-14T05:29:24.016940+0000","last_became_peered":"2024-09-14T05:29:24.016940+0000","last_unstale":"2024-09-14T05:31:21.521367+0000","last_undegraded":"2024-09-14T05:31:21.521367+0000","last_fullsized":"2024-09-14T05:31:21.521367+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:42:00.348036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001928208,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1597,"num_read_kb":4408,"num_write":335,"num_write_kb":4001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"114'642","reported_seq":3022,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189410+0000","last_change":"2024-09-14T05:31:03.069465+0000","last_active":"2024-09-14T05:31:22.189410+0000","last_peered":"2024-09-14T05:31:22.189410+0000","last_clean":"2024-09-14T05:31:22.189410+0000","last_became_active":"2024-09-14T05:29:24.015905+0000","last_became_peered":"2024-09-14T05:29:24.015905+0000","last_unstale":"2024-09-14T05:31:22.189410+0000","last_undegraded":"2024-09-14T05:31:22.189410+0000","last_fullsized":"2024-09-14T05:31:22.189410+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:31:28.697477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00154122,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":2532,"num_read_kb":9709,"num_write":637,"num_write_kb":7772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":243,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128727+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:31:21.128727+0000","last_peered":"2024-09-14T05:31:21.128727+0000","last_clean":"2024-09-14T05:31:21.128727+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:31:21.128727+0000","last_undegraded":"2024-09-14T05:31:21.128727+0000","last_fullsized":"2024-09-14T05:31:21.128727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"113'485","reported_seq":10963,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121177+0000","last_change":"2024-09-14T05:31:03.026777+0000","last_active":"2024-09-14T05:31:20.121177+0000","last_peered":"2024-09-14T05:31:20.121177+0000","last_clean":"2024-09-14T05:31:20.121177+0000","last_became_active":"2024-09-14T05:29:24.011764+0000","last_became_peered":"2024-09-14T05:29:24.011764+0000","last_unstale":"2024-09-14T05:31:20.121177+0000","last_undegraded":"2024-09-14T05:31:20.121177+0000","last_fullsized":"2024-09-14T05:31:20.121177+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:07.795205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057339999999999995,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":11805,"num_read_kb":38991,"num_write":413,"num_write_kb":4959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":237,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.270563+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:31:20.270563+0000","last_peered":"2024-09-14T05:31:20.270563+0000","last_clean":"2024-09-14T05:31:20.270563+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:31:20.270563+0000","last_undegraded":"2024-09-14T05:31:20.270563+0000","last_fullsized":"2024-09-14T05:31:20.270563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"110'701","reported_seq":1977,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130208+0000","last_change":"2024-09-14T05:31:20.123603+0000","last_active":"2024-09-14T05:31:21.130208+0000","last_peered":"2024-09-14T05:31:21.130208+0000","last_clean":"2024-09-14T05:31:21.130208+0000","last_became_active":"2024-09-14T05:31:20.122523+0000","last_became_peered":"2024-09-14T05:31:20.122523+0000","last_unstale":"2024-09-14T05:31:21.130208+0000","last_undegraded":"2024-09-14T05:31:21.130208+0000","last_fullsized":"2024-09-14T05:31:21.130208+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:44:47.461984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022421440000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3140,"num_read_kb":7266,"num_write":925,"num_write_kb":10032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":241,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128547+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:31:21.128547+0000","last_peered":"2024-09-14T05:31:21.128547+0000","last_clean":"2024-09-14T05:31:21.128547+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:31:21.128547+0000","last_undegraded":"2024-09-14T05:31:21.128547+0000","last_fullsized":"2024-09-14T05:31:21.128547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"113'585","reported_seq":2767,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130931+0000","last_change":"2024-09-14T05:31:03.020204+0000","last_active":"2024-09-14T05:31:21.130931+0000","last_peered":"2024-09-14T05:31:21.130931+0000","last_clean":"2024-09-14T05:31:21.130931+0000","last_became_active":"2024-09-14T05:29:24.015792+0000","last_became_peered":"2024-09-14T05:29:24.015792+0000","last_unstale":"2024-09-14T05:31:21.130931+0000","last_undegraded":"2024-09-14T05:31:21.130931+0000","last_fullsized":"2024-09-14T05:31:21.130931+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:12:45.477846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011628109999999999,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":3532,"num_read_kb":9462,"num_write":620,"num_write_kb":5631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":242,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128830+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:31:21.128830+0000","last_peered":"2024-09-14T05:31:21.128830+0000","last_clean":"2024-09-14T05:31:21.128830+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:31:21.128830+0000","last_undegraded":"2024-09-14T05:31:21.128830+0000","last_fullsized":"2024-09-14T05:31:21.128830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":327,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.272917+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:31:20.272917+0000","last_peered":"2024-09-14T05:31:20.272917+0000","last_clean":"2024-09-14T05:31:20.272917+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:31:20.272917+0000","last_undegraded":"2024-09-14T05:31:20.272917+0000","last_fullsized":"2024-09-14T05:31:20.272917+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"113'395","reported_seq":2290,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130890+0000","last_change":"2024-09-14T05:31:03.019738+0000","last_active":"2024-09-14T05:31:21.130890+0000","last_peered":"2024-09-14T05:31:21.130890+0000","last_clean":"2024-09-14T05:31:21.130890+0000","last_became_active":"2024-09-14T05:29:24.012222+0000","last_became_peered":"2024-09-14T05:29:24.012222+0000","last_unstale":"2024-09-14T05:31:21.130890+0000","last_undegraded":"2024-09-14T05:31:21.130890+0000","last_fullsized":"2024-09-14T05:31:21.130890+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:52:56.721362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072525400000000005,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2833,"num_read_kb":6029,"num_write":327,"num_write_kb":3878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"113'608","reported_seq":2440,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128564+0000","last_change":"2024-09-14T05:31:03.020704+0000","last_active":"2024-09-14T05:31:21.128564+0000","last_peered":"2024-09-14T05:31:21.128564+0000","last_clean":"2024-09-14T05:31:21.128564+0000","last_became_active":"2024-09-14T05:29:24.012041+0000","last_became_peered":"2024-09-14T05:29:24.012041+0000","last_unstale":"2024-09-14T05:31:21.128564+0000","last_undegraded":"2024-09-14T05:31:21.128564+0000","last_fullsized":"2024-09-14T05:31:21.128564+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:42:01.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001433679,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2914,"num_read_kb":7354,"num_write":748,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"113'655","reported_seq":2691,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128609+0000","last_change":"2024-09-14T05:31:03.021329+0000","last_active":"2024-09-14T05:31:21.128609+0000","last_peered":"2024-09-14T05:31:21.128609+0000","last_clean":"2024-09-14T05:31:21.128609+0000","last_became_active":"2024-09-14T05:29:24.011403+0000","last_became_peered":"2024-09-14T05:29:24.011403+0000","last_unstale":"2024-09-14T05:31:21.128609+0000","last_undegraded":"2024-09-14T05:31:21.128609+0000","last_fullsized":"2024-09-14T05:31:21.128609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:04:32.706037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00168743,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":3956,"num_read_kb":8037,"num_write":714,"num_write_kb":4980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"112'632","reported_seq":10267,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.270508+0000","last_change":"2024-09-14T05:31:03.032462+0000","last_active":"2024-09-14T05:31:20.270508+0000","last_peered":"2024-09-14T05:31:20.270508+0000","last_clean":"2024-09-14T05:31:20.270508+0000","last_became_active":"2024-09-14T05:29:24.006836+0000","last_became_peered":"2024-09-14T05:29:24.006836+0000","last_unstale":"2024-09-14T05:31:20.270508+0000","last_undegraded":"2024-09-14T05:31:20.270508+0000","last_fullsized":"2024-09-14T05:31:20.270508+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:22:08.301740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060523399999999996,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":12123,"num_read_kb":29901,"num_write":689,"num_write_kb":13072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"113'448","reported_seq":3554,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189014+0000","last_change":"2024-09-14T05:31:03.068410+0000","last_active":"2024-09-14T05:31:22.189014+0000","last_peered":"2024-09-14T05:31:22.189014+0000","last_clean":"2024-09-14T05:31:22.189014+0000","last_became_active":"2024-09-14T05:29:24.012574+0000","last_became_peered":"2024-09-14T05:29:24.012574+0000","last_unstale":"2024-09-14T05:31:22.189014+0000","last_undegraded":"2024-09-14T05:31:22.189014+0000","last_fullsized":"2024-09-14T05:31:22.189014+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:58:16.204790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064954900000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2817,"num_read_kb":13317,"num_write":361,"num_write_kb":9799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"112'1131","reported_seq":11394,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128943+0000","last_change":"2024-09-14T05:31:03.021253+0000","last_active":"2024-09-14T05:31:21.128943+0000","last_peered":"2024-09-14T05:31:21.128943+0000","last_clean":"2024-09-14T05:31:21.128943+0000","last_became_active":"2024-09-14T05:29:24.012401+0000","last_became_peered":"2024-09-14T05:29:24.012401+0000","last_unstale":"2024-09-14T05:31:21.128943+0000","last_undegraded":"2024-09-14T05:31:21.128943+0000","last_fullsized":"2024-09-14T05:31:21.128943+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1131,"log_dups_size":0,"ondisk_log_size":1131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:18:41.728775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018223110000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":14909,"num_read_kb":22344,"num_write":1338,"num_write_kb":6495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"113'367","reported_seq":2862,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521474+0000","last_change":"2024-09-14T05:31:03.024269+0000","last_active":"2024-09-14T05:31:21.521474+0000","last_peered":"2024-09-14T05:31:21.521474+0000","last_clean":"2024-09-14T05:31:21.521474+0000","last_became_active":"2024-09-14T05:29:24.015315+0000","last_became_peered":"2024-09-14T05:29:24.015315+0000","last_unstale":"2024-09-14T05:31:21.521474+0000","last_undegraded":"2024-09-14T05:31:21.521474+0000","last_fullsized":"2024-09-14T05:31:21.521474+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:27:50.755671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001918957,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2150,"num_read_kb":8871,"num_write":290,"num_write_kb":3939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"111'532","reported_seq":11844,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189148+0000","last_change":"2024-09-14T05:31:03.068658+0000","last_active":"2024-09-14T05:31:22.189148+0000","last_peered":"2024-09-14T05:31:22.189148+0000","last_clean":"2024-09-14T05:31:22.189148+0000","last_became_active":"2024-09-14T05:29:24.018854+0000","last_became_peered":"2024-09-14T05:29:24.018854+0000","last_unstale":"2024-09-14T05:31:22.189148+0000","last_undegraded":"2024-09-14T05:31:22.189148+0000","last_fullsized":"2024-09-14T05:31:22.189148+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:12:27.055181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000829428,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":12059,"num_read_kb":31916,"num_write":436,"num_write_kb":11843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"113'389","reported_seq":3039,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.272918+0000","last_change":"2024-09-14T05:31:03.034867+0000","last_active":"2024-09-14T05:31:20.272918+0000","last_peered":"2024-09-14T05:31:20.272918+0000","last_clean":"2024-09-14T05:31:20.272918+0000","last_became_active":"2024-09-14T05:29:24.010897+0000","last_became_peered":"2024-09-14T05:29:24.010897+0000","last_unstale":"2024-09-14T05:31:20.272918+0000","last_undegraded":"2024-09-14T05:31:20.272918+0000","last_fullsized":"2024-09-14T05:31:20.272918+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:33:58.717977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051683800000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2374,"num_read_kb":22456,"num_write":338,"num_write_kb":8939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"112'576","reported_seq":4039,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128547+0000","last_change":"2024-09-14T05:31:03.020915+0000","last_active":"2024-09-14T05:31:21.128547+0000","last_peered":"2024-09-14T05:31:21.128547+0000","last_clean":"2024-09-14T05:31:21.128547+0000","last_became_active":"2024-09-14T05:29:24.013792+0000","last_became_peered":"2024-09-14T05:29:24.013792+0000","last_unstale":"2024-09-14T05:31:21.128547+0000","last_undegraded":"2024-09-14T05:31:21.128547+0000","last_fullsized":"2024-09-14T05:31:21.128547+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:09:14.825887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001768403,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5197,"num_read_kb":24484,"num_write":526,"num_write_kb":3336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"112'490","reported_seq":2632,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.270506+0000","last_change":"2024-09-14T05:31:03.032142+0000","last_active":"2024-09-14T05:31:20.270506+0000","last_peered":"2024-09-14T05:31:20.270506+0000","last_clean":"2024-09-14T05:31:20.270506+0000","last_became_active":"2024-09-14T05:30:38.013627+0000","last_became_peered":"2024-09-14T05:30:38.013627+0000","last_unstale":"2024-09-14T05:31:20.270506+0000","last_undegraded":"2024-09-14T05:31:20.270506+0000","last_fullsized":"2024-09-14T05:31:20.270506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:55:09.154072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050464100000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3016,"num_read_kb":16678,"num_write":502,"num_write_kb":10866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2769134,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"113'680","reported_seq":2493,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128721+0000","last_change":"2024-09-14T05:31:03.019963+0000","last_active":"2024-09-14T05:31:21.128721+0000","last_peered":"2024-09-14T05:31:21.128721+0000","last_clean":"2024-09-14T05:31:21.128721+0000","last_became_active":"2024-09-14T05:29:24.014371+0000","last_became_peered":"2024-09-14T05:29:24.014371+0000","last_unstale":"2024-09-14T05:31:21.128721+0000","last_undegraded":"2024-09-14T05:31:21.128721+0000","last_fullsized":"2024-09-14T05:31:21.128721+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:06.352481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080910499999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3420,"num_read_kb":19916,"num_write":912,"num_write_kb":5503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":594,"num_object_clones":0,"num_object_copies":1188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":594,"num_whiteouts":590,"num_read":182740,"num_read_kb":565764,"num_write":20809,"num_write_kb":280821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19248,"ondisk_log_size":19248,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476768,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50776,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321064,"statfs":{"total":96636764160,"available":96584769536,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509471,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41036,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330804,"statfs":{"total":96636764160,"available":96594743296,"internally_reserved":0,"allocated":1560576,"data_stored":742034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28576,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343264,"statfs":{"total":96636764160,"available":96607502336,"internally_reserved":0,"allocated":1576960,"data_stored":747924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2158592,"data_stored":1332421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28568,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343272,"statfs":{"total":96636764160,"available":96607510528,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72988,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298852,"statfs":{"total":96636764160,"available":96562024448,"internally_reserved":0,"allocated":1576960,"data_stored":747924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509473,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51348,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320492,"statfs":{"total":96636764160,"available":96584183808,"internally_reserved":0,"allocated":2158592,"data_stored":1332421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:31:25.572 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:31:25.853 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:31:25.854 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:31:25.865 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-09-14T05:31:24.458624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":598,"num_object_clones":0,"num_object_copies":1196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":598,"num_whiteouts":590,"num_read":182806,"num_read_kb":565819,"num_write":20912,"num_write_kb":282451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19323,"ondisk_log_size":19323,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":339320,"kb_used_data":13428,"kb_used_omap":12,"kb_used_meta":325683,"kb_avail":754635400,"statfs":{"total":773094113280,"available":772746649600,"internally_reserved":0,"allocated":13750272,"data_stored":7146439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":333499952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000951"},"pg_stats":[{"pgid":"3.1f","version":"114'767","reported_seq":10977,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121322+0000","last_change":"2024-09-14T05:31:03.027720+0000","last_active":"2024-09-14T05:31:20.121322+0000","last_peered":"2024-09-14T05:31:20.121322+0000","last_clean":"2024-09-14T05:31:20.121322+0000","last_became_active":"2024-09-14T05:29:24.018892+0000","last_became_peered":"2024-09-14T05:29:24.018892+0000","last_unstale":"2024-09-14T05:31:20.121322+0000","last_undegraded":"2024-09-14T05:31:20.121322+0000","last_fullsized":"2024-09-14T05:31:20.121322+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:00:33.452652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013315950000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":12275,"num_read_kb":37400,"num_write":832,"num_write_kb":31155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"114'418","reported_seq":3310,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121298+0000","last_change":"2024-09-14T05:31:03.027683+0000","last_active":"2024-09-14T05:31:20.121298+0000","last_peered":"2024-09-14T05:31:20.121298+0000","last_clean":"2024-09-14T05:31:20.121298+0000","last_became_active":"2024-09-14T05:29:24.014468+0000","last_became_peered":"2024-09-14T05:29:24.014468+0000","last_unstale":"2024-09-14T05:31:20.121298+0000","last_undegraded":"2024-09-14T05:31:20.121298+0000","last_fullsized":"2024-09-14T05:31:20.121298+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:58.301444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001315826,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2805,"num_read_kb":32498,"num_write":409,"num_write_kb":21098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"113'1047","reported_seq":9884,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128908+0000","last_change":"2024-09-14T05:31:03.020460+0000","last_active":"2024-09-14T05:31:21.128908+0000","last_peered":"2024-09-14T05:31:21.128908+0000","last_clean":"2024-09-14T05:31:21.128908+0000","last_became_active":"2024-09-14T05:29:24.014251+0000","last_became_peered":"2024-09-14T05:29:24.014251+0000","last_unstale":"2024-09-14T05:31:21.128908+0000","last_undegraded":"2024-09-14T05:31:21.128908+0000","last_fullsized":"2024-09-14T05:31:21.128908+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1047,"log_dups_size":0,"ondisk_log_size":1047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:50:10.775143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012210070000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12253,"num_read_kb":23814,"num_write":1553,"num_write_kb":7077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"114'602","reported_seq":10588,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.273068+0000","last_change":"2024-09-14T05:31:03.035040+0000","last_active":"2024-09-14T05:31:20.273068+0000","last_peered":"2024-09-14T05:31:20.273068+0000","last_clean":"2024-09-14T05:31:20.273068+0000","last_became_active":"2024-09-14T05:29:24.012427+0000","last_became_peered":"2024-09-14T05:29:24.012427+0000","last_unstale":"2024-09-14T05:31:20.273068+0000","last_undegraded":"2024-09-14T05:31:20.273068+0000","last_fullsized":"2024-09-14T05:31:20.273068+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:11:08.020997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073664900000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":10982,"num_read_kb":35239,"num_write":569,"num_write_kb":4638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"112'587","reported_seq":3666,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130947+0000","last_change":"2024-09-14T05:31:03.020201+0000","last_active":"2024-09-14T05:31:21.130947+0000","last_peered":"2024-09-14T05:31:21.130947+0000","last_clean":"2024-09-14T05:31:21.130947+0000","last_became_active":"2024-09-14T05:29:24.012079+0000","last_became_peered":"2024-09-14T05:29:24.012079+0000","last_unstale":"2024-09-14T05:31:21.130947+0000","last_undegraded":"2024-09-14T05:31:21.130947+0000","last_fullsized":"2024-09-14T05:31:21.130947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:30:17.989902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010699310000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4414,"num_read_kb":26220,"num_write":576,"num_write_kb":21867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"110'1005","reported_seq":6499,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.273103+0000","last_change":"2024-09-14T05:31:03.035277+0000","last_active":"2024-09-14T05:31:20.273103+0000","last_peered":"2024-09-14T05:31:20.273103+0000","last_clean":"2024-09-14T05:31:20.273103+0000","last_became_active":"2024-09-14T05:29:24.012461+0000","last_became_peered":"2024-09-14T05:29:24.012461+0000","last_unstale":"2024-09-14T05:31:20.273103+0000","last_undegraded":"2024-09-14T05:31:20.273103+0000","last_fullsized":"2024-09-14T05:31:20.273103+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:23:06.828927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090417700000000004,"stat_sum":{"num_bytes":19,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":10524,"num_read_kb":34488,"num_write":1274,"num_write_kb":8459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"112'581","reported_seq":2237,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130927+0000","last_change":"2024-09-14T05:31:03.019768+0000","last_active":"2024-09-14T05:31:21.130927+0000","last_peered":"2024-09-14T05:31:21.130927+0000","last_clean":"2024-09-14T05:31:21.130927+0000","last_became_active":"2024-09-14T05:29:24.018753+0000","last_became_peered":"2024-09-14T05:29:24.018753+0000","last_unstale":"2024-09-14T05:31:21.130927+0000","last_undegraded":"2024-09-14T05:31:21.130927+0000","last_fullsized":"2024-09-14T05:31:21.130927+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:38:28.243765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074058200000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2590,"num_read_kb":13584,"num_write":688,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"113'699","reported_seq":2937,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121320+0000","last_change":"2024-09-14T05:31:03.027199+0000","last_active":"2024-09-14T05:31:20.121320+0000","last_peered":"2024-09-14T05:31:20.121320+0000","last_clean":"2024-09-14T05:31:20.121320+0000","last_became_active":"2024-09-14T05:29:24.011176+0000","last_became_peered":"2024-09-14T05:29:24.011176+0000","last_unstale":"2024-09-14T05:31:20.121320+0000","last_undegraded":"2024-09-14T05:31:20.121320+0000","last_fullsized":"2024-09-14T05:31:20.121320+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:27:54.861994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091515699999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4708,"num_read_kb":7495,"num_write":700,"num_write_kb":3426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"112'357","reported_seq":3176,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121325+0000","last_change":"2024-09-14T05:31:03.028059+0000","last_active":"2024-09-14T05:31:20.121325+0000","last_peered":"2024-09-14T05:31:20.121325+0000","last_clean":"2024-09-14T05:31:20.121325+0000","last_became_active":"2024-09-14T05:29:24.014005+0000","last_became_peered":"2024-09-14T05:29:24.014005+0000","last_unstale":"2024-09-14T05:31:20.121325+0000","last_undegraded":"2024-09-14T05:31:20.121325+0000","last_fullsized":"2024-09-14T05:31:20.121325+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:18:52.725886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001656189,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3149,"num_read_kb":11342,"num_write":316,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"113'340","reported_seq":9918,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521406+0000","last_change":"2024-09-14T05:31:03.023758+0000","last_active":"2024-09-14T05:31:21.521406+0000","last_peered":"2024-09-14T05:31:21.521406+0000","last_clean":"2024-09-14T05:31:21.521406+0000","last_became_active":"2024-09-14T05:29:24.013434+0000","last_became_peered":"2024-09-14T05:29:24.013434+0000","last_unstale":"2024-09-14T05:31:21.521406+0000","last_undegraded":"2024-09-14T05:31:21.521406+0000","last_fullsized":"2024-09-14T05:31:21.521406+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:44.069349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418475,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":9226,"num_read_kb":22920,"num_write":296,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"113'440","reported_seq":2749,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.188832+0000","last_change":"2024-09-14T05:31:03.068934+0000","last_active":"2024-09-14T05:31:22.188832+0000","last_peered":"2024-09-14T05:31:22.188832+0000","last_clean":"2024-09-14T05:31:22.188832+0000","last_became_active":"2024-09-14T05:29:24.014457+0000","last_became_peered":"2024-09-14T05:29:24.014457+0000","last_unstale":"2024-09-14T05:31:22.188832+0000","last_undegraded":"2024-09-14T05:31:22.188832+0000","last_fullsized":"2024-09-14T05:31:22.188832+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:01:09.992476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011032850000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2612,"num_read_kb":9349,"num_write":416,"num_write_kb":12875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"109'843","reported_seq":2591,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128823+0000","last_change":"2024-09-14T05:31:03.020119+0000","last_active":"2024-09-14T05:31:21.128823+0000","last_peered":"2024-09-14T05:31:21.128823+0000","last_clean":"2024-09-14T05:31:21.128823+0000","last_became_active":"2024-09-14T05:29:24.013266+0000","last_became_peered":"2024-09-14T05:29:24.013266+0000","last_unstale":"2024-09-14T05:31:21.128823+0000","last_undegraded":"2024-09-14T05:31:21.128823+0000","last_fullsized":"2024-09-14T05:31:21.128823+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:34:03.137226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087100800000000005,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":4368,"num_read_kb":5632,"num_write":1075,"num_write_kb":4026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":240,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128815+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:31:21.128815+0000","last_peered":"2024-09-14T05:31:21.128815+0000","last_clean":"2024-09-14T05:31:21.128815+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:31:21.128815+0000","last_undegraded":"2024-09-14T05:31:21.128815+0000","last_fullsized":"2024-09-14T05:31:21.128815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"114'375","reported_seq":2319,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.131016+0000","last_change":"2024-09-14T05:31:03.020581+0000","last_active":"2024-09-14T05:31:21.131016+0000","last_peered":"2024-09-14T05:31:21.131016+0000","last_clean":"2024-09-14T05:31:21.131016+0000","last_became_active":"2024-09-14T05:29:24.012099+0000","last_became_peered":"2024-09-14T05:29:24.012099+0000","last_unstale":"2024-09-14T05:31:21.131016+0000","last_undegraded":"2024-09-14T05:31:21.131016+0000","last_fullsized":"2024-09-14T05:31:21.131016+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:44:38.747225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013882689999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2144,"num_read_kb":3725,"num_write":284,"num_write_kb":5458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":235,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.131025+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:31:21.131025+0000","last_peered":"2024-09-14T05:31:21.131025+0000","last_clean":"2024-09-14T05:31:21.131025+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:31:21.131025+0000","last_undegraded":"2024-09-14T05:31:21.131025+0000","last_fullsized":"2024-09-14T05:31:21.131025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"114'951","reported_seq":4046,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521397+0000","last_change":"2024-09-14T05:31:03.023762+0000","last_active":"2024-09-14T05:31:21.521397+0000","last_peered":"2024-09-14T05:31:21.521397+0000","last_clean":"2024-09-14T05:31:21.521397+0000","last_became_active":"2024-09-14T05:30:31.398064+0000","last_became_peered":"2024-09-14T05:30:31.398064+0000","last_unstale":"2024-09-14T05:31:21.521397+0000","last_undegraded":"2024-09-14T05:31:21.521397+0000","last_fullsized":"2024-09-14T05:31:21.521397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:15:44.039992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014417360000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6829,"num_read_kb":8465,"num_write":1096,"num_write_kb":5167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":648894,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128980+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:31:21.128980+0000","last_peered":"2024-09-14T05:31:21.128980+0000","last_clean":"2024-09-14T05:31:21.128980+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:31:21.128980+0000","last_undegraded":"2024-09-14T05:31:21.128980+0000","last_fullsized":"2024-09-14T05:31:21.128980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"112'569","reported_seq":2718,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130208+0000","last_change":"2024-09-14T05:31:20.123367+0000","last_active":"2024-09-14T05:31:21.130208+0000","last_peered":"2024-09-14T05:31:21.130208+0000","last_clean":"2024-09-14T05:31:21.130208+0000","last_became_active":"2024-09-14T05:31:20.122139+0000","last_became_peered":"2024-09-14T05:31:20.122139+0000","last_unstale":"2024-09-14T05:31:21.130208+0000","last_undegraded":"2024-09-14T05:31:21.130208+0000","last_fullsized":"2024-09-14T05:31:21.130208+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:03:12.206743+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0050460000000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3487,"num_read_kb":12454,"num_write":614,"num_write_kb":11389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":234,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121553+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:31:20.121553+0000","last_peered":"2024-09-14T05:31:20.121553+0000","last_clean":"2024-09-14T05:31:20.121553+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:31:20.121553+0000","last_undegraded":"2024-09-14T05:31:20.121553+0000","last_fullsized":"2024-09-14T05:31:20.121553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"114'351","reported_seq":1951,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521367+0000","last_change":"2024-09-14T05:31:03.024292+0000","last_active":"2024-09-14T05:31:21.521367+0000","last_peered":"2024-09-14T05:31:21.521367+0000","last_clean":"2024-09-14T05:31:21.521367+0000","last_became_active":"2024-09-14T05:29:24.016940+0000","last_became_peered":"2024-09-14T05:29:24.016940+0000","last_unstale":"2024-09-14T05:31:21.521367+0000","last_undegraded":"2024-09-14T05:31:21.521367+0000","last_fullsized":"2024-09-14T05:31:21.521367+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:42:00.348036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001928208,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1597,"num_read_kb":4408,"num_write":335,"num_write_kb":4001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"114'642","reported_seq":3022,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189410+0000","last_change":"2024-09-14T05:31:03.069465+0000","last_active":"2024-09-14T05:31:22.189410+0000","last_peered":"2024-09-14T05:31:22.189410+0000","last_clean":"2024-09-14T05:31:22.189410+0000","last_became_active":"2024-09-14T05:29:24.015905+0000","last_became_peered":"2024-09-14T05:29:24.015905+0000","last_unstale":"2024-09-14T05:31:22.189410+0000","last_undegraded":"2024-09-14T05:31:22.189410+0000","last_fullsized":"2024-09-14T05:31:22.189410+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:31:28.697477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00154122,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":2532,"num_read_kb":9709,"num_write":637,"num_write_kb":7772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":243,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128727+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:31:21.128727+0000","last_peered":"2024-09-14T05:31:21.128727+0000","last_clean":"2024-09-14T05:31:21.128727+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:31:21.128727+0000","last_undegraded":"2024-09-14T05:31:21.128727+0000","last_fullsized":"2024-09-14T05:31:21.128727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"113'485","reported_seq":10963,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121177+0000","last_change":"2024-09-14T05:31:03.026777+0000","last_active":"2024-09-14T05:31:20.121177+0000","last_peered":"2024-09-14T05:31:20.121177+0000","last_clean":"2024-09-14T05:31:20.121177+0000","last_became_active":"2024-09-14T05:29:24.011764+0000","last_became_peered":"2024-09-14T05:29:24.011764+0000","last_unstale":"2024-09-14T05:31:20.121177+0000","last_undegraded":"2024-09-14T05:31:20.121177+0000","last_fullsized":"2024-09-14T05:31:20.121177+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:07.795205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057339999999999995,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":11805,"num_read_kb":38991,"num_write":413,"num_write_kb":4959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":237,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.270563+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:31:20.270563+0000","last_peered":"2024-09-14T05:31:20.270563+0000","last_clean":"2024-09-14T05:31:20.270563+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:31:20.270563+0000","last_undegraded":"2024-09-14T05:31:20.270563+0000","last_fullsized":"2024-09-14T05:31:20.270563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"110'701","reported_seq":1977,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130208+0000","last_change":"2024-09-14T05:31:20.123603+0000","last_active":"2024-09-14T05:31:21.130208+0000","last_peered":"2024-09-14T05:31:21.130208+0000","last_clean":"2024-09-14T05:31:21.130208+0000","last_became_active":"2024-09-14T05:31:20.122523+0000","last_became_peered":"2024-09-14T05:31:20.122523+0000","last_unstale":"2024-09-14T05:31:21.130208+0000","last_undegraded":"2024-09-14T05:31:21.130208+0000","last_fullsized":"2024-09-14T05:31:21.130208+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:44:47.461984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022421440000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3140,"num_read_kb":7266,"num_write":925,"num_write_kb":10032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":241,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128547+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:31:21.128547+0000","last_peered":"2024-09-14T05:31:21.128547+0000","last_clean":"2024-09-14T05:31:21.128547+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:31:21.128547+0000","last_undegraded":"2024-09-14T05:31:21.128547+0000","last_fullsized":"2024-09-14T05:31:21.128547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"113'585","reported_seq":2767,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130931+0000","last_change":"2024-09-14T05:31:03.020204+0000","last_active":"2024-09-14T05:31:21.130931+0000","last_peered":"2024-09-14T05:31:21.130931+0000","last_clean":"2024-09-14T05:31:21.130931+0000","last_became_active":"2024-09-14T05:29:24.015792+0000","last_became_peered":"2024-09-14T05:29:24.015792+0000","last_unstale":"2024-09-14T05:31:21.130931+0000","last_undegraded":"2024-09-14T05:31:21.130931+0000","last_fullsized":"2024-09-14T05:31:21.130931+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:12:45.477846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011628109999999999,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":3532,"num_read_kb":9462,"num_write":620,"num_write_kb":5631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":242,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128830+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:31:21.128830+0000","last_peered":"2024-09-14T05:31:21.128830+0000","last_clean":"2024-09-14T05:31:21.128830+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:31:21.128830+0000","last_undegraded":"2024-09-14T05:31:21.128830+0000","last_fullsized":"2024-09-14T05:31:21.128830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":327,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.272917+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:31:20.272917+0000","last_peered":"2024-09-14T05:31:20.272917+0000","last_clean":"2024-09-14T05:31:20.272917+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:31:20.272917+0000","last_undegraded":"2024-09-14T05:31:20.272917+0000","last_fullsized":"2024-09-14T05:31:20.272917+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"113'395","reported_seq":2290,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130890+0000","last_change":"2024-09-14T05:31:03.019738+0000","last_active":"2024-09-14T05:31:21.130890+0000","last_peered":"2024-09-14T05:31:21.130890+0000","last_clean":"2024-09-14T05:31:21.130890+0000","last_became_active":"2024-09-14T05:29:24.012222+0000","last_became_peered":"2024-09-14T05:29:24.012222+0000","last_unstale":"2024-09-14T05:31:21.130890+0000","last_undegraded":"2024-09-14T05:31:21.130890+0000","last_fullsized":"2024-09-14T05:31:21.130890+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:52:56.721362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072525400000000005,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2833,"num_read_kb":6029,"num_write":327,"num_write_kb":3878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"113'608","reported_seq":2440,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128564+0000","last_change":"2024-09-14T05:31:03.020704+0000","last_active":"2024-09-14T05:31:21.128564+0000","last_peered":"2024-09-14T05:31:21.128564+0000","last_clean":"2024-09-14T05:31:21.128564+0000","last_became_active":"2024-09-14T05:29:24.012041+0000","last_became_peered":"2024-09-14T05:29:24.012041+0000","last_unstale":"2024-09-14T05:31:21.128564+0000","last_undegraded":"2024-09-14T05:31:21.128564+0000","last_fullsized":"2024-09-14T05:31:21.128564+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:42:01.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001433679,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2914,"num_read_kb":7354,"num_write":748,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"113'655","reported_seq":2691,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128609+0000","last_change":"2024-09-14T05:31:03.021329+0000","last_active":"2024-09-14T05:31:21.128609+0000","last_peered":"2024-09-14T05:31:21.128609+0000","last_clean":"2024-09-14T05:31:21.128609+0000","last_became_active":"2024-09-14T05:29:24.011403+0000","last_became_peered":"2024-09-14T05:29:24.011403+0000","last_unstale":"2024-09-14T05:31:21.128609+0000","last_undegraded":"2024-09-14T05:31:21.128609+0000","last_fullsized":"2024-09-14T05:31:21.128609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:04:32.706037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00168743,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":3956,"num_read_kb":8037,"num_write":714,"num_write_kb":4980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"112'632","reported_seq":10267,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.270508+0000","last_change":"2024-09-14T05:31:03.032462+0000","last_active":"2024-09-14T05:31:20.270508+0000","last_peered":"2024-09-14T05:31:20.270508+0000","last_clean":"2024-09-14T05:31:20.270508+0000","last_became_active":"2024-09-14T05:29:24.006836+0000","last_became_peered":"2024-09-14T05:29:24.006836+0000","last_unstale":"2024-09-14T05:31:20.270508+0000","last_undegraded":"2024-09-14T05:31:20.270508+0000","last_fullsized":"2024-09-14T05:31:20.270508+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:22:08.301740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060523399999999996,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":12123,"num_read_kb":29901,"num_write":689,"num_write_kb":13072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"113'448","reported_seq":3554,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189014+0000","last_change":"2024-09-14T05:31:03.068410+0000","last_active":"2024-09-14T05:31:22.189014+0000","last_peered":"2024-09-14T05:31:22.189014+0000","last_clean":"2024-09-14T05:31:22.189014+0000","last_became_active":"2024-09-14T05:29:24.012574+0000","last_became_peered":"2024-09-14T05:29:24.012574+0000","last_unstale":"2024-09-14T05:31:22.189014+0000","last_undegraded":"2024-09-14T05:31:22.189014+0000","last_fullsized":"2024-09-14T05:31:22.189014+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:58:16.204790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064954900000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2817,"num_read_kb":13317,"num_write":361,"num_write_kb":9799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"112'1131","reported_seq":11394,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128943+0000","last_change":"2024-09-14T05:31:03.021253+0000","last_active":"2024-09-14T05:31:21.128943+0000","last_peered":"2024-09-14T05:31:21.128943+0000","last_clean":"2024-09-14T05:31:21.128943+0000","last_became_active":"2024-09-14T05:29:24.012401+0000","last_became_peered":"2024-09-14T05:29:24.012401+0000","last_unstale":"2024-09-14T05:31:21.128943+0000","last_undegraded":"2024-09-14T05:31:21.128943+0000","last_fullsized":"2024-09-14T05:31:21.128943+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1131,"log_dups_size":0,"ondisk_log_size":1131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:18:41.728775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018223110000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":14909,"num_read_kb":22344,"num_write":1338,"num_write_kb":6495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"113'367","reported_seq":2862,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521474+0000","last_change":"2024-09-14T05:31:03.024269+0000","last_active":"2024-09-14T05:31:21.521474+0000","last_peered":"2024-09-14T05:31:21.521474+0000","last_clean":"2024-09-14T05:31:21.521474+0000","last_became_active":"2024-09-14T05:29:24.015315+0000","last_became_peered":"2024-09-14T05:29:24.015315+0000","last_unstale":"2024-09-14T05:31:21.521474+0000","last_undegraded":"2024-09-14T05:31:21.521474+0000","last_fullsized":"2024-09-14T05:31:21.521474+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:27:50.755671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001918957,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2150,"num_read_kb":8871,"num_write":290,"num_write_kb":3939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"111'532","reported_seq":11844,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189148+0000","last_change":"2024-09-14T05:31:03.068658+0000","last_active":"2024-09-14T05:31:22.189148+0000","last_peered":"2024-09-14T05:31:22.189148+0000","last_clean":"2024-09-14T05:31:22.189148+0000","last_became_active":"2024-09-14T05:29:24.018854+0000","last_became_peered":"2024-09-14T05:29:24.018854+0000","last_unstale":"2024-09-14T05:31:22.189148+0000","last_undegraded":"2024-09-14T05:31:22.189148+0000","last_fullsized":"2024-09-14T05:31:22.189148+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:12:27.055181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000829428,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":12059,"num_read_kb":31916,"num_write":436,"num_write_kb":11843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"113'389","reported_seq":3039,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.272918+0000","last_change":"2024-09-14T05:31:03.034867+0000","last_active":"2024-09-14T05:31:20.272918+0000","last_peered":"2024-09-14T05:31:20.272918+0000","last_clean":"2024-09-14T05:31:20.272918+0000","last_became_active":"2024-09-14T05:29:24.010897+0000","last_became_peered":"2024-09-14T05:29:24.010897+0000","last_unstale":"2024-09-14T05:31:20.272918+0000","last_undegraded":"2024-09-14T05:31:20.272918+0000","last_fullsized":"2024-09-14T05:31:20.272918+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:33:58.717977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051683800000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2374,"num_read_kb":22456,"num_write":338,"num_write_kb":8939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"112'576","reported_seq":4039,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128547+0000","last_change":"2024-09-14T05:31:03.020915+0000","last_active":"2024-09-14T05:31:21.128547+0000","last_peered":"2024-09-14T05:31:21.128547+0000","last_clean":"2024-09-14T05:31:21.128547+0000","last_became_active":"2024-09-14T05:29:24.013792+0000","last_became_peered":"2024-09-14T05:29:24.013792+0000","last_unstale":"2024-09-14T05:31:21.128547+0000","last_undegraded":"2024-09-14T05:31:21.128547+0000","last_fullsized":"2024-09-14T05:31:21.128547+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:09:14.825887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001768403,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5197,"num_read_kb":24484,"num_write":526,"num_write_kb":3336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"112'490","reported_seq":2632,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.270506+0000","last_change":"2024-09-14T05:31:03.032142+0000","last_active":"2024-09-14T05:31:20.270506+0000","last_peered":"2024-09-14T05:31:20.270506+0000","last_clean":"2024-09-14T05:31:20.270506+0000","last_became_active":"2024-09-14T05:30:38.013627+0000","last_became_peered":"2024-09-14T05:30:38.013627+0000","last_unstale":"2024-09-14T05:31:20.270506+0000","last_undegraded":"2024-09-14T05:31:20.270506+0000","last_fullsized":"2024-09-14T05:31:20.270506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:55:09.154072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050464100000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3016,"num_read_kb":16678,"num_write":502,"num_write_kb":10866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2769134,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"113'680","reported_seq":2493,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128721+0000","last_change":"2024-09-14T05:31:03.019963+0000","last_active":"2024-09-14T05:31:21.128721+0000","last_peered":"2024-09-14T05:31:21.128721+0000","last_clean":"2024-09-14T05:31:21.128721+0000","last_became_active":"2024-09-14T05:29:24.014371+0000","last_became_peered":"2024-09-14T05:29:24.014371+0000","last_unstale":"2024-09-14T05:31:21.128721+0000","last_undegraded":"2024-09-14T05:31:21.128721+0000","last_fullsized":"2024-09-14T05:31:21.128721+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:06.352481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080910499999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3420,"num_read_kb":19916,"num_write":912,"num_write_kb":5503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":594,"num_object_clones":0,"num_object_copies":1188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":594,"num_whiteouts":590,"num_read":182740,"num_read_kb":565764,"num_write":20809,"num_write_kb":280821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19248,"ondisk_log_size":19248,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476768,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50776,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321064,"statfs":{"total":96636764160,"available":96584769536,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509471,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41036,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330804,"statfs":{"total":96636764160,"available":96594743296,"internally_reserved":0,"allocated":1560576,"data_stored":742034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28576,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343264,"statfs":{"total":96636764160,"available":96607502336,"internally_reserved":0,"allocated":1576960,"data_stored":747924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2158592,"data_stored":1332421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28568,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343272,"statfs":{"total":96636764160,"available":96607510528,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72988,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298852,"statfs":{"total":96636764160,"available":96562024448,"internally_reserved":0,"allocated":1576960,"data_stored":747924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509473,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51348,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320492,"statfs":{"total":96636764160,"available":96584183808,"internally_reserved":0,"allocated":2158592,"data_stored":1332421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:31:25.867 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T05:31:25.867 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:31:26.146 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:31:26.147 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:31:26.158 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-09-14T05:31:24.458624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":598,"num_object_clones":0,"num_object_copies":1196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":598,"num_whiteouts":590,"num_read":182806,"num_read_kb":565819,"num_write":20912,"num_write_kb":282451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19323,"ondisk_log_size":19323,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":339320,"kb_used_data":13428,"kb_used_omap":12,"kb_used_meta":325683,"kb_avail":754635400,"statfs":{"total":773094113280,"available":772746649600,"internally_reserved":0,"allocated":13750272,"data_stored":7146439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":333499952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000951"},"pg_stats":[{"pgid":"3.1f","version":"114'767","reported_seq":10977,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121322+0000","last_change":"2024-09-14T05:31:03.027720+0000","last_active":"2024-09-14T05:31:20.121322+0000","last_peered":"2024-09-14T05:31:20.121322+0000","last_clean":"2024-09-14T05:31:20.121322+0000","last_became_active":"2024-09-14T05:29:24.018892+0000","last_became_peered":"2024-09-14T05:29:24.018892+0000","last_unstale":"2024-09-14T05:31:20.121322+0000","last_undegraded":"2024-09-14T05:31:20.121322+0000","last_fullsized":"2024-09-14T05:31:20.121322+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:00:33.452652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013315950000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":12275,"num_read_kb":37400,"num_write":832,"num_write_kb":31155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"114'418","reported_seq":3310,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121298+0000","last_change":"2024-09-14T05:31:03.027683+0000","last_active":"2024-09-14T05:31:20.121298+0000","last_peered":"2024-09-14T05:31:20.121298+0000","last_clean":"2024-09-14T05:31:20.121298+0000","last_became_active":"2024-09-14T05:29:24.014468+0000","last_became_peered":"2024-09-14T05:29:24.014468+0000","last_unstale":"2024-09-14T05:31:20.121298+0000","last_undegraded":"2024-09-14T05:31:20.121298+0000","last_fullsized":"2024-09-14T05:31:20.121298+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:58.301444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001315826,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2805,"num_read_kb":32498,"num_write":409,"num_write_kb":21098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"113'1047","reported_seq":9884,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128908+0000","last_change":"2024-09-14T05:31:03.020460+0000","last_active":"2024-09-14T05:31:21.128908+0000","last_peered":"2024-09-14T05:31:21.128908+0000","last_clean":"2024-09-14T05:31:21.128908+0000","last_became_active":"2024-09-14T05:29:24.014251+0000","last_became_peered":"2024-09-14T05:29:24.014251+0000","last_unstale":"2024-09-14T05:31:21.128908+0000","last_undegraded":"2024-09-14T05:31:21.128908+0000","last_fullsized":"2024-09-14T05:31:21.128908+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1047,"log_dups_size":0,"ondisk_log_size":1047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:50:10.775143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012210070000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12253,"num_read_kb":23814,"num_write":1553,"num_write_kb":7077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"114'602","reported_seq":10588,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.273068+0000","last_change":"2024-09-14T05:31:03.035040+0000","last_active":"2024-09-14T05:31:20.273068+0000","last_peered":"2024-09-14T05:31:20.273068+0000","last_clean":"2024-09-14T05:31:20.273068+0000","last_became_active":"2024-09-14T05:29:24.012427+0000","last_became_peered":"2024-09-14T05:29:24.012427+0000","last_unstale":"2024-09-14T05:31:20.273068+0000","last_undegraded":"2024-09-14T05:31:20.273068+0000","last_fullsized":"2024-09-14T05:31:20.273068+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:11:08.020997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073664900000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":10982,"num_read_kb":35239,"num_write":569,"num_write_kb":4638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"112'587","reported_seq":3666,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130947+0000","last_change":"2024-09-14T05:31:03.020201+0000","last_active":"2024-09-14T05:31:21.130947+0000","last_peered":"2024-09-14T05:31:21.130947+0000","last_clean":"2024-09-14T05:31:21.130947+0000","last_became_active":"2024-09-14T05:29:24.012079+0000","last_became_peered":"2024-09-14T05:29:24.012079+0000","last_unstale":"2024-09-14T05:31:21.130947+0000","last_undegraded":"2024-09-14T05:31:21.130947+0000","last_fullsized":"2024-09-14T05:31:21.130947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:30:17.989902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010699310000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4414,"num_read_kb":26220,"num_write":576,"num_write_kb":21867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"110'1005","reported_seq":6499,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.273103+0000","last_change":"2024-09-14T05:31:03.035277+0000","last_active":"2024-09-14T05:31:20.273103+0000","last_peered":"2024-09-14T05:31:20.273103+0000","last_clean":"2024-09-14T05:31:20.273103+0000","last_became_active":"2024-09-14T05:29:24.012461+0000","last_became_peered":"2024-09-14T05:29:24.012461+0000","last_unstale":"2024-09-14T05:31:20.273103+0000","last_undegraded":"2024-09-14T05:31:20.273103+0000","last_fullsized":"2024-09-14T05:31:20.273103+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:23:06.828927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090417700000000004,"stat_sum":{"num_bytes":19,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":10524,"num_read_kb":34488,"num_write":1274,"num_write_kb":8459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"112'581","reported_seq":2237,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130927+0000","last_change":"2024-09-14T05:31:03.019768+0000","last_active":"2024-09-14T05:31:21.130927+0000","last_peered":"2024-09-14T05:31:21.130927+0000","last_clean":"2024-09-14T05:31:21.130927+0000","last_became_active":"2024-09-14T05:29:24.018753+0000","last_became_peered":"2024-09-14T05:29:24.018753+0000","last_unstale":"2024-09-14T05:31:21.130927+0000","last_undegraded":"2024-09-14T05:31:21.130927+0000","last_fullsized":"2024-09-14T05:31:21.130927+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:38:28.243765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074058200000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2590,"num_read_kb":13584,"num_write":688,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"113'699","reported_seq":2937,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121320+0000","last_change":"2024-09-14T05:31:03.027199+0000","last_active":"2024-09-14T05:31:20.121320+0000","last_peered":"2024-09-14T05:31:20.121320+0000","last_clean":"2024-09-14T05:31:20.121320+0000","last_became_active":"2024-09-14T05:29:24.011176+0000","last_became_peered":"2024-09-14T05:29:24.011176+0000","last_unstale":"2024-09-14T05:31:20.121320+0000","last_undegraded":"2024-09-14T05:31:20.121320+0000","last_fullsized":"2024-09-14T05:31:20.121320+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:27:54.861994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091515699999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4708,"num_read_kb":7495,"num_write":700,"num_write_kb":3426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"112'357","reported_seq":3176,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121325+0000","last_change":"2024-09-14T05:31:03.028059+0000","last_active":"2024-09-14T05:31:20.121325+0000","last_peered":"2024-09-14T05:31:20.121325+0000","last_clean":"2024-09-14T05:31:20.121325+0000","last_became_active":"2024-09-14T05:29:24.014005+0000","last_became_peered":"2024-09-14T05:29:24.014005+0000","last_unstale":"2024-09-14T05:31:20.121325+0000","last_undegraded":"2024-09-14T05:31:20.121325+0000","last_fullsized":"2024-09-14T05:31:20.121325+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:18:52.725886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001656189,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3149,"num_read_kb":11342,"num_write":316,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"113'340","reported_seq":9918,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521406+0000","last_change":"2024-09-14T05:31:03.023758+0000","last_active":"2024-09-14T05:31:21.521406+0000","last_peered":"2024-09-14T05:31:21.521406+0000","last_clean":"2024-09-14T05:31:21.521406+0000","last_became_active":"2024-09-14T05:29:24.013434+0000","last_became_peered":"2024-09-14T05:29:24.013434+0000","last_unstale":"2024-09-14T05:31:21.521406+0000","last_undegraded":"2024-09-14T05:31:21.521406+0000","last_fullsized":"2024-09-14T05:31:21.521406+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:44.069349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418475,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":9226,"num_read_kb":22920,"num_write":296,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"113'440","reported_seq":2749,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.188832+0000","last_change":"2024-09-14T05:31:03.068934+0000","last_active":"2024-09-14T05:31:22.188832+0000","last_peered":"2024-09-14T05:31:22.188832+0000","last_clean":"2024-09-14T05:31:22.188832+0000","last_became_active":"2024-09-14T05:29:24.014457+0000","last_became_peered":"2024-09-14T05:29:24.014457+0000","last_unstale":"2024-09-14T05:31:22.188832+0000","last_undegraded":"2024-09-14T05:31:22.188832+0000","last_fullsized":"2024-09-14T05:31:22.188832+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:01:09.992476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011032850000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2612,"num_read_kb":9349,"num_write":416,"num_write_kb":12875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"109'843","reported_seq":2591,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128823+0000","last_change":"2024-09-14T05:31:03.020119+0000","last_active":"2024-09-14T05:31:21.128823+0000","last_peered":"2024-09-14T05:31:21.128823+0000","last_clean":"2024-09-14T05:31:21.128823+0000","last_became_active":"2024-09-14T05:29:24.013266+0000","last_became_peered":"2024-09-14T05:29:24.013266+0000","last_unstale":"2024-09-14T05:31:21.128823+0000","last_undegraded":"2024-09-14T05:31:21.128823+0000","last_fullsized":"2024-09-14T05:31:21.128823+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:34:03.137226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087100800000000005,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":4368,"num_read_kb":5632,"num_write":1075,"num_write_kb":4026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":240,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128815+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:31:21.128815+0000","last_peered":"2024-09-14T05:31:21.128815+0000","last_clean":"2024-09-14T05:31:21.128815+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:31:21.128815+0000","last_undegraded":"2024-09-14T05:31:21.128815+0000","last_fullsized":"2024-09-14T05:31:21.128815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"114'375","reported_seq":2319,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.131016+0000","last_change":"2024-09-14T05:31:03.020581+0000","last_active":"2024-09-14T05:31:21.131016+0000","last_peered":"2024-09-14T05:31:21.131016+0000","last_clean":"2024-09-14T05:31:21.131016+0000","last_became_active":"2024-09-14T05:29:24.012099+0000","last_became_peered":"2024-09-14T05:29:24.012099+0000","last_unstale":"2024-09-14T05:31:21.131016+0000","last_undegraded":"2024-09-14T05:31:21.131016+0000","last_fullsized":"2024-09-14T05:31:21.131016+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:44:38.747225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013882689999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2144,"num_read_kb":3725,"num_write":284,"num_write_kb":5458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":235,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.131025+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:31:21.131025+0000","last_peered":"2024-09-14T05:31:21.131025+0000","last_clean":"2024-09-14T05:31:21.131025+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:31:21.131025+0000","last_undegraded":"2024-09-14T05:31:21.131025+0000","last_fullsized":"2024-09-14T05:31:21.131025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"114'951","reported_seq":4046,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521397+0000","last_change":"2024-09-14T05:31:03.023762+0000","last_active":"2024-09-14T05:31:21.521397+0000","last_peered":"2024-09-14T05:31:21.521397+0000","last_clean":"2024-09-14T05:31:21.521397+0000","last_became_active":"2024-09-14T05:30:31.398064+0000","last_became_peered":"2024-09-14T05:30:31.398064+0000","last_unstale":"2024-09-14T05:31:21.521397+0000","last_undegraded":"2024-09-14T05:31:21.521397+0000","last_fullsized":"2024-09-14T05:31:21.521397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:15:44.039992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014417360000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6829,"num_read_kb":8465,"num_write":1096,"num_write_kb":5167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":648894,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128980+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:31:21.128980+0000","last_peered":"2024-09-14T05:31:21.128980+0000","last_clean":"2024-09-14T05:31:21.128980+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:31:21.128980+0000","last_undegraded":"2024-09-14T05:31:21.128980+0000","last_fullsized":"2024-09-14T05:31:21.128980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"112'569","reported_seq":2718,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130208+0000","last_change":"2024-09-14T05:31:20.123367+0000","last_active":"2024-09-14T05:31:21.130208+0000","last_peered":"2024-09-14T05:31:21.130208+0000","last_clean":"2024-09-14T05:31:21.130208+0000","last_became_active":"2024-09-14T05:31:20.122139+0000","last_became_peered":"2024-09-14T05:31:20.122139+0000","last_unstale":"2024-09-14T05:31:21.130208+0000","last_undegraded":"2024-09-14T05:31:21.130208+0000","last_fullsized":"2024-09-14T05:31:21.130208+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:03:12.206743+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0050460000000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3487,"num_read_kb":12454,"num_write":614,"num_write_kb":11389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":234,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121553+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:31:20.121553+0000","last_peered":"2024-09-14T05:31:20.121553+0000","last_clean":"2024-09-14T05:31:20.121553+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:31:20.121553+0000","last_undegraded":"2024-09-14T05:31:20.121553+0000","last_fullsized":"2024-09-14T05:31:20.121553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"114'351","reported_seq":1951,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521367+0000","last_change":"2024-09-14T05:31:03.024292+0000","last_active":"2024-09-14T05:31:21.521367+0000","last_peered":"2024-09-14T05:31:21.521367+0000","last_clean":"2024-09-14T05:31:21.521367+0000","last_became_active":"2024-09-14T05:29:24.016940+0000","last_became_peered":"2024-09-14T05:29:24.016940+0000","last_unstale":"2024-09-14T05:31:21.521367+0000","last_undegraded":"2024-09-14T05:31:21.521367+0000","last_fullsized":"2024-09-14T05:31:21.521367+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:42:00.348036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001928208,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1597,"num_read_kb":4408,"num_write":335,"num_write_kb":4001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"114'642","reported_seq":3022,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189410+0000","last_change":"2024-09-14T05:31:03.069465+0000","last_active":"2024-09-14T05:31:22.189410+0000","last_peered":"2024-09-14T05:31:22.189410+0000","last_clean":"2024-09-14T05:31:22.189410+0000","last_became_active":"2024-09-14T05:29:24.015905+0000","last_became_peered":"2024-09-14T05:29:24.015905+0000","last_unstale":"2024-09-14T05:31:22.189410+0000","last_undegraded":"2024-09-14T05:31:22.189410+0000","last_fullsized":"2024-09-14T05:31:22.189410+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:31:28.697477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00154122,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":2532,"num_read_kb":9709,"num_write":637,"num_write_kb":7772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":243,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128727+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:31:21.128727+0000","last_peered":"2024-09-14T05:31:21.128727+0000","last_clean":"2024-09-14T05:31:21.128727+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:31:21.128727+0000","last_undegraded":"2024-09-14T05:31:21.128727+0000","last_fullsized":"2024-09-14T05:31:21.128727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"113'485","reported_seq":10963,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.121177+0000","last_change":"2024-09-14T05:31:03.026777+0000","last_active":"2024-09-14T05:31:20.121177+0000","last_peered":"2024-09-14T05:31:20.121177+0000","last_clean":"2024-09-14T05:31:20.121177+0000","last_became_active":"2024-09-14T05:29:24.011764+0000","last_became_peered":"2024-09-14T05:29:24.011764+0000","last_unstale":"2024-09-14T05:31:20.121177+0000","last_undegraded":"2024-09-14T05:31:20.121177+0000","last_fullsized":"2024-09-14T05:31:20.121177+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:07.795205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057339999999999995,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":11805,"num_read_kb":38991,"num_write":413,"num_write_kb":4959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":237,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.270563+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:31:20.270563+0000","last_peered":"2024-09-14T05:31:20.270563+0000","last_clean":"2024-09-14T05:31:20.270563+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:31:20.270563+0000","last_undegraded":"2024-09-14T05:31:20.270563+0000","last_fullsized":"2024-09-14T05:31:20.270563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"110'701","reported_seq":1977,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130208+0000","last_change":"2024-09-14T05:31:20.123603+0000","last_active":"2024-09-14T05:31:21.130208+0000","last_peered":"2024-09-14T05:31:21.130208+0000","last_clean":"2024-09-14T05:31:21.130208+0000","last_became_active":"2024-09-14T05:31:20.122523+0000","last_became_peered":"2024-09-14T05:31:20.122523+0000","last_unstale":"2024-09-14T05:31:21.130208+0000","last_undegraded":"2024-09-14T05:31:21.130208+0000","last_fullsized":"2024-09-14T05:31:21.130208+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:44:47.461984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022421440000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3140,"num_read_kb":7266,"num_write":925,"num_write_kb":10032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":241,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128547+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:31:21.128547+0000","last_peered":"2024-09-14T05:31:21.128547+0000","last_clean":"2024-09-14T05:31:21.128547+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:31:21.128547+0000","last_undegraded":"2024-09-14T05:31:21.128547+0000","last_fullsized":"2024-09-14T05:31:21.128547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"113'585","reported_seq":2767,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130931+0000","last_change":"2024-09-14T05:31:03.020204+0000","last_active":"2024-09-14T05:31:21.130931+0000","last_peered":"2024-09-14T05:31:21.130931+0000","last_clean":"2024-09-14T05:31:21.130931+0000","last_became_active":"2024-09-14T05:29:24.015792+0000","last_became_peered":"2024-09-14T05:29:24.015792+0000","last_unstale":"2024-09-14T05:31:21.130931+0000","last_undegraded":"2024-09-14T05:31:21.130931+0000","last_fullsized":"2024-09-14T05:31:21.130931+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:12:45.477846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011628109999999999,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":3532,"num_read_kb":9462,"num_write":620,"num_write_kb":5631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":242,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128830+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:31:21.128830+0000","last_peered":"2024-09-14T05:31:21.128830+0000","last_clean":"2024-09-14T05:31:21.128830+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:31:21.128830+0000","last_undegraded":"2024-09-14T05:31:21.128830+0000","last_fullsized":"2024-09-14T05:31:21.128830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":327,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.272917+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:31:20.272917+0000","last_peered":"2024-09-14T05:31:20.272917+0000","last_clean":"2024-09-14T05:31:20.272917+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:31:20.272917+0000","last_undegraded":"2024-09-14T05:31:20.272917+0000","last_fullsized":"2024-09-14T05:31:20.272917+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"113'395","reported_seq":2290,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130890+0000","last_change":"2024-09-14T05:31:03.019738+0000","last_active":"2024-09-14T05:31:21.130890+0000","last_peered":"2024-09-14T05:31:21.130890+0000","last_clean":"2024-09-14T05:31:21.130890+0000","last_became_active":"2024-09-14T05:29:24.012222+0000","last_became_peered":"2024-09-14T05:29:24.012222+0000","last_unstale":"2024-09-14T05:31:21.130890+0000","last_undegraded":"2024-09-14T05:31:21.130890+0000","last_fullsized":"2024-09-14T05:31:21.130890+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:52:56.721362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072525400000000005,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2833,"num_read_kb":6029,"num_write":327,"num_write_kb":3878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"113'608","reported_seq":2440,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128564+0000","last_change":"2024-09-14T05:31:03.020704+0000","last_active":"2024-09-14T05:31:21.128564+0000","last_peered":"2024-09-14T05:31:21.128564+0000","last_clean":"2024-09-14T05:31:21.128564+0000","last_became_active":"2024-09-14T05:29:24.012041+0000","last_became_peered":"2024-09-14T05:29:24.012041+0000","last_unstale":"2024-09-14T05:31:21.128564+0000","last_undegraded":"2024-09-14T05:31:21.128564+0000","last_fullsized":"2024-09-14T05:31:21.128564+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:42:01.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001433679,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2914,"num_read_kb":7354,"num_write":748,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"113'655","reported_seq":2691,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128609+0000","last_change":"2024-09-14T05:31:03.021329+0000","last_active":"2024-09-14T05:31:21.128609+0000","last_peered":"2024-09-14T05:31:21.128609+0000","last_clean":"2024-09-14T05:31:21.128609+0000","last_became_active":"2024-09-14T05:29:24.011403+0000","last_became_peered":"2024-09-14T05:29:24.011403+0000","last_unstale":"2024-09-14T05:31:21.128609+0000","last_undegraded":"2024-09-14T05:31:21.128609+0000","last_fullsized":"2024-09-14T05:31:21.128609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:04:32.706037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00168743,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":3956,"num_read_kb":8037,"num_write":714,"num_write_kb":4980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"112'632","reported_seq":10267,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.270508+0000","last_change":"2024-09-14T05:31:03.032462+0000","last_active":"2024-09-14T05:31:20.270508+0000","last_peered":"2024-09-14T05:31:20.270508+0000","last_clean":"2024-09-14T05:31:20.270508+0000","last_became_active":"2024-09-14T05:29:24.006836+0000","last_became_peered":"2024-09-14T05:29:24.006836+0000","last_unstale":"2024-09-14T05:31:20.270508+0000","last_undegraded":"2024-09-14T05:31:20.270508+0000","last_fullsized":"2024-09-14T05:31:20.270508+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:22:08.301740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060523399999999996,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":12123,"num_read_kb":29901,"num_write":689,"num_write_kb":13072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"113'448","reported_seq":3554,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189014+0000","last_change":"2024-09-14T05:31:03.068410+0000","last_active":"2024-09-14T05:31:22.189014+0000","last_peered":"2024-09-14T05:31:22.189014+0000","last_clean":"2024-09-14T05:31:22.189014+0000","last_became_active":"2024-09-14T05:29:24.012574+0000","last_became_peered":"2024-09-14T05:29:24.012574+0000","last_unstale":"2024-09-14T05:31:22.189014+0000","last_undegraded":"2024-09-14T05:31:22.189014+0000","last_fullsized":"2024-09-14T05:31:22.189014+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:58:16.204790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064954900000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2817,"num_read_kb":13317,"num_write":361,"num_write_kb":9799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"112'1131","reported_seq":11394,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128943+0000","last_change":"2024-09-14T05:31:03.021253+0000","last_active":"2024-09-14T05:31:21.128943+0000","last_peered":"2024-09-14T05:31:21.128943+0000","last_clean":"2024-09-14T05:31:21.128943+0000","last_became_active":"2024-09-14T05:29:24.012401+0000","last_became_peered":"2024-09-14T05:29:24.012401+0000","last_unstale":"2024-09-14T05:31:21.128943+0000","last_undegraded":"2024-09-14T05:31:21.128943+0000","last_fullsized":"2024-09-14T05:31:21.128943+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1131,"log_dups_size":0,"ondisk_log_size":1131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:18:41.728775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018223110000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":14909,"num_read_kb":22344,"num_write":1338,"num_write_kb":6495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"113'367","reported_seq":2862,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521474+0000","last_change":"2024-09-14T05:31:03.024269+0000","last_active":"2024-09-14T05:31:21.521474+0000","last_peered":"2024-09-14T05:31:21.521474+0000","last_clean":"2024-09-14T05:31:21.521474+0000","last_became_active":"2024-09-14T05:29:24.015315+0000","last_became_peered":"2024-09-14T05:29:24.015315+0000","last_unstale":"2024-09-14T05:31:21.521474+0000","last_undegraded":"2024-09-14T05:31:21.521474+0000","last_fullsized":"2024-09-14T05:31:21.521474+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:27:50.755671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001918957,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2150,"num_read_kb":8871,"num_write":290,"num_write_kb":3939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"111'532","reported_seq":11844,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189148+0000","last_change":"2024-09-14T05:31:03.068658+0000","last_active":"2024-09-14T05:31:22.189148+0000","last_peered":"2024-09-14T05:31:22.189148+0000","last_clean":"2024-09-14T05:31:22.189148+0000","last_became_active":"2024-09-14T05:29:24.018854+0000","last_became_peered":"2024-09-14T05:29:24.018854+0000","last_unstale":"2024-09-14T05:31:22.189148+0000","last_undegraded":"2024-09-14T05:31:22.189148+0000","last_fullsized":"2024-09-14T05:31:22.189148+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:12:27.055181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000829428,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":12059,"num_read_kb":31916,"num_write":436,"num_write_kb":11843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"113'389","reported_seq":3039,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.272918+0000","last_change":"2024-09-14T05:31:03.034867+0000","last_active":"2024-09-14T05:31:20.272918+0000","last_peered":"2024-09-14T05:31:20.272918+0000","last_clean":"2024-09-14T05:31:20.272918+0000","last_became_active":"2024-09-14T05:29:24.010897+0000","last_became_peered":"2024-09-14T05:29:24.010897+0000","last_unstale":"2024-09-14T05:31:20.272918+0000","last_undegraded":"2024-09-14T05:31:20.272918+0000","last_fullsized":"2024-09-14T05:31:20.272918+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:33:58.717977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051683800000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2374,"num_read_kb":22456,"num_write":338,"num_write_kb":8939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"112'576","reported_seq":4039,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128547+0000","last_change":"2024-09-14T05:31:03.020915+0000","last_active":"2024-09-14T05:31:21.128547+0000","last_peered":"2024-09-14T05:31:21.128547+0000","last_clean":"2024-09-14T05:31:21.128547+0000","last_became_active":"2024-09-14T05:29:24.013792+0000","last_became_peered":"2024-09-14T05:29:24.013792+0000","last_unstale":"2024-09-14T05:31:21.128547+0000","last_undegraded":"2024-09-14T05:31:21.128547+0000","last_fullsized":"2024-09-14T05:31:21.128547+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:09:14.825887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001768403,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5197,"num_read_kb":24484,"num_write":526,"num_write_kb":3336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"112'490","reported_seq":2632,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-14T05:31:20.270506+0000","last_change":"2024-09-14T05:31:03.032142+0000","last_active":"2024-09-14T05:31:20.270506+0000","last_peered":"2024-09-14T05:31:20.270506+0000","last_clean":"2024-09-14T05:31:20.270506+0000","last_became_active":"2024-09-14T05:30:38.013627+0000","last_became_peered":"2024-09-14T05:30:38.013627+0000","last_unstale":"2024-09-14T05:31:20.270506+0000","last_undegraded":"2024-09-14T05:31:20.270506+0000","last_fullsized":"2024-09-14T05:31:20.270506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:55:09.154072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050464100000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3016,"num_read_kb":16678,"num_write":502,"num_write_kb":10866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2769134,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"113'680","reported_seq":2493,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128721+0000","last_change":"2024-09-14T05:31:03.019963+0000","last_active":"2024-09-14T05:31:21.128721+0000","last_peered":"2024-09-14T05:31:21.128721+0000","last_clean":"2024-09-14T05:31:21.128721+0000","last_became_active":"2024-09-14T05:29:24.014371+0000","last_became_peered":"2024-09-14T05:29:24.014371+0000","last_unstale":"2024-09-14T05:31:21.128721+0000","last_undegraded":"2024-09-14T05:31:21.128721+0000","last_fullsized":"2024-09-14T05:31:21.128721+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:06.352481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080910499999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3420,"num_read_kb":19916,"num_write":912,"num_write_kb":5503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":594,"num_object_clones":0,"num_object_copies":1188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":594,"num_whiteouts":590,"num_read":182740,"num_read_kb":565764,"num_write":20809,"num_write_kb":280821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19248,"ondisk_log_size":19248,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476768,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50776,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321064,"statfs":{"total":96636764160,"available":96584769536,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509471,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41036,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330804,"statfs":{"total":96636764160,"available":96594743296,"internally_reserved":0,"allocated":1560576,"data_stored":742034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28576,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343264,"statfs":{"total":96636764160,"available":96607502336,"internally_reserved":0,"allocated":1576960,"data_stored":747924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2158592,"data_stored":1332421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28568,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343272,"statfs":{"total":96636764160,"available":96607510528,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72988,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298852,"statfs":{"total":96636764160,"available":96562024448,"internally_reserved":0,"allocated":1576960,"data_stored":747924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509473,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51348,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320492,"statfs":{"total":96636764160,"available":96584183808,"internally_reserved":0,"allocated":2158592,"data_stored":1332421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:31:26.160 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T05:31:26.446 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:31:26.447 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":132,"fsid":"1127be44-5f14-4e7d-ad23-f608afe154f5","created":"2024-09-14T05:28:54.203072+0000","modified":"2024-09-14T05:31:21.124952+0000","last_up_change":"2024-09-14T05:29:06.755531+0000","last_in_change":"2024-09-14T05:28:56.535048+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T05:29:06.452744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T05:29:12.380843+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T05:29:22.335419+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":40,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7c7817f-4a65-4f66-9af5-d31d0a92cf30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6819","nonce":2926109885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6821","nonce":2926109885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6825","nonce":2926109885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2926109885},{"type":"v1","addr":"172.21.15.71:6823","nonce":2926109885}]},"public_addr":"172.21.15.71:6819/2926109885","cluster_addr":"172.21.15.71:6821/2926109885","heartbeat_back_addr":"172.21.15.71:6825/2926109885","heartbeat_front_addr":"172.21.15.71:6823/2926109885","state":["exists","up"]},{"osd":1,"uuid":"e184604e-b973-4b7a-bba2-6f471753a0fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6827","nonce":345649648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6829","nonce":345649648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6833","nonce":345649648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":345649648},{"type":"v1","addr":"172.21.15.71:6831","nonce":345649648}]},"public_addr":"172.21.15.71:6827/345649648","cluster_addr":"172.21.15.71:6829/345649648","heartbeat_back_addr":"172.21.15.71:6833/345649648","heartbeat_front_addr":"172.21.15.71:6831/345649648","state":["exists","up"]},{"osd":2,"uuid":"d8b70f09-5afb-4296-b8c9-719c172e0e70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6801","nonce":2721860691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6803","nonce":2721860691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6807","nonce":2721860691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2721860691},{"type":"v1","addr":"172.21.15.71:6805","nonce":2721860691}]},"public_addr":"172.21.15.71:6801/2721860691","cluster_addr":"172.21.15.71:6803/2721860691","heartbeat_back_addr":"172.21.15.71:6807/2721860691","heartbeat_front_addr":"172.21.15.71:6805/2721860691","state":["exists","up"]},{"osd":3,"uuid":"26d58cd0-c7d5-4a8e-b2e0-8c3f29da096c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6809","nonce":494607463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6811","nonce":494607463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6815","nonce":494607463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":494607463},{"type":"v1","addr":"172.21.15.71:6813","nonce":494607463}]},"public_addr":"172.21.15.71:6809/494607463","cluster_addr":"172.21.15.71:6811/494607463","heartbeat_back_addr":"172.21.15.71:6815/494607463","heartbeat_front_addr":"172.21.15.71:6813/494607463","state":["exists","up"]},{"osd":4,"uuid":"515cb7d6-16eb-4aff-973a-7f7ed49325ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6817","nonce":2240133417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6819","nonce":2240133417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6823","nonce":2240133417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2240133417},{"type":"v1","addr":"172.21.15.150:6821","nonce":2240133417}]},"public_addr":"172.21.15.150:6817/2240133417","cluster_addr":"172.21.15.150:6819/2240133417","heartbeat_back_addr":"172.21.15.150:6823/2240133417","heartbeat_front_addr":"172.21.15.150:6821/2240133417","state":["exists","up"]},{"osd":5,"uuid":"c434b53a-04be-45b9-b89a-193654133950","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6809","nonce":2393825223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6811","nonce":2393825223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6815","nonce":2393825223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2393825223},{"type":"v1","addr":"172.21.15.150:6813","nonce":2393825223}]},"public_addr":"172.21.15.150:6809/2393825223","cluster_addr":"172.21.15.150:6811/2393825223","heartbeat_back_addr":"172.21.15.150:6815/2393825223","heartbeat_front_addr":"172.21.15.150:6813/2393825223","state":["exists","up"]},{"osd":6,"uuid":"3d2f8405-9286-4b40-8acd-036f67373ac6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6801","nonce":3235729544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6803","nonce":3235729544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6807","nonce":3235729544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3235729544},{"type":"v1","addr":"172.21.15.150:6805","nonce":3235729544}]},"public_addr":"172.21.15.150:6801/3235729544","cluster_addr":"172.21.15.150:6803/3235729544","heartbeat_back_addr":"172.21.15.150:6807/3235729544","heartbeat_front_addr":"172.21.15.150:6805/3235729544","state":["exists","up"]},{"osd":7,"uuid":"6724f080-4dc3-465a-939f-b023985bf998","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6825","nonce":2381001355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6827","nonce":2381001355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6831","nonce":2381001355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2381001355},{"type":"v1","addr":"172.21.15.150:6829","nonce":2381001355}]},"public_addr":"172.21.15.150:6825/2381001355","cluster_addr":"172.21.15.150:6827/2381001355","heartbeat_back_addr":"172.21.15.150:6831/2381001355","heartbeat_front_addr":"172.21.15.150:6829/2381001355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.548803+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.075023+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.117036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.332113+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.330371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:02.753946+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:02.071186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T05:29:03.378781+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.7","osds":[2,4]},{"pgid":"3.12","osds":[7,0]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T05:31:27.459 INFO:tasks.ceph:Scrubbing osd.0 2024-09-14T05:31:27.460 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-14T05:31:27.592 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:27.592 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T05:31:27.593 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:27.603 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-14T05:31:27.887 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 0 to deep-scrub 2024-09-14T05:31:27.900 INFO:tasks.ceph:Scrubbing osd.1 2024-09-14T05:31:27.900 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-14T05:31:28.023 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:28.024 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T05:31:28.024 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:28.034 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-14T05:31:28.324 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 1 to deep-scrub 2024-09-14T05:31:28.337 INFO:tasks.ceph:Scrubbing osd.2 2024-09-14T05:31:28.337 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-14T05:31:28.460 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:28.460 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T05:31:28.461 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:28.471 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-14T05:31:28.755 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 2 to deep-scrub 2024-09-14T05:31:28.768 INFO:tasks.ceph:Scrubbing osd.3 2024-09-14T05:31:28.769 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-14T05:31:28.895 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:28.895 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T05:31:28.895 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:28.905 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-14T05:31:29.191 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 3 to deep-scrub 2024-09-14T05:31:29.204 INFO:tasks.ceph:Scrubbing osd.4 2024-09-14T05:31:29.204 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-14T05:31:29.329 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:29.329 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T05:31:29.329 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:29.339 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-14T05:31:29.623 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 4 to deep-scrub 2024-09-14T05:31:29.636 INFO:tasks.ceph:Scrubbing osd.5 2024-09-14T05:31:29.637 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-14T05:31:29.759 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:29.759 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T05:31:29.759 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:29.770 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-14T05:31:30.058 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 5 to deep-scrub 2024-09-14T05:31:30.075 INFO:tasks.ceph:Scrubbing osd.6 2024-09-14T05:31:30.076 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-14T05:31:30.198 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:30.198 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T05:31:30.198 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:30.209 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-14T05:31:30.499 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 6 to deep-scrub 2024-09-14T05:31:30.512 INFO:tasks.ceph:Scrubbing osd.7 2024-09-14T05:31:30.512 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-14T05:31:30.635 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-14T05:31:30.636 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T05:31:30.636 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-14T05:31:30.646 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-14T05:31:30.933 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 7 to deep-scrub 2024-09-14T05:31:30.946 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:31:31.236 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:31:31.237 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:31:31.248 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-09-14T05:31:30.459496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":598,"num_object_clones":0,"num_object_copies":1196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":598,"num_whiteouts":590,"num_read":182806,"num_read_kb":565819,"num_write":20912,"num_write_kb":282451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19323,"ondisk_log_size":19323,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":339352,"kb_used_data":13464,"kb_used_omap":12,"kb_used_meta":325683,"kb_avail":754635368,"statfs":{"total":773094113280,"available":772746616832,"internally_reserved":0,"allocated":13787136,"data_stored":7164052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":333499952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001581"},"pg_stats":[{"pgid":"3.1f","version":"114'767","reported_seq":10979,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.188855+0000","last_change":"2024-09-14T05:31:03.027720+0000","last_active":"2024-09-14T05:31:22.188855+0000","last_peered":"2024-09-14T05:31:22.188855+0000","last_clean":"2024-09-14T05:31:22.188855+0000","last_became_active":"2024-09-14T05:29:24.018892+0000","last_became_peered":"2024-09-14T05:29:24.018892+0000","last_unstale":"2024-09-14T05:31:22.188855+0000","last_undegraded":"2024-09-14T05:31:22.188855+0000","last_fullsized":"2024-09-14T05:31:22.188855+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:00:33.452652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013315950000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":12275,"num_read_kb":37400,"num_write":832,"num_write_kb":31155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"114'418","reported_seq":3313,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.188844+0000","last_change":"2024-09-14T05:31:03.027683+0000","last_active":"2024-09-14T05:31:22.188844+0000","last_peered":"2024-09-14T05:31:22.188844+0000","last_clean":"2024-09-14T05:31:22.188844+0000","last_became_active":"2024-09-14T05:29:24.014468+0000","last_became_peered":"2024-09-14T05:29:24.014468+0000","last_unstale":"2024-09-14T05:31:22.188844+0000","last_undegraded":"2024-09-14T05:31:22.188844+0000","last_fullsized":"2024-09-14T05:31:22.188844+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:58.301444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001315826,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2805,"num_read_kb":32498,"num_write":409,"num_write_kb":21098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"113'1047","reported_seq":9884,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128908+0000","last_change":"2024-09-14T05:31:03.020460+0000","last_active":"2024-09-14T05:31:21.128908+0000","last_peered":"2024-09-14T05:31:21.128908+0000","last_clean":"2024-09-14T05:31:21.128908+0000","last_became_active":"2024-09-14T05:29:24.014251+0000","last_became_peered":"2024-09-14T05:29:24.014251+0000","last_unstale":"2024-09-14T05:31:21.128908+0000","last_undegraded":"2024-09-14T05:31:21.128908+0000","last_fullsized":"2024-09-14T05:31:21.128908+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1047,"log_dups_size":0,"ondisk_log_size":1047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:50:10.775143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012210070000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12253,"num_read_kb":23814,"num_write":1553,"num_write_kb":7077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"114'602","reported_seq":10590,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189279+0000","last_change":"2024-09-14T05:31:03.035040+0000","last_active":"2024-09-14T05:31:22.189279+0000","last_peered":"2024-09-14T05:31:22.189279+0000","last_clean":"2024-09-14T05:31:22.189279+0000","last_became_active":"2024-09-14T05:29:24.012427+0000","last_became_peered":"2024-09-14T05:29:24.012427+0000","last_unstale":"2024-09-14T05:31:22.189279+0000","last_undegraded":"2024-09-14T05:31:22.189279+0000","last_fullsized":"2024-09-14T05:31:22.189279+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:11:08.020997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073664900000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":10982,"num_read_kb":35239,"num_write":569,"num_write_kb":4638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"112'587","reported_seq":3666,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130947+0000","last_change":"2024-09-14T05:31:03.020201+0000","last_active":"2024-09-14T05:31:21.130947+0000","last_peered":"2024-09-14T05:31:21.130947+0000","last_clean":"2024-09-14T05:31:21.130947+0000","last_became_active":"2024-09-14T05:29:24.012079+0000","last_became_peered":"2024-09-14T05:29:24.012079+0000","last_unstale":"2024-09-14T05:31:21.130947+0000","last_undegraded":"2024-09-14T05:31:21.130947+0000","last_fullsized":"2024-09-14T05:31:21.130947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:30:17.989902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010699310000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4414,"num_read_kb":26220,"num_write":576,"num_write_kb":21867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"110'1005","reported_seq":6501,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.188973+0000","last_change":"2024-09-14T05:31:03.035277+0000","last_active":"2024-09-14T05:31:22.188973+0000","last_peered":"2024-09-14T05:31:22.188973+0000","last_clean":"2024-09-14T05:31:22.188973+0000","last_became_active":"2024-09-14T05:29:24.012461+0000","last_became_peered":"2024-09-14T05:29:24.012461+0000","last_unstale":"2024-09-14T05:31:22.188973+0000","last_undegraded":"2024-09-14T05:31:22.188973+0000","last_fullsized":"2024-09-14T05:31:22.188973+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:23:06.828927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090417700000000004,"stat_sum":{"num_bytes":19,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":10524,"num_read_kb":34488,"num_write":1274,"num_write_kb":8459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"112'581","reported_seq":2237,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130927+0000","last_change":"2024-09-14T05:31:03.019768+0000","last_active":"2024-09-14T05:31:21.130927+0000","last_peered":"2024-09-14T05:31:21.130927+0000","last_clean":"2024-09-14T05:31:21.130927+0000","last_became_active":"2024-09-14T05:29:24.018753+0000","last_became_peered":"2024-09-14T05:29:24.018753+0000","last_unstale":"2024-09-14T05:31:21.130927+0000","last_undegraded":"2024-09-14T05:31:21.130927+0000","last_fullsized":"2024-09-14T05:31:21.130927+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:38:28.243765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074058200000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2590,"num_read_kb":13584,"num_write":688,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"113'699","reported_seq":2939,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189080+0000","last_change":"2024-09-14T05:31:03.027199+0000","last_active":"2024-09-14T05:31:22.189080+0000","last_peered":"2024-09-14T05:31:22.189080+0000","last_clean":"2024-09-14T05:31:22.189080+0000","last_became_active":"2024-09-14T05:29:24.011176+0000","last_became_peered":"2024-09-14T05:29:24.011176+0000","last_unstale":"2024-09-14T05:31:22.189080+0000","last_undegraded":"2024-09-14T05:31:22.189080+0000","last_fullsized":"2024-09-14T05:31:22.189080+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:27:54.861994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091515699999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4708,"num_read_kb":7495,"num_write":700,"num_write_kb":3426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"112'357","reported_seq":3178,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.188922+0000","last_change":"2024-09-14T05:31:03.028059+0000","last_active":"2024-09-14T05:31:22.188922+0000","last_peered":"2024-09-14T05:31:22.188922+0000","last_clean":"2024-09-14T05:31:22.188922+0000","last_became_active":"2024-09-14T05:29:24.014005+0000","last_became_peered":"2024-09-14T05:29:24.014005+0000","last_unstale":"2024-09-14T05:31:22.188922+0000","last_undegraded":"2024-09-14T05:31:22.188922+0000","last_fullsized":"2024-09-14T05:31:22.188922+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:18:52.725886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001656189,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3149,"num_read_kb":11342,"num_write":316,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"113'340","reported_seq":9918,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521406+0000","last_change":"2024-09-14T05:31:03.023758+0000","last_active":"2024-09-14T05:31:21.521406+0000","last_peered":"2024-09-14T05:31:21.521406+0000","last_clean":"2024-09-14T05:31:21.521406+0000","last_became_active":"2024-09-14T05:29:24.013434+0000","last_became_peered":"2024-09-14T05:29:24.013434+0000","last_unstale":"2024-09-14T05:31:21.521406+0000","last_undegraded":"2024-09-14T05:31:21.521406+0000","last_fullsized":"2024-09-14T05:31:21.521406+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:44.069349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418475,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":9226,"num_read_kb":22920,"num_write":296,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"113'440","reported_seq":2749,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.188832+0000","last_change":"2024-09-14T05:31:03.068934+0000","last_active":"2024-09-14T05:31:22.188832+0000","last_peered":"2024-09-14T05:31:22.188832+0000","last_clean":"2024-09-14T05:31:22.188832+0000","last_became_active":"2024-09-14T05:29:24.014457+0000","last_became_peered":"2024-09-14T05:29:24.014457+0000","last_unstale":"2024-09-14T05:31:22.188832+0000","last_undegraded":"2024-09-14T05:31:22.188832+0000","last_fullsized":"2024-09-14T05:31:22.188832+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:01:09.992476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011032850000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2612,"num_read_kb":9349,"num_write":416,"num_write_kb":12875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"109'843","reported_seq":2591,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128823+0000","last_change":"2024-09-14T05:31:03.020119+0000","last_active":"2024-09-14T05:31:21.128823+0000","last_peered":"2024-09-14T05:31:21.128823+0000","last_clean":"2024-09-14T05:31:21.128823+0000","last_became_active":"2024-09-14T05:29:24.013266+0000","last_became_peered":"2024-09-14T05:29:24.013266+0000","last_unstale":"2024-09-14T05:31:21.128823+0000","last_undegraded":"2024-09-14T05:31:21.128823+0000","last_fullsized":"2024-09-14T05:31:21.128823+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:34:03.137226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087100800000000005,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":4368,"num_read_kb":5632,"num_write":1075,"num_write_kb":4026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":241,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128857+0000","last_change":"2024-09-14T05:29:15.849668+0000","last_active":"2024-09-14T05:31:21.128857+0000","last_peered":"2024-09-14T05:31:21.128857+0000","last_clean":"2024-09-14T05:31:21.128857+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:31:21.128857+0000","last_undegraded":"2024-09-14T05:31:21.128857+0000","last_fullsized":"2024-09-14T05:31:21.128857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:33.000816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"114'375","reported_seq":2319,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.131016+0000","last_change":"2024-09-14T05:31:03.020581+0000","last_active":"2024-09-14T05:31:21.131016+0000","last_peered":"2024-09-14T05:31:21.131016+0000","last_clean":"2024-09-14T05:31:21.131016+0000","last_became_active":"2024-09-14T05:29:24.012099+0000","last_became_peered":"2024-09-14T05:29:24.012099+0000","last_unstale":"2024-09-14T05:31:21.131016+0000","last_undegraded":"2024-09-14T05:31:21.131016+0000","last_fullsized":"2024-09-14T05:31:21.131016+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:44:38.747225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013882689999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2144,"num_read_kb":3725,"num_write":284,"num_write_kb":5458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":236,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.131085+0000","last_change":"2024-09-14T05:29:15.852000+0000","last_active":"2024-09-14T05:31:21.131085+0000","last_peered":"2024-09-14T05:31:21.131085+0000","last_clean":"2024-09-14T05:31:21.131085+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:31:21.131085+0000","last_undegraded":"2024-09-14T05:31:21.131085+0000","last_fullsized":"2024-09-14T05:31:21.131085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:50.841256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"114'951","reported_seq":4046,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521397+0000","last_change":"2024-09-14T05:31:03.023762+0000","last_active":"2024-09-14T05:31:21.521397+0000","last_peered":"2024-09-14T05:31:21.521397+0000","last_clean":"2024-09-14T05:31:21.521397+0000","last_became_active":"2024-09-14T05:30:31.398064+0000","last_became_peered":"2024-09-14T05:30:31.398064+0000","last_unstale":"2024-09-14T05:31:21.521397+0000","last_undegraded":"2024-09-14T05:31:21.521397+0000","last_fullsized":"2024-09-14T05:31:21.521397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:15:44.039992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014417360000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6829,"num_read_kb":8465,"num_write":1096,"num_write_kb":5167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":648894,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128980+0000","last_change":"2024-09-14T05:29:15.850751+0000","last_active":"2024-09-14T05:31:21.128980+0000","last_peered":"2024-09-14T05:31:21.128980+0000","last_clean":"2024-09-14T05:31:21.128980+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:31:21.128980+0000","last_undegraded":"2024-09-14T05:31:21.128980+0000","last_fullsized":"2024-09-14T05:31:21.128980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:39.496842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"112'569","reported_seq":2718,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130208+0000","last_change":"2024-09-14T05:31:20.123367+0000","last_active":"2024-09-14T05:31:21.130208+0000","last_peered":"2024-09-14T05:31:21.130208+0000","last_clean":"2024-09-14T05:31:21.130208+0000","last_became_active":"2024-09-14T05:31:20.122139+0000","last_became_peered":"2024-09-14T05:31:20.122139+0000","last_unstale":"2024-09-14T05:31:21.130208+0000","last_undegraded":"2024-09-14T05:31:21.130208+0000","last_fullsized":"2024-09-14T05:31:21.130208+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:03:12.206743+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0050460000000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3487,"num_read_kb":12454,"num_write":614,"num_write_kb":11389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":236,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.188647+0000","last_change":"2024-09-14T05:29:15.994953+0000","last_active":"2024-09-14T05:31:22.188647+0000","last_peered":"2024-09-14T05:31:22.188647+0000","last_clean":"2024-09-14T05:31:22.188647+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:31:22.188647+0000","last_undegraded":"2024-09-14T05:31:22.188647+0000","last_fullsized":"2024-09-14T05:31:22.188647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:31.839292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"114'351","reported_seq":1951,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521367+0000","last_change":"2024-09-14T05:31:03.024292+0000","last_active":"2024-09-14T05:31:21.521367+0000","last_peered":"2024-09-14T05:31:21.521367+0000","last_clean":"2024-09-14T05:31:21.521367+0000","last_became_active":"2024-09-14T05:29:24.016940+0000","last_became_peered":"2024-09-14T05:29:24.016940+0000","last_unstale":"2024-09-14T05:31:21.521367+0000","last_undegraded":"2024-09-14T05:31:21.521367+0000","last_fullsized":"2024-09-14T05:31:21.521367+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:42:00.348036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001928208,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1597,"num_read_kb":4408,"num_write":335,"num_write_kb":4001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"114'642","reported_seq":3022,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189410+0000","last_change":"2024-09-14T05:31:03.069465+0000","last_active":"2024-09-14T05:31:22.189410+0000","last_peered":"2024-09-14T05:31:22.189410+0000","last_clean":"2024-09-14T05:31:22.189410+0000","last_became_active":"2024-09-14T05:29:24.015905+0000","last_became_peered":"2024-09-14T05:29:24.015905+0000","last_unstale":"2024-09-14T05:31:22.189410+0000","last_undegraded":"2024-09-14T05:31:22.189410+0000","last_fullsized":"2024-09-14T05:31:22.189410+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:31:28.697477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00154122,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":2532,"num_read_kb":9709,"num_write":637,"num_write_kb":7772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":243,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128727+0000","last_change":"2024-09-14T05:29:15.850287+0000","last_active":"2024-09-14T05:31:21.128727+0000","last_peered":"2024-09-14T05:31:21.128727+0000","last_clean":"2024-09-14T05:31:21.128727+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:31:21.128727+0000","last_undegraded":"2024-09-14T05:31:21.128727+0000","last_fullsized":"2024-09-14T05:31:21.128727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:04:13.546670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"113'485","reported_seq":10966,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189025+0000","last_change":"2024-09-14T05:31:03.026777+0000","last_active":"2024-09-14T05:31:22.189025+0000","last_peered":"2024-09-14T05:31:22.189025+0000","last_clean":"2024-09-14T05:31:22.189025+0000","last_became_active":"2024-09-14T05:29:24.011764+0000","last_became_peered":"2024-09-14T05:29:24.011764+0000","last_unstale":"2024-09-14T05:31:22.189025+0000","last_undegraded":"2024-09-14T05:31:22.189025+0000","last_fullsized":"2024-09-14T05:31:22.189025+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:07.795205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057339999999999995,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":11805,"num_read_kb":38991,"num_write":413,"num_write_kb":4959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":239,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.188709+0000","last_change":"2024-09-14T05:29:15.994837+0000","last_active":"2024-09-14T05:31:22.188709+0000","last_peered":"2024-09-14T05:31:22.188709+0000","last_clean":"2024-09-14T05:31:22.188709+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:31:22.188709+0000","last_undegraded":"2024-09-14T05:31:22.188709+0000","last_fullsized":"2024-09-14T05:31:22.188709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:59:41.913844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"110'701","reported_seq":1977,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130208+0000","last_change":"2024-09-14T05:31:20.123603+0000","last_active":"2024-09-14T05:31:21.130208+0000","last_peered":"2024-09-14T05:31:21.130208+0000","last_clean":"2024-09-14T05:31:21.130208+0000","last_became_active":"2024-09-14T05:31:20.122523+0000","last_became_peered":"2024-09-14T05:31:20.122523+0000","last_unstale":"2024-09-14T05:31:21.130208+0000","last_undegraded":"2024-09-14T05:31:21.130208+0000","last_fullsized":"2024-09-14T05:31:21.130208+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:44:47.461984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022421440000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3140,"num_read_kb":7266,"num_write":925,"num_write_kb":10032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":241,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128547+0000","last_change":"2024-09-14T05:29:15.850350+0000","last_active":"2024-09-14T05:31:21.128547+0000","last_peered":"2024-09-14T05:31:21.128547+0000","last_clean":"2024-09-14T05:31:21.128547+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:31:21.128547+0000","last_undegraded":"2024-09-14T05:31:21.128547+0000","last_fullsized":"2024-09-14T05:31:21.128547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:45.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"113'585","reported_seq":2767,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130931+0000","last_change":"2024-09-14T05:31:03.020204+0000","last_active":"2024-09-14T05:31:21.130931+0000","last_peered":"2024-09-14T05:31:21.130931+0000","last_clean":"2024-09-14T05:31:21.130931+0000","last_became_active":"2024-09-14T05:29:24.015792+0000","last_became_peered":"2024-09-14T05:29:24.015792+0000","last_unstale":"2024-09-14T05:31:21.130931+0000","last_undegraded":"2024-09-14T05:31:21.130931+0000","last_fullsized":"2024-09-14T05:31:21.130931+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:12:45.477846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011628109999999999,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":3532,"num_read_kb":9462,"num_write":620,"num_write_kb":5631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":242,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128830+0000","last_change":"2024-09-14T05:29:15.850582+0000","last_active":"2024-09-14T05:31:21.128830+0000","last_peered":"2024-09-14T05:31:21.128830+0000","last_clean":"2024-09-14T05:31:21.128830+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:31:21.128830+0000","last_undegraded":"2024-09-14T05:31:21.128830+0000","last_fullsized":"2024-09-14T05:31:21.128830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:12.817468+0000","last_clean_scrub_stamp":"2024-09-14T05:29:12.817468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:39:37.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":329,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189177+0000","last_change":"2024-09-14T05:29:08.021654+0000","last_active":"2024-09-14T05:31:22.189177+0000","last_peered":"2024-09-14T05:31:22.189177+0000","last_clean":"2024-09-14T05:31:22.189177+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:31:22.189177+0000","last_undegraded":"2024-09-14T05:31:22.189177+0000","last_fullsized":"2024-09-14T05:31:22.189177+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:06.755531+0000","last_clean_scrub_stamp":"2024-09-14T05:29:06.755531+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:02.114684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"113'395","reported_seq":2290,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.130890+0000","last_change":"2024-09-14T05:31:03.019738+0000","last_active":"2024-09-14T05:31:21.130890+0000","last_peered":"2024-09-14T05:31:21.130890+0000","last_clean":"2024-09-14T05:31:21.130890+0000","last_became_active":"2024-09-14T05:29:24.012222+0000","last_became_peered":"2024-09-14T05:29:24.012222+0000","last_unstale":"2024-09-14T05:31:21.130890+0000","last_undegraded":"2024-09-14T05:31:21.130890+0000","last_fullsized":"2024-09-14T05:31:21.130890+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:52:56.721362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072525400000000005,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2833,"num_read_kb":6029,"num_write":327,"num_write_kb":3878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"113'608","reported_seq":2440,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128564+0000","last_change":"2024-09-14T05:31:03.020704+0000","last_active":"2024-09-14T05:31:21.128564+0000","last_peered":"2024-09-14T05:31:21.128564+0000","last_clean":"2024-09-14T05:31:21.128564+0000","last_became_active":"2024-09-14T05:29:24.012041+0000","last_became_peered":"2024-09-14T05:29:24.012041+0000","last_unstale":"2024-09-14T05:31:21.128564+0000","last_undegraded":"2024-09-14T05:31:21.128564+0000","last_fullsized":"2024-09-14T05:31:21.128564+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:42:01.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001433679,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2914,"num_read_kb":7354,"num_write":748,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"113'655","reported_seq":2691,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128609+0000","last_change":"2024-09-14T05:31:03.021329+0000","last_active":"2024-09-14T05:31:21.128609+0000","last_peered":"2024-09-14T05:31:21.128609+0000","last_clean":"2024-09-14T05:31:21.128609+0000","last_became_active":"2024-09-14T05:29:24.011403+0000","last_became_peered":"2024-09-14T05:29:24.011403+0000","last_unstale":"2024-09-14T05:31:21.128609+0000","last_undegraded":"2024-09-14T05:31:21.128609+0000","last_fullsized":"2024-09-14T05:31:21.128609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:04:32.706037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00168743,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":3956,"num_read_kb":8037,"num_write":714,"num_write_kb":4980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"112'632","reported_seq":10269,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.188743+0000","last_change":"2024-09-14T05:31:03.032462+0000","last_active":"2024-09-14T05:31:22.188743+0000","last_peered":"2024-09-14T05:31:22.188743+0000","last_clean":"2024-09-14T05:31:22.188743+0000","last_became_active":"2024-09-14T05:29:24.006836+0000","last_became_peered":"2024-09-14T05:29:24.006836+0000","last_unstale":"2024-09-14T05:31:22.188743+0000","last_undegraded":"2024-09-14T05:31:22.188743+0000","last_fullsized":"2024-09-14T05:31:22.188743+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:22:08.301740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060523399999999996,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":12123,"num_read_kb":29901,"num_write":689,"num_write_kb":13072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"113'448","reported_seq":3554,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189014+0000","last_change":"2024-09-14T05:31:03.068410+0000","last_active":"2024-09-14T05:31:22.189014+0000","last_peered":"2024-09-14T05:31:22.189014+0000","last_clean":"2024-09-14T05:31:22.189014+0000","last_became_active":"2024-09-14T05:29:24.012574+0000","last_became_peered":"2024-09-14T05:29:24.012574+0000","last_unstale":"2024-09-14T05:31:22.189014+0000","last_undegraded":"2024-09-14T05:31:22.189014+0000","last_fullsized":"2024-09-14T05:31:22.189014+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:58:16.204790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064954900000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2817,"num_read_kb":13317,"num_write":361,"num_write_kb":9799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"112'1131","reported_seq":11394,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128943+0000","last_change":"2024-09-14T05:31:03.021253+0000","last_active":"2024-09-14T05:31:21.128943+0000","last_peered":"2024-09-14T05:31:21.128943+0000","last_clean":"2024-09-14T05:31:21.128943+0000","last_became_active":"2024-09-14T05:29:24.012401+0000","last_became_peered":"2024-09-14T05:29:24.012401+0000","last_unstale":"2024-09-14T05:31:21.128943+0000","last_undegraded":"2024-09-14T05:31:21.128943+0000","last_fullsized":"2024-09-14T05:31:21.128943+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":1131,"log_dups_size":0,"ondisk_log_size":1131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:18:41.728775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018223110000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":14909,"num_read_kb":22344,"num_write":1338,"num_write_kb":6495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"113'367","reported_seq":2862,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.521474+0000","last_change":"2024-09-14T05:31:03.024269+0000","last_active":"2024-09-14T05:31:21.521474+0000","last_peered":"2024-09-14T05:31:21.521474+0000","last_clean":"2024-09-14T05:31:21.521474+0000","last_became_active":"2024-09-14T05:29:24.015315+0000","last_became_peered":"2024-09-14T05:29:24.015315+0000","last_unstale":"2024-09-14T05:31:21.521474+0000","last_undegraded":"2024-09-14T05:31:21.521474+0000","last_fullsized":"2024-09-14T05:31:21.521474+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:27:50.755671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001918957,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2150,"num_read_kb":8871,"num_write":290,"num_write_kb":3939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"111'532","reported_seq":11844,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189148+0000","last_change":"2024-09-14T05:31:03.068658+0000","last_active":"2024-09-14T05:31:22.189148+0000","last_peered":"2024-09-14T05:31:22.189148+0000","last_clean":"2024-09-14T05:31:22.189148+0000","last_became_active":"2024-09-14T05:29:24.018854+0000","last_became_peered":"2024-09-14T05:29:24.018854+0000","last_unstale":"2024-09-14T05:31:22.189148+0000","last_undegraded":"2024-09-14T05:31:22.189148+0000","last_fullsized":"2024-09-14T05:31:22.189148+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:12:27.055181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000829428,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":12059,"num_read_kb":31916,"num_write":436,"num_write_kb":11843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"113'389","reported_seq":3041,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.189082+0000","last_change":"2024-09-14T05:31:03.034867+0000","last_active":"2024-09-14T05:31:22.189082+0000","last_peered":"2024-09-14T05:31:22.189082+0000","last_clean":"2024-09-14T05:31:22.189082+0000","last_became_active":"2024-09-14T05:29:24.010897+0000","last_became_peered":"2024-09-14T05:29:24.010897+0000","last_unstale":"2024-09-14T05:31:22.189082+0000","last_undegraded":"2024-09-14T05:31:22.189082+0000","last_fullsized":"2024-09-14T05:31:22.189082+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:33:58.717977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051683800000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2374,"num_read_kb":22456,"num_write":338,"num_write_kb":8939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"112'576","reported_seq":4039,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128547+0000","last_change":"2024-09-14T05:31:03.020915+0000","last_active":"2024-09-14T05:31:21.128547+0000","last_peered":"2024-09-14T05:31:21.128547+0000","last_clean":"2024-09-14T05:31:21.128547+0000","last_became_active":"2024-09-14T05:29:24.013792+0000","last_became_peered":"2024-09-14T05:29:24.013792+0000","last_unstale":"2024-09-14T05:31:21.128547+0000","last_undegraded":"2024-09-14T05:31:21.128547+0000","last_fullsized":"2024-09-14T05:31:21.128547+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:09:14.825887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001768403,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5197,"num_read_kb":24484,"num_write":526,"num_write_kb":3336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"112'490","reported_seq":2634,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:22.188911+0000","last_change":"2024-09-14T05:31:03.032142+0000","last_active":"2024-09-14T05:31:22.188911+0000","last_peered":"2024-09-14T05:31:22.188911+0000","last_clean":"2024-09-14T05:31:22.188911+0000","last_became_active":"2024-09-14T05:30:38.013627+0000","last_became_peered":"2024-09-14T05:30:38.013627+0000","last_unstale":"2024-09-14T05:31:22.188911+0000","last_undegraded":"2024-09-14T05:31:22.188911+0000","last_fullsized":"2024-09-14T05:31:22.188911+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:55:09.154072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050464100000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3016,"num_read_kb":16678,"num_write":502,"num_write_kb":10866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2769134,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"113'680","reported_seq":2493,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:21.128721+0000","last_change":"2024-09-14T05:31:03.019963+0000","last_active":"2024-09-14T05:31:21.128721+0000","last_peered":"2024-09-14T05:31:21.128721+0000","last_clean":"2024-09-14T05:31:21.128721+0000","last_became_active":"2024-09-14T05:29:24.014371+0000","last_became_peered":"2024-09-14T05:29:24.014371+0000","last_unstale":"2024-09-14T05:31:21.128721+0000","last_undegraded":"2024-09-14T05:31:21.128721+0000","last_fullsized":"2024-09-14T05:31:21.128721+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:29:22.995117+0000","last_clean_scrub_stamp":"2024-09-14T05:29:22.995117+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:06.352481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080910499999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3420,"num_read_kb":19916,"num_write":912,"num_write_kb":5503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":594,"num_object_clones":0,"num_object_copies":1188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":594,"num_whiteouts":590,"num_read":182740,"num_read_kb":565764,"num_write":20809,"num_write_kb":280821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19248,"ondisk_log_size":19248,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476769,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50776,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321064,"statfs":{"total":96636764160,"available":96584769536,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509472,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41044,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330796,"statfs":{"total":96636764160,"available":96594735104,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509473,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28576,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343264,"statfs":{"total":96636764160,"available":96607502336,"internally_reserved":0,"allocated":1576960,"data_stored":747924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509471,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2170880,"data_stored":1338292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509472,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28568,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343272,"statfs":{"total":96636764160,"available":96607510528,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509472,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72988,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298852,"statfs":{"total":96636764160,"available":96562024448,"internally_reserved":0,"allocated":1576960,"data_stored":747924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509474,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51364,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320476,"statfs":{"total":96636764160,"available":96584167424,"internally_reserved":0,"allocated":2170880,"data_stored":1338292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:31:31.254 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.255 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.255 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.255 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.255 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.255 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.256 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.256 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.256 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.256 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.257 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.257 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.257 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-14T05:29:12.817468+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=12, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.257 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.257 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-14T05:29:12.817468+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=12, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.257 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.258 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-14T05:29:12.817468+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=12, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.258 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.258 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-14T05:29:12.817468+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=12, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.258 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.258 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.258 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-14T05:29:12.817468+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=12, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.258 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.258 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-14T05:29:12.817468+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=12, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.259 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.259 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-14T05:29:12.817468+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=12, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.259 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.259 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-14T05:29:12.817468+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=12, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.259 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-14T05:29:06.755531+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=6, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.259 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.259 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.259 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.259 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.260 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.260 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.260 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.260 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.260 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.260 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.260 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.260 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-14T05:29:22.995117+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=29, tm_sec=22, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=5, tm_min=31, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T05:31:31.261 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-14T05:31:51.262 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T05:31:51.559 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-14T05:31:51.560 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-14T05:31:51.571 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-09-14T05:31:50.463784+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":598,"num_object_clones":0,"num_object_copies":1196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":598,"num_whiteouts":590,"num_read":182806,"num_read_kb":565819,"num_write":20912,"num_write_kb":282451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19919,"ondisk_log_size":19919,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":347608,"kb_used_data":13464,"kb_used_omap":12,"kb_used_meta":334003,"kb_avail":754627112,"statfs":{"total":773094113280,"available":772738162688,"internally_reserved":0,"allocated":13787136,"data_stored":7164052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":342019632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002697"},"pg_stats":[{"pgid":"3.1f","version":"132'788","reported_seq":11009,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:35.663246+0000","last_change":"2024-09-14T05:31:35.663246+0000","last_active":"2024-09-14T05:31:35.663246+0000","last_peered":"2024-09-14T05:31:35.663246+0000","last_clean":"2024-09-14T05:31:35.663246+0000","last_became_active":"2024-09-14T05:29:24.018892+0000","last_became_peered":"2024-09-14T05:29:24.018892+0000","last_unstale":"2024-09-14T05:31:35.663246+0000","last_undegraded":"2024-09-14T05:31:35.663246+0000","last_fullsized":"2024-09-14T05:31:35.663246+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'788","last_scrub_stamp":"2024-09-14T05:31:35.663151+0000","last_deep_scrub":"132'788","last_deep_scrub_stamp":"2024-09-14T05:31:35.663151+0000","last_clean_scrub_stamp":"2024-09-14T05:31:35.663151+0000","objects_scrubbed":21,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:39:01.093349+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0013315950000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":12275,"num_read_kb":37400,"num_write":832,"num_write_kb":31155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"132'432","reported_seq":3335,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:34.666046+0000","last_change":"2024-09-14T05:31:34.666046+0000","last_active":"2024-09-14T05:31:34.666046+0000","last_peered":"2024-09-14T05:31:34.666046+0000","last_clean":"2024-09-14T05:31:34.666046+0000","last_became_active":"2024-09-14T05:29:24.014468+0000","last_became_peered":"2024-09-14T05:29:24.014468+0000","last_unstale":"2024-09-14T05:31:34.666046+0000","last_undegraded":"2024-09-14T05:31:34.666046+0000","last_fullsized":"2024-09-14T05:31:34.666046+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'432","last_scrub_stamp":"2024-09-14T05:31:34.665921+0000","last_deep_scrub":"132'432","last_deep_scrub_stamp":"2024-09-14T05:31:34.665921+0000","last_clean_scrub_stamp":"2024-09-14T05:31:34.665921+0000","objects_scrubbed":14,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:22:07.014704+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001315826,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2805,"num_read_kb":32498,"num_write":409,"num_write_kb":21098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"132'1059","reported_seq":9904,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:40.210187+0000","last_change":"2024-09-14T05:31:40.210187+0000","last_active":"2024-09-14T05:31:40.210187+0000","last_peered":"2024-09-14T05:31:40.210187+0000","last_clean":"2024-09-14T05:31:40.210187+0000","last_became_active":"2024-09-14T05:29:24.014251+0000","last_became_peered":"2024-09-14T05:29:24.014251+0000","last_unstale":"2024-09-14T05:31:40.210187+0000","last_undegraded":"2024-09-14T05:31:40.210187+0000","last_fullsized":"2024-09-14T05:31:40.210187+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'1059","last_scrub_stamp":"2024-09-14T05:31:40.210101+0000","last_deep_scrub":"132'1059","last_deep_scrub_stamp":"2024-09-14T05:31:40.210101+0000","last_clean_scrub_stamp":"2024-09-14T05:31:40.210101+0000","objects_scrubbed":12,"log_size":1059,"log_dups_size":0,"ondisk_log_size":1059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:23:13.081520+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.0012210070000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12253,"num_read_kb":23814,"num_write":1553,"num_write_kb":7077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"132'624","reported_seq":10620,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:32.117292+0000","last_change":"2024-09-14T05:31:32.117292+0000","last_active":"2024-09-14T05:31:32.117292+0000","last_peered":"2024-09-14T05:31:32.117292+0000","last_clean":"2024-09-14T05:31:32.117292+0000","last_became_active":"2024-09-14T05:29:24.012427+0000","last_became_peered":"2024-09-14T05:29:24.012427+0000","last_unstale":"2024-09-14T05:31:32.117292+0000","last_undegraded":"2024-09-14T05:31:32.117292+0000","last_fullsized":"2024-09-14T05:31:32.117292+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'624","last_scrub_stamp":"2024-09-14T05:31:32.117177+0000","last_deep_scrub":"132'624","last_deep_scrub_stamp":"2024-09-14T05:31:32.117177+0000","last_clean_scrub_stamp":"2024-09-14T05:31:32.117177+0000","objects_scrubbed":23,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:33:10.576299+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.00073664900000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":10982,"num_read_kb":35239,"num_write":569,"num_write_kb":4638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"132'606","reported_seq":3693,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:34.549431+0000","last_change":"2024-09-14T05:31:34.549431+0000","last_active":"2024-09-14T05:31:34.549431+0000","last_peered":"2024-09-14T05:31:34.549431+0000","last_clean":"2024-09-14T05:31:34.549431+0000","last_became_active":"2024-09-14T05:29:24.012079+0000","last_became_peered":"2024-09-14T05:29:24.012079+0000","last_unstale":"2024-09-14T05:31:34.549431+0000","last_undegraded":"2024-09-14T05:31:34.549431+0000","last_fullsized":"2024-09-14T05:31:34.549431+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'606","last_scrub_stamp":"2024-09-14T05:31:34.549326+0000","last_deep_scrub":"132'606","last_deep_scrub_stamp":"2024-09-14T05:31:34.549326+0000","last_clean_scrub_stamp":"2024-09-14T05:31:34.549326+0000","objects_scrubbed":19,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:03:50.670241+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.0010699310000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4414,"num_read_kb":26220,"num_write":576,"num_write_kb":21867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"132'1026","reported_seq":6530,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:31.093309+0000","last_change":"2024-09-14T05:31:31.093309+0000","last_active":"2024-09-14T05:31:31.093309+0000","last_peered":"2024-09-14T05:31:31.093309+0000","last_clean":"2024-09-14T05:31:31.093309+0000","last_became_active":"2024-09-14T05:29:24.012461+0000","last_became_peered":"2024-09-14T05:29:24.012461+0000","last_unstale":"2024-09-14T05:31:31.093309+0000","last_undegraded":"2024-09-14T05:31:31.093309+0000","last_fullsized":"2024-09-14T05:31:31.093309+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'1026","last_scrub_stamp":"2024-09-14T05:31:31.093113+0000","last_deep_scrub":"132'1026","last_deep_scrub_stamp":"2024-09-14T05:31:31.093113+0000","last_clean_scrub_stamp":"2024-09-14T05:31:31.093113+0000","objects_scrubbed":21,"log_size":1026,"log_dups_size":0,"ondisk_log_size":1026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:34.707681+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.00090417700000000004,"stat_sum":{"num_bytes":19,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":10524,"num_read_kb":34488,"num_write":1274,"num_write_kb":8459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"132'600","reported_seq":2264,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:33.571236+0000","last_change":"2024-09-14T05:31:33.571236+0000","last_active":"2024-09-14T05:31:33.571236+0000","last_peered":"2024-09-14T05:31:33.571236+0000","last_clean":"2024-09-14T05:31:33.571236+0000","last_became_active":"2024-09-14T05:29:24.018753+0000","last_became_peered":"2024-09-14T05:29:24.018753+0000","last_unstale":"2024-09-14T05:31:33.571236+0000","last_undegraded":"2024-09-14T05:31:33.571236+0000","last_fullsized":"2024-09-14T05:31:33.571236+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'600","last_scrub_stamp":"2024-09-14T05:31:33.571135+0000","last_deep_scrub":"132'600","last_deep_scrub_stamp":"2024-09-14T05:31:33.571135+0000","last_clean_scrub_stamp":"2024-09-14T05:31:33.571135+0000","objects_scrubbed":19,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:38:12.986762+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.00074058200000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2590,"num_read_kb":13584,"num_write":688,"num_write_kb":9379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"132'715","reported_seq":2963,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:33.643235+0000","last_change":"2024-09-14T05:31:33.643235+0000","last_active":"2024-09-14T05:31:33.643235+0000","last_peered":"2024-09-14T05:31:33.643235+0000","last_clean":"2024-09-14T05:31:33.643235+0000","last_became_active":"2024-09-14T05:29:24.011176+0000","last_became_peered":"2024-09-14T05:29:24.011176+0000","last_unstale":"2024-09-14T05:31:33.643235+0000","last_undegraded":"2024-09-14T05:31:33.643235+0000","last_fullsized":"2024-09-14T05:31:33.643235+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'715","last_scrub_stamp":"2024-09-14T05:31:33.643143+0000","last_deep_scrub":"132'715","last_deep_scrub_stamp":"2024-09-14T05:31:33.643143+0000","last_clean_scrub_stamp":"2024-09-14T05:31:33.643143+0000","objects_scrubbed":16,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:15:04.670271+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00091515699999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4708,"num_read_kb":7495,"num_write":700,"num_write_kb":3426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"132'371","reported_seq":3200,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:31.593676+0000","last_change":"2024-09-14T05:31:31.593676+0000","last_active":"2024-09-14T05:31:31.593676+0000","last_peered":"2024-09-14T05:31:31.593676+0000","last_clean":"2024-09-14T05:31:31.593676+0000","last_became_active":"2024-09-14T05:29:24.014005+0000","last_became_peered":"2024-09-14T05:29:24.014005+0000","last_unstale":"2024-09-14T05:31:31.593676+0000","last_undegraded":"2024-09-14T05:31:31.593676+0000","last_fullsized":"2024-09-14T05:31:31.593676+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'371","last_scrub_stamp":"2024-09-14T05:31:31.593568+0000","last_deep_scrub":"132'371","last_deep_scrub_stamp":"2024-09-14T05:31:31.593568+0000","last_clean_scrub_stamp":"2024-09-14T05:31:31.593568+0000","objects_scrubbed":14,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:49:21.973750+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.001656189,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3149,"num_read_kb":11342,"num_write":316,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"132'362","reported_seq":9948,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:31.896962+0000","last_change":"2024-09-14T05:31:31.896962+0000","last_active":"2024-09-14T05:31:31.896962+0000","last_peered":"2024-09-14T05:31:31.896962+0000","last_clean":"2024-09-14T05:31:31.896962+0000","last_became_active":"2024-09-14T05:29:24.013434+0000","last_became_peered":"2024-09-14T05:29:24.013434+0000","last_unstale":"2024-09-14T05:31:31.896962+0000","last_undegraded":"2024-09-14T05:31:31.896962+0000","last_fullsized":"2024-09-14T05:31:31.896962+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'362","last_scrub_stamp":"2024-09-14T05:31:31.896854+0000","last_deep_scrub":"132'362","last_deep_scrub_stamp":"2024-09-14T05:31:31.896854+0000","last_clean_scrub_stamp":"2024-09-14T05:31:31.896854+0000","objects_scrubbed":22,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:13:44.857163+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.001418475,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":9226,"num_read_kb":22920,"num_write":296,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"132'453","reported_seq":2770,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:31.215819+0000","last_change":"2024-09-14T05:31:31.215819+0000","last_active":"2024-09-14T05:31:31.215819+0000","last_peered":"2024-09-14T05:31:31.215819+0000","last_clean":"2024-09-14T05:31:31.215819+0000","last_became_active":"2024-09-14T05:29:24.014457+0000","last_became_peered":"2024-09-14T05:29:24.014457+0000","last_unstale":"2024-09-14T05:31:31.215819+0000","last_undegraded":"2024-09-14T05:31:31.215819+0000","last_fullsized":"2024-09-14T05:31:31.215819+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'453","last_scrub_stamp":"2024-09-14T05:31:31.215750+0000","last_deep_scrub":"132'453","last_deep_scrub_stamp":"2024-09-14T05:31:31.215750+0000","last_clean_scrub_stamp":"2024-09-14T05:31:31.215750+0000","objects_scrubbed":13,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:44:31.825676+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0011032850000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":2612,"num_read_kb":9349,"num_write":416,"num_write_kb":12875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"132'869","reported_seq":2627,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:38.250193+0000","last_change":"2024-09-14T05:31:38.250122+0000","last_active":"2024-09-14T05:31:38.250193+0000","last_peered":"2024-09-14T05:31:38.250193+0000","last_clean":"2024-09-14T05:31:38.250193+0000","last_became_active":"2024-09-14T05:29:24.013266+0000","last_became_peered":"2024-09-14T05:29:24.013266+0000","last_unstale":"2024-09-14T05:31:38.250193+0000","last_undegraded":"2024-09-14T05:31:38.250193+0000","last_fullsized":"2024-09-14T05:31:38.250193+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'869","last_scrub_stamp":"2024-09-14T05:31:38.250037+0000","last_deep_scrub":"132'869","last_deep_scrub_stamp":"2024-09-14T05:31:38.250037+0000","last_clean_scrub_stamp":"2024-09-14T05:31:38.250037+0000","objects_scrubbed":26,"log_size":869,"log_dups_size":0,"ondisk_log_size":869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:29:12.427871+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0.00087100800000000005,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":4368,"num_read_kb":5632,"num_write":1075,"num_write_kb":4026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":249,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:34.145443+0000","last_change":"2024-09-14T05:31:34.145443+0000","last_active":"2024-09-14T05:31:34.145443+0000","last_peered":"2024-09-14T05:31:34.145443+0000","last_clean":"2024-09-14T05:31:34.145443+0000","last_became_active":"2024-09-14T05:29:14.413374+0000","last_became_peered":"2024-09-14T05:29:14.413374+0000","last_unstale":"2024-09-14T05:31:34.145443+0000","last_undegraded":"2024-09-14T05:31:34.145443+0000","last_fullsized":"2024-09-14T05:31:34.145443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:31:34.145352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:31:34.145352+0000","last_clean_scrub_stamp":"2024-09-14T05:31:34.145352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:27.853575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000431612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"132'390","reported_seq":2343,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:35.513995+0000","last_change":"2024-09-14T05:31:35.513933+0000","last_active":"2024-09-14T05:31:35.513995+0000","last_peered":"2024-09-14T05:31:35.513995+0000","last_clean":"2024-09-14T05:31:35.513995+0000","last_became_active":"2024-09-14T05:29:24.012099+0000","last_became_peered":"2024-09-14T05:29:24.012099+0000","last_unstale":"2024-09-14T05:31:35.513995+0000","last_undegraded":"2024-09-14T05:31:35.513995+0000","last_fullsized":"2024-09-14T05:31:35.513995+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'390","last_scrub_stamp":"2024-09-14T05:31:35.513803+0000","last_deep_scrub":"132'390","last_deep_scrub_stamp":"2024-09-14T05:31:35.513803+0000","last_clean_scrub_stamp":"2024-09-14T05:31:35.513803+0000","objects_scrubbed":15,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:31:06.510346+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0013882689999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2144,"num_read_kb":3725,"num_write":284,"num_write_kb":5458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":244,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:30.590962+0000","last_change":"2024-09-14T05:31:30.590962+0000","last_active":"2024-09-14T05:31:30.590962+0000","last_peered":"2024-09-14T05:31:30.590962+0000","last_clean":"2024-09-14T05:31:30.590962+0000","last_became_active":"2024-09-14T05:29:14.413262+0000","last_became_peered":"2024-09-14T05:29:14.413262+0000","last_unstale":"2024-09-14T05:31:30.590962+0000","last_undegraded":"2024-09-14T05:31:30.590962+0000","last_fullsized":"2024-09-14T05:31:30.590962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:31:30.590873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:31:30.590873+0000","last_clean_scrub_stamp":"2024-09-14T05:31:30.590873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:55:34.282915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065455500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"132'969","reported_seq":4072,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:29.861030+0000","last_change":"2024-09-14T05:31:29.861030+0000","last_active":"2024-09-14T05:31:29.861030+0000","last_peered":"2024-09-14T05:31:29.861030+0000","last_clean":"2024-09-14T05:31:29.861030+0000","last_became_active":"2024-09-14T05:30:31.398064+0000","last_became_peered":"2024-09-14T05:30:31.398064+0000","last_unstale":"2024-09-14T05:31:29.861030+0000","last_undegraded":"2024-09-14T05:31:29.861030+0000","last_fullsized":"2024-09-14T05:31:29.861030+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'969","last_scrub_stamp":"2024-09-14T05:31:29.860932+0000","last_deep_scrub":"132'969","last_deep_scrub_stamp":"2024-09-14T05:31:29.860932+0000","last_clean_scrub_stamp":"2024-09-14T05:31:29.860932+0000","objects_scrubbed":18,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:31:13.598836+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0014417360000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6829,"num_read_kb":8465,"num_write":1096,"num_write_kb":5167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":648894,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":249,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:36.178900+0000","last_change":"2024-09-14T05:31:36.178900+0000","last_active":"2024-09-14T05:31:36.178900+0000","last_peered":"2024-09-14T05:31:36.178900+0000","last_clean":"2024-09-14T05:31:36.178900+0000","last_became_active":"2024-09-14T05:29:13.829617+0000","last_became_peered":"2024-09-14T05:29:13.829617+0000","last_unstale":"2024-09-14T05:31:36.178900+0000","last_undegraded":"2024-09-14T05:31:36.178900+0000","last_fullsized":"2024-09-14T05:31:36.178900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:31:36.178793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:31:36.178793+0000","last_clean_scrub_stamp":"2024-09-14T05:31:36.178793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:08:44.914675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458034,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"132'592","reported_seq":2750,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:31.920390+0000","last_change":"2024-09-14T05:31:31.920390+0000","last_active":"2024-09-14T05:31:31.920390+0000","last_peered":"2024-09-14T05:31:31.920390+0000","last_clean":"2024-09-14T05:31:31.920390+0000","last_became_active":"2024-09-14T05:31:20.122139+0000","last_became_peered":"2024-09-14T05:31:20.122139+0000","last_unstale":"2024-09-14T05:31:31.920390+0000","last_undegraded":"2024-09-14T05:31:31.920390+0000","last_fullsized":"2024-09-14T05:31:31.920390+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'592","last_scrub_stamp":"2024-09-14T05:31:31.920267+0000","last_deep_scrub":"132'592","last_deep_scrub_stamp":"2024-09-14T05:31:31.920267+0000","last_clean_scrub_stamp":"2024-09-14T05:31:31.920267+0000","objects_scrubbed":23,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:05:34.649245+0000","scrub_duration":18,"objects_trimmed":2,"snaptrim_duration":0.0050460000000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3487,"num_read_kb":12454,"num_write":614,"num_write_kb":11389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":244,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:30.627146+0000","last_change":"2024-09-14T05:31:30.627146+0000","last_active":"2024-09-14T05:31:30.627146+0000","last_peered":"2024-09-14T05:31:30.627146+0000","last_clean":"2024-09-14T05:31:30.627146+0000","last_became_active":"2024-09-14T05:29:13.968242+0000","last_became_peered":"2024-09-14T05:29:13.968242+0000","last_unstale":"2024-09-14T05:31:30.627146+0000","last_undegraded":"2024-09-14T05:31:30.627146+0000","last_fullsized":"2024-09-14T05:31:30.627146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:31:30.627052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:31:30.627052+0000","last_clean_scrub_stamp":"2024-09-14T05:31:30.627052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:44:45.624203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051226699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"132'367","reported_seq":1976,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:30.861181+0000","last_change":"2024-09-14T05:31:30.861128+0000","last_active":"2024-09-14T05:31:30.861181+0000","last_peered":"2024-09-14T05:31:30.861181+0000","last_clean":"2024-09-14T05:31:30.861181+0000","last_became_active":"2024-09-14T05:29:24.016940+0000","last_became_peered":"2024-09-14T05:29:24.016940+0000","last_unstale":"2024-09-14T05:31:30.861181+0000","last_undegraded":"2024-09-14T05:31:30.861181+0000","last_fullsized":"2024-09-14T05:31:30.861181+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'367","last_scrub_stamp":"2024-09-14T05:31:30.861052+0000","last_deep_scrub":"132'367","last_deep_scrub_stamp":"2024-09-14T05:31:30.861052+0000","last_clean_scrub_stamp":"2024-09-14T05:31:30.861052+0000","objects_scrubbed":16,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:14.723224+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.001928208,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1597,"num_read_kb":4408,"num_write":335,"num_write_kb":4001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"132'666","reported_seq":3056,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:28.313078+0000","last_change":"2024-09-14T05:31:28.312997+0000","last_active":"2024-09-14T05:31:28.313078+0000","last_peered":"2024-09-14T05:31:28.313078+0000","last_clean":"2024-09-14T05:31:28.313078+0000","last_became_active":"2024-09-14T05:29:24.015905+0000","last_became_peered":"2024-09-14T05:29:24.015905+0000","last_unstale":"2024-09-14T05:31:28.313078+0000","last_undegraded":"2024-09-14T05:31:28.313078+0000","last_fullsized":"2024-09-14T05:31:28.313078+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'666","last_scrub_stamp":"2024-09-14T05:31:28.312905+0000","last_deep_scrub":"132'666","last_deep_scrub_stamp":"2024-09-14T05:31:28.312905+0000","last_clean_scrub_stamp":"2024-09-14T05:31:28.312905+0000","objects_scrubbed":25,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:20:28.583461+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.00154122,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":2532,"num_read_kb":9709,"num_write":637,"num_write_kb":7772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"132'3","reported_seq":252,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:30.266761+0000","last_change":"2024-09-14T05:31:30.266761+0000","last_active":"2024-09-14T05:31:30.266761+0000","last_peered":"2024-09-14T05:31:30.266761+0000","last_clean":"2024-09-14T05:31:30.266761+0000","last_became_active":"2024-09-14T05:29:13.968376+0000","last_became_peered":"2024-09-14T05:29:13.968376+0000","last_unstale":"2024-09-14T05:31:30.266761+0000","last_undegraded":"2024-09-14T05:31:30.266761+0000","last_fullsized":"2024-09-14T05:31:30.266761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'3","last_scrub_stamp":"2024-09-14T05:31:30.266667+0000","last_deep_scrub":"132'3","last_deep_scrub_stamp":"2024-09-14T05:31:30.266667+0000","last_clean_scrub_stamp":"2024-09-14T05:31:30.266667+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:08:13.545783+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.001030244,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"132'503","reported_seq":10992,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:32.629357+0000","last_change":"2024-09-14T05:31:32.629357+0000","last_active":"2024-09-14T05:31:32.629357+0000","last_peered":"2024-09-14T05:31:32.629357+0000","last_clean":"2024-09-14T05:31:32.629357+0000","last_became_active":"2024-09-14T05:29:24.011764+0000","last_became_peered":"2024-09-14T05:29:24.011764+0000","last_unstale":"2024-09-14T05:31:32.629357+0000","last_undegraded":"2024-09-14T05:31:32.629357+0000","last_fullsized":"2024-09-14T05:31:32.629357+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'503","last_scrub_stamp":"2024-09-14T05:31:32.629256+0000","last_deep_scrub":"132'503","last_deep_scrub_stamp":"2024-09-14T05:31:32.629256+0000","last_clean_scrub_stamp":"2024-09-14T05:31:32.629256+0000","objects_scrubbed":18,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:50:40.118480+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.00057339999999999995,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":11805,"num_read_kb":38991,"num_write":413,"num_write_kb":4959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":247,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:32.151311+0000","last_change":"2024-09-14T05:31:32.151311+0000","last_active":"2024-09-14T05:31:32.151311+0000","last_peered":"2024-09-14T05:31:32.151311+0000","last_clean":"2024-09-14T05:31:32.151311+0000","last_became_active":"2024-09-14T05:29:13.834933+0000","last_became_peered":"2024-09-14T05:29:13.834933+0000","last_unstale":"2024-09-14T05:31:32.151311+0000","last_undegraded":"2024-09-14T05:31:32.151311+0000","last_fullsized":"2024-09-14T05:31:32.151311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:31:32.151185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:31:32.151185+0000","last_clean_scrub_stamp":"2024-09-14T05:31:32.151185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:51:09.895349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044814200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"132'717","reported_seq":2002,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:30.906961+0000","last_change":"2024-09-14T05:31:30.906892+0000","last_active":"2024-09-14T05:31:30.906961+0000","last_peered":"2024-09-14T05:31:30.906961+0000","last_clean":"2024-09-14T05:31:30.906961+0000","last_became_active":"2024-09-14T05:31:20.122523+0000","last_became_peered":"2024-09-14T05:31:20.122523+0000","last_unstale":"2024-09-14T05:31:30.906961+0000","last_undegraded":"2024-09-14T05:31:30.906961+0000","last_fullsized":"2024-09-14T05:31:30.906961+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'717","last_scrub_stamp":"2024-09-14T05:31:30.906780+0000","last_deep_scrub":"132'717","last_deep_scrub_stamp":"2024-09-14T05:31:30.906780+0000","last_clean_scrub_stamp":"2024-09-14T05:31:30.906780+0000","objects_scrubbed":16,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:49:46.165597+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0022421440000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3140,"num_read_kb":7266,"num_write":925,"num_write_kb":10032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":249,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:29.255807+0000","last_change":"2024-09-14T05:31:29.255807+0000","last_active":"2024-09-14T05:31:29.255807+0000","last_peered":"2024-09-14T05:31:29.255807+0000","last_clean":"2024-09-14T05:31:29.255807+0000","last_became_active":"2024-09-14T05:29:13.834987+0000","last_became_peered":"2024-09-14T05:29:13.834987+0000","last_unstale":"2024-09-14T05:31:29.255807+0000","last_undegraded":"2024-09-14T05:31:29.255807+0000","last_fullsized":"2024-09-14T05:31:29.255807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T05:31:29.255736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T05:31:29.255736+0000","last_clean_scrub_stamp":"2024-09-14T05:31:29.255736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:52:27.182500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00106529,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"132'611","reported_seq":2803,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:31.596862+0000","last_change":"2024-09-14T05:31:31.596862+0000","last_active":"2024-09-14T05:31:31.596862+0000","last_peered":"2024-09-14T05:31:31.596862+0000","last_clean":"2024-09-14T05:31:31.596862+0000","last_became_active":"2024-09-14T05:29:24.015792+0000","last_became_peered":"2024-09-14T05:29:24.015792+0000","last_unstale":"2024-09-14T05:31:31.596862+0000","last_undegraded":"2024-09-14T05:31:31.596862+0000","last_fullsized":"2024-09-14T05:31:31.596862+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'611","last_scrub_stamp":"2024-09-14T05:31:31.596773+0000","last_deep_scrub":"132'611","last_deep_scrub_stamp":"2024-09-14T05:31:31.596773+0000","last_clean_scrub_stamp":"2024-09-14T05:31:31.596773+0000","objects_scrubbed":26,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:16:48.366456+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0011628109999999999,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":3532,"num_read_kb":9462,"num_write":620,"num_write_kb":5631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"132'2","reported_seq":251,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:31.219464+0000","last_change":"2024-09-14T05:31:31.219464+0000","last_active":"2024-09-14T05:31:31.219464+0000","last_peered":"2024-09-14T05:31:31.219464+0000","last_clean":"2024-09-14T05:31:31.219464+0000","last_became_active":"2024-09-14T05:29:13.830307+0000","last_became_peered":"2024-09-14T05:29:13.830307+0000","last_unstale":"2024-09-14T05:31:31.219464+0000","last_undegraded":"2024-09-14T05:31:31.219464+0000","last_fullsized":"2024-09-14T05:31:31.219464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'2","last_scrub_stamp":"2024-09-14T05:31:31.219380+0000","last_deep_scrub":"132'2","last_deep_scrub_stamp":"2024-09-14T05:31:31.219380+0000","last_clean_scrub_stamp":"2024-09-14T05:31:31.219380+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:55:07.443105+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0013388009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"132'74","reported_seq":339,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:29.113635+0000","last_change":"2024-09-14T05:31:29.113635+0000","last_active":"2024-09-14T05:31:29.113635+0000","last_peered":"2024-09-14T05:31:29.113635+0000","last_clean":"2024-09-14T05:31:29.113635+0000","last_became_active":"2024-09-14T05:29:08.020620+0000","last_became_peered":"2024-09-14T05:29:08.020620+0000","last_unstale":"2024-09-14T05:31:29.113635+0000","last_undegraded":"2024-09-14T05:31:29.113635+0000","last_fullsized":"2024-09-14T05:31:29.113635+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'74","last_scrub_stamp":"2024-09-14T05:31:29.113568+0000","last_deep_scrub":"132'74","last_deep_scrub_stamp":"2024-09-14T05:31:29.113568+0000","last_clean_scrub_stamp":"2024-09-14T05:31:29.113568+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:06:52.356039+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"132'413","reported_seq":2317,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:32.552728+0000","last_change":"2024-09-14T05:31:32.552665+0000","last_active":"2024-09-14T05:31:32.552728+0000","last_peered":"2024-09-14T05:31:32.552728+0000","last_clean":"2024-09-14T05:31:32.552728+0000","last_became_active":"2024-09-14T05:29:24.012222+0000","last_became_peered":"2024-09-14T05:29:24.012222+0000","last_unstale":"2024-09-14T05:31:32.552728+0000","last_undegraded":"2024-09-14T05:31:32.552728+0000","last_fullsized":"2024-09-14T05:31:32.552728+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'413","last_scrub_stamp":"2024-09-14T05:31:32.552545+0000","last_deep_scrub":"132'413","last_deep_scrub_stamp":"2024-09-14T05:31:32.552545+0000","last_clean_scrub_stamp":"2024-09-14T05:31:32.552545+0000","objects_scrubbed":18,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:28:57.170662+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00072525400000000005,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2833,"num_read_kb":6029,"num_write":327,"num_write_kb":3878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"132'623","reported_seq":2464,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:33.162204+0000","last_change":"2024-09-14T05:31:33.162148+0000","last_active":"2024-09-14T05:31:33.162204+0000","last_peered":"2024-09-14T05:31:33.162204+0000","last_clean":"2024-09-14T05:31:33.162204+0000","last_became_active":"2024-09-14T05:29:24.012041+0000","last_became_peered":"2024-09-14T05:29:24.012041+0000","last_unstale":"2024-09-14T05:31:33.162204+0000","last_undegraded":"2024-09-14T05:31:33.162204+0000","last_fullsized":"2024-09-14T05:31:33.162204+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'623","last_scrub_stamp":"2024-09-14T05:31:33.162056+0000","last_deep_scrub":"132'623","last_deep_scrub_stamp":"2024-09-14T05:31:33.162056+0000","last_clean_scrub_stamp":"2024-09-14T05:31:33.162056+0000","objects_scrubbed":15,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:57:03.671736+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001433679,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2914,"num_read_kb":7354,"num_write":748,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"132'671","reported_seq":2716,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:32.205476+0000","last_change":"2024-09-14T05:31:32.205419+0000","last_active":"2024-09-14T05:31:32.205476+0000","last_peered":"2024-09-14T05:31:32.205476+0000","last_clean":"2024-09-14T05:31:32.205476+0000","last_became_active":"2024-09-14T05:29:24.011403+0000","last_became_peered":"2024-09-14T05:29:24.011403+0000","last_unstale":"2024-09-14T05:31:32.205476+0000","last_undegraded":"2024-09-14T05:31:32.205476+0000","last_fullsized":"2024-09-14T05:31:32.205476+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'671","last_scrub_stamp":"2024-09-14T05:31:32.205316+0000","last_deep_scrub":"132'671","last_deep_scrub_stamp":"2024-09-14T05:31:32.205316+0000","last_clean_scrub_stamp":"2024-09-14T05:31:32.205316+0000","objects_scrubbed":16,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:23:44.107998+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.00168743,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":3956,"num_read_kb":8037,"num_write":714,"num_write_kb":4980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"132'657","reported_seq":10304,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:31.202387+0000","last_change":"2024-09-14T05:31:31.202315+0000","last_active":"2024-09-14T05:31:31.202387+0000","last_peered":"2024-09-14T05:31:31.202387+0000","last_clean":"2024-09-14T05:31:31.202387+0000","last_became_active":"2024-09-14T05:29:24.006836+0000","last_became_peered":"2024-09-14T05:29:24.006836+0000","last_unstale":"2024-09-14T05:31:31.202387+0000","last_undegraded":"2024-09-14T05:31:31.202387+0000","last_fullsized":"2024-09-14T05:31:31.202387+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'657","last_scrub_stamp":"2024-09-14T05:31:31.202235+0000","last_deep_scrub":"132'657","last_deep_scrub_stamp":"2024-09-14T05:31:31.202235+0000","last_clean_scrub_stamp":"2024-09-14T05:31:31.202235+0000","objects_scrubbed":25,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:51:05.512376+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0.00060523399999999996,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":12123,"num_read_kb":29901,"num_write":689,"num_write_kb":13072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"132'463","reported_seq":3578,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:29.278328+0000","last_change":"2024-09-14T05:31:29.278279+0000","last_active":"2024-09-14T05:31:29.278328+0000","last_peered":"2024-09-14T05:31:29.278328+0000","last_clean":"2024-09-14T05:31:29.278328+0000","last_became_active":"2024-09-14T05:29:24.012574+0000","last_became_peered":"2024-09-14T05:29:24.012574+0000","last_unstale":"2024-09-14T05:31:29.278328+0000","last_undegraded":"2024-09-14T05:31:29.278328+0000","last_fullsized":"2024-09-14T05:31:29.278328+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'463","last_scrub_stamp":"2024-09-14T05:31:29.278171+0000","last_deep_scrub":"132'463","last_deep_scrub_stamp":"2024-09-14T05:31:29.278171+0000","last_clean_scrub_stamp":"2024-09-14T05:31:29.278171+0000","objects_scrubbed":15,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:01:43.325421+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00064954900000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2817,"num_read_kb":13317,"num_write":361,"num_write_kb":9799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"132'1155","reported_seq":11427,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:39.211439+0000","last_change":"2024-09-14T05:31:39.211363+0000","last_active":"2024-09-14T05:31:39.211439+0000","last_peered":"2024-09-14T05:31:39.211439+0000","last_clean":"2024-09-14T05:31:39.211439+0000","last_became_active":"2024-09-14T05:29:24.012401+0000","last_became_peered":"2024-09-14T05:29:24.012401+0000","last_unstale":"2024-09-14T05:31:39.211439+0000","last_undegraded":"2024-09-14T05:31:39.211439+0000","last_fullsized":"2024-09-14T05:31:39.211439+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'1155","last_scrub_stamp":"2024-09-14T05:31:39.211215+0000","last_deep_scrub":"132'1155","last_deep_scrub_stamp":"2024-09-14T05:31:39.211215+0000","last_clean_scrub_stamp":"2024-09-14T05:31:39.211215+0000","objects_scrubbed":24,"log_size":1155,"log_dups_size":0,"ondisk_log_size":1155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:48.926400+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0.0018223110000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":14909,"num_read_kb":22344,"num_write":1338,"num_write_kb":6495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"132'386","reported_seq":2890,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:28.831371+0000","last_change":"2024-09-14T05:31:28.831314+0000","last_active":"2024-09-14T05:31:28.831371+0000","last_peered":"2024-09-14T05:31:28.831371+0000","last_clean":"2024-09-14T05:31:28.831371+0000","last_became_active":"2024-09-14T05:29:24.015315+0000","last_became_peered":"2024-09-14T05:29:24.015315+0000","last_unstale":"2024-09-14T05:31:28.831371+0000","last_undegraded":"2024-09-14T05:31:28.831371+0000","last_fullsized":"2024-09-14T05:31:28.831371+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'386","last_scrub_stamp":"2024-09-14T05:31:28.831222+0000","last_deep_scrub":"132'386","last_deep_scrub_stamp":"2024-09-14T05:31:28.831222+0000","last_clean_scrub_stamp":"2024-09-14T05:31:28.831222+0000","objects_scrubbed":19,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:59:09.622341+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001918957,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2150,"num_read_kb":8871,"num_write":290,"num_write_kb":3939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"132'556","reported_seq":11877,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:30.263954+0000","last_change":"2024-09-14T05:31:30.263895+0000","last_active":"2024-09-14T05:31:30.263954+0000","last_peered":"2024-09-14T05:31:30.263954+0000","last_clean":"2024-09-14T05:31:30.263954+0000","last_became_active":"2024-09-14T05:29:24.018854+0000","last_became_peered":"2024-09-14T05:29:24.018854+0000","last_unstale":"2024-09-14T05:31:30.263954+0000","last_undegraded":"2024-09-14T05:31:30.263954+0000","last_fullsized":"2024-09-14T05:31:30.263954+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'556","last_scrub_stamp":"2024-09-14T05:31:30.263784+0000","last_deep_scrub":"132'556","last_deep_scrub_stamp":"2024-09-14T05:31:30.263784+0000","last_clean_scrub_stamp":"2024-09-14T05:31:30.263784+0000","objects_scrubbed":24,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:54:08.652953+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.000829428,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":12059,"num_read_kb":31916,"num_write":436,"num_write_kb":11843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"132'405","reported_seq":3066,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:30.073434+0000","last_change":"2024-09-14T05:31:30.073369+0000","last_active":"2024-09-14T05:31:30.073434+0000","last_peered":"2024-09-14T05:31:30.073434+0000","last_clean":"2024-09-14T05:31:30.073434+0000","last_became_active":"2024-09-14T05:29:24.010897+0000","last_became_peered":"2024-09-14T05:29:24.010897+0000","last_unstale":"2024-09-14T05:31:30.073434+0000","last_undegraded":"2024-09-14T05:31:30.073434+0000","last_fullsized":"2024-09-14T05:31:30.073434+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'405","last_scrub_stamp":"2024-09-14T05:31:30.073260+0000","last_deep_scrub":"132'405","last_deep_scrub_stamp":"2024-09-14T05:31:30.073260+0000","last_clean_scrub_stamp":"2024-09-14T05:31:30.073260+0000","objects_scrubbed":16,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:15:03.502947+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00051683800000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":2374,"num_read_kb":22456,"num_write":338,"num_write_kb":8939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"132'595","reported_seq":4067,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:37.210583+0000","last_change":"2024-09-14T05:31:37.210501+0000","last_active":"2024-09-14T05:31:37.210583+0000","last_peered":"2024-09-14T05:31:37.210583+0000","last_clean":"2024-09-14T05:31:37.210583+0000","last_became_active":"2024-09-14T05:29:24.013792+0000","last_became_peered":"2024-09-14T05:29:24.013792+0000","last_unstale":"2024-09-14T05:31:37.210583+0000","last_undegraded":"2024-09-14T05:31:37.210583+0000","last_fullsized":"2024-09-14T05:31:37.210583+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'595","last_scrub_stamp":"2024-09-14T05:31:37.210374+0000","last_deep_scrub":"132'595","last_deep_scrub_stamp":"2024-09-14T05:31:37.210374+0000","last_clean_scrub_stamp":"2024-09-14T05:31:37.210374+0000","objects_scrubbed":19,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:52:57.315921+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.001768403,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5197,"num_read_kb":24484,"num_write":526,"num_write_kb":3336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"132'503","reported_seq":2655,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:33.147183+0000","last_change":"2024-09-14T05:31:33.147183+0000","last_active":"2024-09-14T05:31:33.147183+0000","last_peered":"2024-09-14T05:31:33.147183+0000","last_clean":"2024-09-14T05:31:33.147183+0000","last_became_active":"2024-09-14T05:30:38.013627+0000","last_became_peered":"2024-09-14T05:30:38.013627+0000","last_unstale":"2024-09-14T05:31:33.147183+0000","last_undegraded":"2024-09-14T05:31:33.147183+0000","last_fullsized":"2024-09-14T05:31:33.147183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'503","last_scrub_stamp":"2024-09-14T05:31:33.147078+0000","last_deep_scrub":"132'503","last_deep_scrub_stamp":"2024-09-14T05:31:33.147078+0000","last_clean_scrub_stamp":"2024-09-14T05:31:33.147078+0000","objects_scrubbed":13,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:06.839124+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00050464100000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3016,"num_read_kb":16678,"num_write":502,"num_write_kb":10866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2769134,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"132'693","reported_seq":2515,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-14T05:31:35.174783+0000","last_change":"2024-09-14T05:31:35.174726+0000","last_active":"2024-09-14T05:31:35.174783+0000","last_peered":"2024-09-14T05:31:35.174783+0000","last_clean":"2024-09-14T05:31:35.174783+0000","last_became_active":"2024-09-14T05:29:24.014371+0000","last_became_peered":"2024-09-14T05:29:24.014371+0000","last_unstale":"2024-09-14T05:31:35.174783+0000","last_undegraded":"2024-09-14T05:31:35.174783+0000","last_fullsized":"2024-09-14T05:31:35.174783+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'693","last_scrub_stamp":"2024-09-14T05:31:35.174612+0000","last_deep_scrub":"132'693","last_deep_scrub_stamp":"2024-09-14T05:31:35.174612+0000","last_clean_scrub_stamp":"2024-09-14T05:31:35.174612+0000","objects_scrubbed":13,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:29:36.817380+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.00080910499999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3420,"num_read_kb":19916,"num_write":912,"num_write_kb":5503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":594,"num_object_clones":0,"num_object_copies":1188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":594,"num_whiteouts":590,"num_read":182740,"num_read_kb":565764,"num_write":20809,"num_write_kb":280821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3418028,"num_keys_recovered":45,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19840,"ondisk_log_size":19840,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476773,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54928,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316912,"statfs":{"total":96636764160,"available":96580517888,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509476,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41040,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330800,"statfs":{"total":96636764160,"available":96594739200,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509477,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343276,"statfs":{"total":96636764160,"available":96607514624,"internally_reserved":0,"allocated":1576960,"data_stored":747924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509475,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2170880,"data_stored":1338292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509476,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28560,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343280,"statfs":{"total":96636764160,"available":96607518720,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509476,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72980,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298860,"statfs":{"total":96636764160,"available":96562032640,"internally_reserved":0,"allocated":1576960,"data_stored":747924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1572864,"data_stored":747905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509478,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55512,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316328,"statfs":{"total":96636764160,"available":96579919872,"internally_reserved":0,"allocated":2170880,"data_stored":1338292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T05:31:51.575 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-14T05:31:51.877 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-14T05:31:51.877 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-14T05:31:51.877 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-14T05:31:51.877 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T05:31:51.914 INFO:tasks.ceph.osd.0:Stopped 2024-09-14T05:31:51.915 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-14T05:31:51.915 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T05:31:51.984 INFO:tasks.ceph.osd.1:Stopped 2024-09-14T05:31:51.984 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-14T05:31:51.985 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T05:31:52.058 INFO:tasks.ceph.osd.2:Stopped 2024-09-14T05:31:52.058 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-14T05:31:52.058 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T05:31:52.129 INFO:tasks.ceph.osd.3:Stopped 2024-09-14T05:31:52.129 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-14T05:31:52.129 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T05:31:52.167 INFO:tasks.ceph.osd.4:Stopped 2024-09-14T05:31:52.167 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-14T05:31:52.168 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T05:31:52.255 INFO:tasks.ceph.osd.5:Stopped 2024-09-14T05:31:52.256 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-14T05:31:52.256 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T05:31:52.289 INFO:tasks.ceph.osd.6:Stopped 2024-09-14T05:31:52.289 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-14T05:31:52.289 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T05:31:52.358 INFO:tasks.ceph.osd.7:Stopped 2024-09-14T05:31:52.359 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-14T05:31:52.359 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-14T05:31:52.359 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T05:31:52.384 INFO:tasks.ceph.mgr.y:Stopped 2024-09-14T05:31:52.384 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-14T05:31:52.385 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T05:31:52.427 INFO:tasks.ceph.mgr.x:Stopped 2024-09-14T05:31:52.427 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-14T05:31:52.428 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-14T05:31:52.428 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T05:31:52.438 INFO:tasks.ceph.mon.a:Stopped 2024-09-14T05:31:52.439 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-14T05:31:52.439 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T05:31:52.489 INFO:tasks.ceph.mon.c:Stopped 2024-09-14T05:31:52.489 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-14T05:31:52.490 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T05:31:52.502 INFO:tasks.ceph.mon.b:Stopped 2024-09-14T05:31:52.502 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-14T05:31:52.503 DEBUG:teuthology.orchestra.run.smithi071:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-14T05:31:52.575 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-14T05:31:52.575 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-14T05:31:52.764 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-14T05:31:52.764 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-14T05:31:52.860 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-14T05:31:52.860 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-14T05:31:52.941 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-14T05:31:52.941 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-14T05:31:52.991 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-14T05:31:52.991 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-14T05:31:53.166 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-14T05:31:53.166 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-14T05:31:53.260 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-14T05:31:53.260 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-14T05:31:53.310 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-14T05:31:53.315 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-14T05:31:53.357 INFO:tasks.ceph:Archiving mon data... 2024-09-14T05:31:53.358 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900319/data/mon.a.tgz 2024-09-14T05:31:53.358 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-09-14T05:31:53.374 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.yRa8F7UB4H 2024-09-14T05:31:53.374 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.yRa8F7UB4H 2024-09-14T05:31:53.518 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.yRa8F7UB4H 2024-09-14T05:31:53.601 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.yRa8F7UB4H is 353KB 2024-09-14T05:31:53.699 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.yRa8F7UB4H 2024-09-14T05:31:53.713 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900319/data/mon.c.tgz 2024-09-14T05:31:53.713 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-09-14T05:31:53.767 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.vJLloN8S8n 2024-09-14T05:31:53.768 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.vJLloN8S8n 2024-09-14T05:31:53.926 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.vJLloN8S8n 2024-09-14T05:31:54.009 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.vJLloN8S8n is 478KB 2024-09-14T05:31:54.107 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.vJLloN8S8n 2024-09-14T05:31:54.122 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900319/data/mon.b.tgz 2024-09-14T05:31:54.123 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-09-14T05:31:54.141 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.Cu7G6KTWIZ 2024-09-14T05:31:54.141 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Cu7G6KTWIZ 2024-09-14T05:31:54.321 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.Cu7G6KTWIZ 2024-09-14T05:31:54.405 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.Cu7G6KTWIZ is 478KB 2024-09-14T05:31:54.501 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.Cu7G6KTWIZ 2024-09-14T05:31:54.516 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-14T05:31:54.516 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-14T05:31:54.519 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-14T05:31:54.597 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-14T05:31:54.635 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-14T05:31:54.635 INFO:tasks.ceph:Archiving crash dumps... 2024-09-14T05:31:54.637 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900319/remote/smithi071/crash 2024-09-14T05:31:54.638 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-14T05:31:54.662 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900319/remote/smithi150/crash 2024-09-14T05:31:54.663 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-14T05:31:54.699 INFO:tasks.ceph:Compressing logs... 2024-09-14T05:31:54.699 DEBUG:teuthology.orchestra.run.smithi071:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-14T05:31:54.707 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-14T05:31:54.779 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33510.log 2024-09-14T05:31:54.780 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33390.log 2024-09-14T05:31:54.780 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33510.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-14T05:31:54.780 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33510.log.gz 2024-09-14T05:31:54.780 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33259.log 2024-09-14T05:31:54.780 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33390.log.gz 2024-09-14T05:31:54.780 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33430.log 2024-09-14T05:31:54.780 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33259.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33259.log.gz 2024-09-14T05:31:54.781 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33339.log 2024-09-14T05:31:54.781 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33470.log 2024-09-14T05:31:54.781 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-14T05:31:54.781 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-14T05:31:54.781 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33430.log: /var/log/ceph/ceph-client.admin.33339.log: /var/log/ceph/ceph-client.admin.33470.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33430.log.gz 2024-09-14T05:31:54.781 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33339.log.gz -- replaced with /var/log/ceph/ceph-client.admin.33470.log.gz 2024-09-14T05:31:54.781 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-09-14T05:31:54.781 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33219.log 2024-09-14T05:31:54.781 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33299.log 2024-09-14T05:31:54.781 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.33219.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-14T05:31:54.781 INFO:teuthology.orchestra.run.smithi150.stderr: 28.9% -- replaced with /var/log/ceph/ceph-client.admin.33219.log.gz 2024-09-14T05:31:54.781 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33299.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-14T05:31:54.782 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33299.log.gz 2024-09-14T05:31:54.782 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-14T05:31:54.782 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-14T05:31:54.782 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-14T05:31:54.782 INFO:teuthology.orchestra.run.smithi150.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-14T05:31:54.782 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-mon.b.log: 85.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-14T05:31:54.782 INFO:teuthology.orchestra.run.smithi150.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-14T05:31:54.834 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-14T05:31:54.834 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38897.log 2024-09-14T05:31:54.834 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39498.log 2024-09-14T05:31:54.834 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51950.log 2024-09-14T05:31:54.834 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38897.log.gz 2024-09-14T05:31:54.835 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54817.log 2024-09-14T05:31:54.835 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39498.log.gz 2024-09-14T05:31:54.835 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52712.log 2024-09-14T05:31:54.835 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51950.log.gz 2024-09-14T05:31:54.835 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52799.log 2024-09-14T05:31:54.835 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54817.log.gz 2024-09-14T05:31:54.835 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54268.log 2024-09-14T05:31:54.835 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52712.log.gz 2024-09-14T05:31:54.835 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49199.log 2024-09-14T05:31:54.835 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52799.log.gz 2024-09-14T05:31:54.835 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53221.log 2024-09-14T05:31:54.835 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54268.log.gz 2024-09-14T05:31:54.835 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33653.log 2024-09-14T05:31:54.836 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49199.log.gz 2024-09-14T05:31:54.836 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38804.log 2024-09-14T05:31:54.836 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53221.log.gz 2024-09-14T05:31:54.836 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44973.log 2024-09-14T05:31:54.836 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33653.log.gz 2024-09-14T05:31:54.836 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39496.log 2024-09-14T05:31:54.836 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38804.log.gz 2024-09-14T05:31:54.836 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44973.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.44557.log 2024-09-14T05:31:54.836 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44973.log.gz 2024-09-14T05:31:54.836 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41223.log 2024-09-14T05:31:54.836 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39496.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39496.log.gz 2024-09-14T05:31:54.836 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39158.log 2024-09-14T05:31:54.836 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44557.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44557.log.gz 2024-09-14T05:31:54.837 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54030.log 2024-09-14T05:31:54.837 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41223.log.gz 2024-09-14T05:31:54.837 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47812.log 2024-09-14T05:31:54.837 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39158.log.gz 2024-09-14T05:31:54.837 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55321.log 2024-09-14T05:31:54.837 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54030.log.gz 2024-09-14T05:31:54.837 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52373.log 2024-09-14T05:31:54.837 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47812.log.gz 2024-09-14T05:31:54.837 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55195.log 2024-09-14T05:31:54.837 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55321.log.gz 2024-09-14T05:31:54.837 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42641.log 2024-09-14T05:31:54.837 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52373.log.gz 2024-09-14T05:31:54.837 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43556.log 2024-09-14T05:31:54.838 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55195.log.gz 2024-09-14T05:31:54.838 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54363.log 2024-09-14T05:31:54.838 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42641.log.gz 2024-09-14T05:31:54.838 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50512.log 2024-09-14T05:31:54.838 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43556.log.gz 2024-09-14T05:31:54.838 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48538.log 2024-09-14T05:31:54.838 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54363.log.gz 2024-09-14T05:31:54.838 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50512.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-14T05:31:54.838 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50512.log.gz 2024-09-14T05:31:54.838 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52092.log 2024-09-14T05:31:54.838 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48538.log.gz 2024-09-14T05:31:54.838 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53137.log 2024-09-14T05:31:54.838 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33262.log 2024-09-14T05:31:54.839 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52092.log.gz 2024-09-14T05:31:54.839 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53137.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53137.log.gz 2024-09-14T05:31:54.839 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46793.log 2024-09-14T05:31:54.839 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53389.log 2024-09-14T05:31:54.839 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33262.log.gz 2024-09-14T05:31:54.839 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39154.log 2024-09-14T05:31:54.839 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46793.log.gz 2024-09-14T05:31:54.839 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53389.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53389.log.gz 2024-09-14T05:31:54.839 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50308.log 2024-09-14T05:31:54.839 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49580.log 2024-09-14T05:31:54.839 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39154.log.gz 2024-09-14T05:31:54.839 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50308.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50308.log.gz 2024-09-14T05:31:54.839 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39162.log 2024-09-14T05:31:54.839 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42480.log 2024-09-14T05:31:54.840 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49580.log.gz 2024-09-14T05:31:54.840 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39162.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39162.log.gz 2024-09-14T05:31:54.840 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42423.log 2024-09-14T05:31:54.840 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46728.log 2024-09-14T05:31:54.840 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42480.log.gz 2024-09-14T05:31:54.840 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42423.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42423.log.gz 2024-09-14T05:31:54.840 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40283.log 2024-09-14T05:31:54.840 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40484.log 2024-09-14T05:31:54.840 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46728.log.gz 2024-09-14T05:31:54.840 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40283.log.gz 2024-09-14T05:31:54.840 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39487.log 2024-09-14T05:31:54.840 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44896.log 2024-09-14T05:31:54.840 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40484.log.gz 2024-09-14T05:31:54.841 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39487.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39487.log.gz 2024-09-14T05:31:54.841 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55613.log 2024-09-14T05:31:54.841 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54033.log 2024-09-14T05:31:54.841 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44896.log.gz 2024-09-14T05:31:54.841 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55613.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55613.log.gz 2024-09-14T05:31:54.841 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39497.log 2024-09-14T05:31:54.841 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45054.log 2024-09-14T05:31:54.841 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54033.log.gz 2024-09-14T05:31:54.841 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39497.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39497.log.gz 2024-09-14T05:31:54.841 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54649.log 2024-09-14T05:31:54.841 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42860.log 2024-09-14T05:31:54.841 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45054.log.gz 2024-09-14T05:31:54.841 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54649.log.gz 2024-09-14T05:31:54.842 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46546.log 2024-09-14T05:31:54.842 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54366.log 2024-09-14T05:31:54.842 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42860.log.gz 2024-09-14T05:31:54.842 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46546.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46546.log.gz 2024-09-14T05:31:54.842 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49684.log 2024-09-14T05:31:54.842 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40073.log 2024-09-14T05:31:54.842 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54366.log.gz 2024-09-14T05:31:54.842 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49684.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49684.log.gz 2024-09-14T05:31:54.842 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38723.log 2024-09-14T05:31:54.842 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48677.log 2024-09-14T05:31:54.842 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40073.log.gz 2024-09-14T05:31:54.842 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38723.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38723.log.gz 2024-09-14T05:31:54.842 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40480.log 2024-09-14T05:31:54.842 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54025.log 2024-09-14T05:31:54.843 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48677.log.gz 2024-09-14T05:31:54.843 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40480.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40480.log.gz 2024-09-14T05:31:54.843 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43476.log 2024-09-14T05:31:54.843 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55489.log 2024-09-14T05:31:54.843 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54025.log.gz 2024-09-14T05:31:54.843 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43476.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43476.log.gz 2024-09-14T05:31:54.843 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38623.log 2024-09-14T05:31:54.843 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44734.log 2024-09-14T05:31:54.843 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55489.log.gz 2024-09-14T05:31:54.843 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38623.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38623.log.gz 2024-09-14T05:31:54.843 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54691.log 2024-09-14T05:31:54.843 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50817.log 2024-09-14T05:31:54.843 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44734.log.gz 2024-09-14T05:31:54.843 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54691.log.gz 2024-09-14T05:31:54.844 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39755.log 2024-09-14T05:31:54.844 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48239.log 2024-09-14T05:31:54.844 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50817.log.gz 2024-09-14T05:31:54.844 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52885.log 2024-09-14T05:31:54.844 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39755.log.gz 2024-09-14T05:31:54.844 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49501.log 2024-09-14T05:31:54.844 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48239.log.gz 2024-09-14T05:31:54.844 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54032.log 2024-09-14T05:31:54.844 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52885.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52885.log.gz 2024-09-14T05:31:54.844 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49501.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49501.log.gz 2024-09-14T05:31:54.844 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51354.log 2024-09-14T05:31:54.844 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44116.log 2024-09-14T05:31:54.844 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54032.log.gz 2024-09-14T05:31:54.845 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51354.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51354.log.gz 2024-09-14T05:31:54.845 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38981.log 2024-09-14T05:31:54.845 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39486.log 2024-09-14T05:31:54.845 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44116.log.gz 2024-09-14T05:31:54.845 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38981.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38981.log.gz -5 2024-09-14T05:31:54.845 INFO:teuthology.orchestra.run.smithi071.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45333.log 2024-09-14T05:31:54.845 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53515.log 2024-09-14T05:31:54.845 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39486.log.gz 2024-09-14T05:31:54.845 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45333.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45333.log.gz 2024-09-14T05:31:54.845 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51893.log 2024-09-14T05:31:54.845 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41923.log 2024-09-14T05:31:54.845 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53515.log.gz 2024-09-14T05:31:54.845 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51893.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51893.log.gz 2024-09-14T05:31:54.845 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41843.log 2024-09-14T05:31:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51213.log 2024-09-14T05:31:54.846 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41923.log.gz 2024-09-14T05:31:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41843.log.gz 2024-09-14T05:31:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47366.log 2024-09-14T05:31:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53347.log 2024-09-14T05:31:54.846 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51213.log.gz 2024-09-14T05:31:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47366.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47366.log.gz 2024-09-14T05:31:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27425.log 2024-09-14T05:31:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39155.log 2024-09-14T05:31:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53347.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53347.log.gz 2024-09-14T05:31:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph.tmp-client.admin.27425.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27425.log.gz 2024-09-14T05:31:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39695.log 2024-09-14T05:31:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39875.log 2024-09-14T05:31:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39155.log.gz 2024-09-14T05:31:54.847 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39695.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-14T05:31:54.847 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39695.log.gz 2024-09-14T05:31:54.847 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39871.log 2024-09-14T05:31:54.847 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39875.log.gz 2024-09-14T05:31:54.847 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52843.log 2024-09-14T05:31:54.847 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53179.log 2024-09-14T05:31:54.847 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39871.log.gz 2024-09-14T05:31:54.847 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52843.log.gz 2024-09-14T05:31:54.847 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54368.log 2024-09-14T05:31:54.847 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46462.log 2024-09-14T05:31:54.847 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53179.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53179.log.gz 2024-09-14T05:31:54.847 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54368.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54368.log.gz 2024-09-14T05:31:54.847 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40881.log 2024-09-14T05:31:54.848 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54369.log 2024-09-14T05:31:54.848 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46462.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46462.log.gz 2024-09-14T05:31:54.848 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40881.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-14T05:31:54.848 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50533.log 2024-09-14T05:31:54.848 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40881.log.gz 2024-09-14T05:31:54.848 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54369.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.54369.log.gz --verbose 2024-09-14T05:31:54.848 INFO:teuthology.orchestra.run.smithi071.stderr: -- /var/log/ceph/ceph-client.admin.55069.log 2024-09-14T05:31:54.848 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50065.log 2024-09-14T05:31:54.848 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50533.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50533.log.gz 2024-09-14T05:31:54.848 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45764.log 2024-09-14T05:31:54.848 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55069.log.gz 2024-09-14T05:31:54.848 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38855.log 2024-09-14T05:31:54.848 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50065.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50065.log.gz 2024-09-14T05:31:54.849 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55363.log 2024-09-14T05:31:54.849 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45764.log.gz 2024-09-14T05:31:54.849 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38855.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38855.log.gz 2024-09-14T05:31:54.849 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40398.log 2024-09-14T05:31:54.849 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47308.log 2024-09-14T05:31:54.849 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55363.log.gz 2024-09-14T05:31:54.849 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.40398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44514.log 2024-09-14T05:31:54.849 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47308.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.47308.log.gz 2024-09-14T05:31:54.849 INFO:teuthology.orchestra.run.smithi071.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54901.log 2024-09-14T05:31:54.849 INFO:teuthology.orchestra.run.smithi071.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.0.40398.log.gz 2024-09-14T05:31:54.849 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44377.log 2024-09-14T05:31:54.849 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44514.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44514.log.gz 2024-09-14T05:31:54.849 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48301.log 2024-09-14T05:31:54.850 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54901.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54901.log.gz 2024-09-14T05:31:54.850 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53692.log 2024-09-14T05:31:54.850 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44377.log.gz 2024-09-14T05:31:54.850 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48923.log 2024-09-14T05:31:54.850 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48301.log.gz 2024-09-14T05:31:54.850 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45410.log 2024-09-14T05:31:54.850 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53692.log.gz 2024-09-14T05:31:54.850 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42042.log 2024-09-14T05:31:54.850 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48923.log.gz 2024-09-14T05:31:54.850 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53473.log 2024-09-14T05:31:54.850 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45410.log.gz 2024-09-14T05:31:54.850 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41604.log 2024-09-14T05:31:54.850 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42042.log.gz 2024-09-14T05:31:54.850 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54565.log 2024-09-14T05:31:54.851 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53473.log.gz 2024-09-14T05:31:54.851 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41604.log.gz 2024-09-14T05:31:54.851 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54269.log 2024-09-14T05:31:54.851 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39161.log 2024-09-14T05:31:54.851 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54565.log.gz 2024-09-14T05:31:54.851 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39359.log 2024-09-14T05:31:54.851 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54269.log.gz 2024-09-14T05:31:54.851 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50974.log 2024-09-14T05:31:54.851 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39161.log.gz 2024-09-14T05:31:54.851 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55279.log 2024-09-14T05:31:54.851 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39359.log.gz 2024-09-14T05:31:54.851 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48943.log 2024-09-14T05:31:54.851 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50974.log.gz 2024-09-14T05:31:54.851 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55279.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55279.log.gz 2024-09-14T05:31:54.852 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-14T05:31:54.852 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54943.log 2024-09-14T05:31:54.852 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48943.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48943.log.gz 2024-09-14T05:31:54.852 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33564.log 2024-09-14T05:31:54.852 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54943.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54943.log.gz 2024-09-14T05:31:54.852 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54733.log 2024-09-14T05:31:54.852 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53697.log 2024-09-14T05:31:54.852 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33564.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33564.log.gz 2024-09-14T05:31:54.852 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54733.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54733.log.gz 2024-09-14T05:31:54.852 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51116.log 2024-09-14T05:31:54.852 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41387.log 2024-09-14T05:31:54.852 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53697.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53697.log.gz 2024-09-14T05:31:54.852 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51116.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.50045.log 2024-09-14T05:31:54.853 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51116.log.gz 2024-09-14T05:31:54.853 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53011.log 2024-09-14T05:31:54.853 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41387.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41387.log.gz 2024-09-14T05:31:54.853 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50045.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50045.log.gz 2024-09-14T05:31:54.853 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43737.log 2024-09-14T05:31:54.853 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48082.log 2024-09-14T05:31:54.853 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53011.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53011.log.gz 2024-09-14T05:31:54.853 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43737.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43737.log.gz 2024-09-14T05:31:54.853 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45940.log 2024-09-14T05:31:54.853 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50651.log 2024-09-14T05:31:54.853 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48082.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48082.log.gz 2024-09-14T05:31:54.853 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39869.log 2024-09-14T05:31:54.853 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45940.log.gz 2024-09-14T05:31:54.853 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50735.log 2024-09-14T05:31:54.854 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50651.log.gz 2024-09-14T05:31:54.854 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39869.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39869.log.gz 2024-09-14T05:31:54.854 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47960.log 2024-09-14T05:31:54.854 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49481.log 2024-09-14T05:31:54.854 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50735.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50735.log.gz 2024-09-14T05:31:54.854 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47960.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.47960.log.gz /var/log/ceph/ceph-client.admin.54859.log 2024-09-14T05:31:54.854 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-09-14T05:31:54.854 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54365.log 2024-09-14T05:31:54.854 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49481.log.gz 2024-09-14T05:31:54.854 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54859.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54859.log.gz 2024-09-14T05:31:54.854 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47226.log 2024-09-14T05:31:54.854 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-14T05:31:54.854 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54365.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54365.log.gz 2024-09-14T05:31:54.854 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55027.log 2024-09-14T05:31:54.855 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47226.log.gz 2024-09-14T05:31:54.855 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41243.log 2024-09-14T05:31:54.855 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55027.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55027.log.gz -5 2024-09-14T05:31:54.855 INFO:teuthology.orchestra.run.smithi071.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54028.log 2024-09-14T05:31:54.855 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.admin.41243.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.55237.log 2024-09-14T05:31:54.855 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41243.log.gz 2024-09-14T05:31:54.855 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54028.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40031.log 2024-09-14T05:31:54.855 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54028.log.gz 2024-09-14T05:31:54.855 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52331.log 2024-09-14T05:31:54.855 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55237.log.gz 2024-09-14T05:31:54.855 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53686.log 2024-09-14T05:31:54.855 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40031.log.gz 2024-09-14T05:31:54.855 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45555.log 2024-09-14T05:31:54.855 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52331.log.gz 2024-09-14T05:31:54.856 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47426.log 2024-09-14T05:31:54.856 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53686.log.gz 2024-09-14T05:31:54.856 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53696.log 2024-09-14T05:31:54.856 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45555.log.gz 2024-09-14T05:31:54.856 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39868.log 2024-09-14T05:31:54.856 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47426.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47426.log.gz 2024-09-14T05:31:54.856 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51574.log 2024-09-14T05:31:54.856 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53696.log.gz 2024-09-14T05:31:54.856 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39868.log.gz 2024-09-14T05:31:54.856 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39159.log 2024-09-14T05:31:54.856 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-14T05:31:54.856 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51574.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51574.log.gz 2024-09-14T05:31:54.856 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52969.log 2024-09-14T05:31:54.856 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39159.log.gz 2024-09-14T05:31:54.857 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40115.log 2024-09-14T05:31:54.857 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52969.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52969.log.gz 2024-09-14T05:31:54.857 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53305.log 2024-09-14T05:31:54.857 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40115.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40115.log.gz 2024-09-14T05:31:54.857 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48619.log 2024-09-14T05:31:54.857 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53305.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53305.log.gz 2024-09-14T05:31:54.857 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39493.log 2024-09-14T05:31:54.857 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48619.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48619.log.gz 2024-09-14T05:31:54.857 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43680.log 2024-09-14T05:31:54.857 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39493.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39493.log.gz 2024-09-14T05:31:54.857 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51992.log 2024-09-14T05:31:54.857 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43680.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43680.log.gz 2024-09-14T05:31:54.857 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43858.log 2024-09-14T05:31:54.857 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51992.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51992.log.gz 2024-09-14T05:31:54.858 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54775.log 2024-09-14T05:31:54.858 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43858.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43858.log.gz 2024-09-14T05:31:54.858 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53431.log 2024-09-14T05:31:54.858 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54775.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54775.log.gz 2024-09-14T05:31:54.858 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39870.log 2024-09-14T05:31:54.858 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53431.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53431.log.gz 2024-09-14T05:31:54.858 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43160.log 2024-09-14T05:31:54.858 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39870.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39870.log.gz 2024-09-14T05:31:54.858 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47705.log 2024-09-14T05:31:54.858 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43160.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43160.log.gz 2024-09-14T05:31:54.858 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-14T05:31:54.858 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47705.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47705.log.gz 2024-09-14T05:31:54.858 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50186.log 2024-09-14T05:31:54.859 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42117.log 2024-09-14T05:31:54.859 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50186.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51452.log 2024-09-14T05:31:54.859 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50186.log.gz 2024-09-14T05:31:54.859 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52732.log 2024-09-14T05:31:54.859 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42117.log: /var/log/ceph/ceph-client.admin.51452.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42117.log.gz 2024-09-14T05:31:54.859 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51452.log.gz 2024-09-14T05:31:54.859 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53909.log 2024-09-14T05:31:54.859 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49744.log 2024-09-14T05:31:54.859 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52732.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52732.log.gz 2024-09-14T05:31:54.859 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53909.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53909.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46403.log 2024-09-14T05:31:54.859 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-09-14T05:31:54.859 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-14T05:31:54.859 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49744.log: /var/log/ceph/ceph-client.admin.46403.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49744.log.gz 2024-09-14T05:31:54.859 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46403.log.gz 2024-09-14T05:31:54.860 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53913.log 2024-09-14T05:31:54.860 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose --/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.41646.log 2024-09-14T05:31:54.860 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53913.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53913.log.gz 2024-09-14T05:31:54.860 INFO:teuthology.orchestra.run.smithi071.stderr: 91.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-14T05:31:54.860 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47725.log 2024-09-14T05:31:54.860 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41646.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.50693.log 2024-09-14T05:31:54.860 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41646.log.gz 2024-09-14T05:31:54.860 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40241.log 2024-09-14T05:31:54.860 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47725.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47725.log.gz 2024-09-14T05:31:54.860 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46927.log 2024-09-14T05:31:54.860 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50693.log.gz 2024-09-14T05:31:54.860 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40241.log.gz 2024-09-14T05:31:54.860 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54607.log 2024-09-14T05:31:54.861 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46927.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47876.log 2024-09-14T05:31:54.861 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46927.log.gz 2024-09-14T05:31:54.861 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39163.log 2024-09-14T05:31:54.861 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54607.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54607.log.gz 2024-09-14T05:31:54.861 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47876.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39160.log 2024-09-14T05:31:54.861 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47876.log.gz 2024-09-14T05:31:54.861 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39163.log.gz 2024-09-14T05:31:54.861 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43223.log 2024-09-14T05:31:54.861 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39160.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39160.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50127.log 2024-09-14T05:31:54.861 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-09-14T05:31:54.861 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53095.log 2024-09-14T05:31:54.861 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50127.log: /var/log/ceph/ceph-client.admin.43223.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43223.log.gz 2024-09-14T05:31:54.862 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50127.log.gz 2024-09-14T05:31:54.862 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53693.log 2024-09-14T05:31:54.862 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49600.log 2024-09-14T05:31:54.862 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53095.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53095.log.gz 2024-09-14T05:31:54.862 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53693.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53693.log.gz 2024-09-14T05:31:54.862 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43339.log 2024-09-14T05:31:54.862 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53691.log 2024-09-14T05:31:54.862 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43339.log: /var/log/ceph/ceph-client.admin.49600.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49600.log.gz 2024-09-14T05:31:54.862 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43339.log.gz 2024-09-14T05:31:54.862 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39715.log 2024-09-14T05:31:54.862 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38779.log 2024-09-14T05:31:54.862 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53691.log.gz 2024-09-14T05:31:54.862 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39715.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39715.log.gz 2024-09-14T05:31:54.863 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55531.log 2024-09-14T05:31:54.863 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45676.log 2024-09-14T05:31:54.863 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38779.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38779.log.gz 2024-09-14T05:31:54.863 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55531.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55531.log.gz 2024-09-14T05:31:54.863 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49142.log 2024-09-14T05:31:54.863 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45676.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45676.log.gz 2024-09-14T05:31:54.863 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53053.log 2024-09-14T05:31:54.863 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48719.log 2024-09-14T05:31:54.863 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49142.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49142.log.gz 2024-09-14T05:31:54.863 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53053.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53053.log.gz 2024-09-14T05:31:54.863 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44076.log 2024-09-14T05:31:54.863 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48719.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48719.log.gz 2024-09-14T05:31:54.863 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42361.log 2024-09-14T05:31:54.863 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41126.log 2024-09-14T05:31:54.863 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44076.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44076.log.gz 2024-09-14T05:31:54.864 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42361.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42361.log.gz 2024-09-14T05:31:54.864 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42819.log 2024-09-14T05:31:54.864 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55111.log 2024-09-14T05:31:54.864 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41126.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41126.log.gz 2024-09-14T05:31:54.864 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42819.log.gz 2024-09-14T05:31:54.864 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45292.log 2024-09-14T05:31:54.864 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38939.log 2024-09-14T05:31:54.864 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55111.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55111.log.gz 2024-09-14T05:31:54.864 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45292.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45292.log.gz 2024-09-14T05:31:54.864 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40157.log 2024-09-14T05:31:54.864 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55573.log 2024-09-14T05:31:54.864 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38939.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38939.log.gz 2024-09-14T05:31:54.864 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40157.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40157.log.gz 2024-09-14T05:31:54.865 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54985.log 2024-09-14T05:31:54.865 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52629.log 2024-09-14T05:31:54.865 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55573.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55573.log.gz 2024-09-14T05:31:54.865 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54985.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54985.log.gz 2024-09-14T05:31:54.865 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40824.log 2024-09-14T05:31:54.865 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55447.log 2024-09-14T05:31:54.865 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52629.log: /var/log/ceph/ceph-client.admin.40824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52629.log.gz 2024-09-14T05:31:54.865 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40824.log.gz 2024-09-14T05:31:54.865 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50449.log 2024-09-14T05:31:54.865 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39499.log 2024-09-14T05:31:54.865 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55447.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55447.log.gz 2024-09-14T05:31:54.865 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50449.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50449.log.gz 2024-09-14T05:31:54.865 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40345.log 2024-09-14T05:31:54.865 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46341.log 2024-09-14T05:31:54.866 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39499.log: /var/log/ceph/ceph-client.admin.40345.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39499.log.gz 2024-09-14T05:31:54.866 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40345.log.gz 2024-09-14T05:31:54.866 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54267.log 2024-09-14T05:31:54.866 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53689.log 2024-09-14T05:31:54.866 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46341.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46341.log.gz 2024-09-14T05:31:54.866 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54267.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54267.log.gz 2024-09-14T05:31:54.866 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42937.log 2024-09-14T05:31:54.866 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40485.log 2024-09-14T05:31:54.866 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53689.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53689.log.gz 2024-09-14T05:31:54.866 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42937.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42937.log.gz 2024-09-14T05:31:54.866 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51573.log 2024-09-14T05:31:54.866 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55153.log 2024-09-14T05:31:54.866 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40485.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40485.log.gz 2024-09-14T05:31:54.866 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51573.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51573.log.gz 2024-09-14T05:31:54.867 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47044.log 2024-09-14T05:31:54.867 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45411.log 2024-09-14T05:31:54.867 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55153.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55153.log.gz 2024-09-14T05:31:54.867 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47044.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47044.log.gz 2024-09-14T05:31:54.867 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49923.log 2024-09-14T05:31:54.867 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53263.log 2024-09-14T05:31:54.867 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49923.log: /var/log/ceph/ceph-client.admin.45411.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49923.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45411.log.gz 2024-09-14T05:31:54.867 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-09-14T05:31:54.867 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53695.log 2024-09-14T05:31:54.867 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48761.log 2024-09-14T05:31:54.867 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53263.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53263.log.gz 2024-09-14T05:31:54.867 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53695.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53695.log.gz 2024-09-14T05:31:54.867 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-14T05:31:54.867 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44136.log 2024-09-14T05:31:54.868 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.48761.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48761.log.gz 2024-09-14T05:31:54.868 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46159.log 2024-09-14T05:31:54.868 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44136.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44136.log.gz 2024-09-14T05:31:54.868 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52927.log 2024-09-14T05:31:54.868 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46159.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46159.log.gz 2024-09-14T05:31:54.868 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40199.log 2024-09-14T05:31:54.868 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52927.log.gz 2024-09-14T05:31:54.868 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46321.log 2024-09-14T05:31:54.868 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40199.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40199.log.gz 2024-09-14T05:31:54.868 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45876.log 2024-09-14T05:31:54.868 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46321.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46321.log.gz 2024-09-14T05:31:54.868 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46504.log 2024-09-14T05:31:54.868 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45876.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45876.log.gz 2024-09-14T05:31:54.868 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46020.log 2024-09-14T05:31:54.869 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46504.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46504.log.gz 2024-09-14T05:31:54.869 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45875.log 2024-09-14T05:31:54.869 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46020.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46020.log.gz 2024-09-14T05:31:54.869 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54029.log 2024-09-14T05:31:54.869 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45875.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45875.log.gz 2024-09-14T05:31:54.869 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55405.log 2024-09-14T05:31:54.869 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54029.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54029.log.gz 2024-09-14T05:31:54.869 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55405.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55405.log.gz 2024-09-14T05:31:54.869 INFO:teuthology.orchestra.run.smithi071.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-14T05:31:54.891 INFO:teuthology.orchestra.run.smithi150.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-14T05:31:54.899 INFO:teuthology.orchestra.run.smithi071.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-14T05:31:55.003 INFO:teuthology.orchestra.run.smithi071.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-14T05:31:55.277 INFO:teuthology.orchestra.run.smithi150.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-14T05:31:55.804 INFO:teuthology.orchestra.run.smithi150.stderr: 95.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-14T05:31:56.519 INFO:teuthology.orchestra.run.smithi071.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-14T05:31:56.676 INFO:teuthology.orchestra.run.smithi150.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-14T05:31:56.882 INFO:teuthology.orchestra.run.smithi071.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-14T05:31:56.898 INFO:teuthology.orchestra.run.smithi071.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-14T05:31:57.760 INFO:teuthology.orchestra.run.smithi150.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-14T05:31:57.761 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-09-14T05:31:57.762 INFO:teuthology.orchestra.run.smithi150.stderr:real 0m3.006s 2024-09-14T05:31:57.762 INFO:teuthology.orchestra.run.smithi150.stderr:user 0m6.310s 2024-09-14T05:31:57.762 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m0.286s 2024-09-14T05:31:58.662 INFO:teuthology.orchestra.run.smithi071.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-14T05:31:58.664 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-09-14T05:31:58.665 INFO:teuthology.orchestra.run.smithi071.stderr:real 0m3.945s 2024-09-14T05:31:58.665 INFO:teuthology.orchestra.run.smithi071.stderr:user 0m9.987s 2024-09-14T05:31:58.665 INFO:teuthology.orchestra.run.smithi071.stderr:sys 0m0.537s 2024-09-14T05:31:58.665 INFO:tasks.ceph:Archiving logs... 2024-09-14T05:31:58.665 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900319/remote/smithi071/log 2024-09-14T05:31:58.666 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-14T05:32:00.127 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900319/remote/smithi150/log 2024-09-14T05:32:00.128 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-14T05:32:00.950 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-14T05:32:00.959 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-14T05:32:00.959 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-14T05:32:00.962 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-14T05:32:01.052 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-14T05:32:01.060 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-14T05:32:01.060 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T05:32:01.063 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T05:32:01.077 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-09-14T05:32:01.080 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-09-14T05:32:01.081 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-14T05:32:01.081 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-09-14T05:32:01.082 INFO:teuthology.orchestra.run.smithi071.stdout:^* hv01.front.sepia.ceph.com 3 6 377 36 -26us[ -19us] +/- 58ms 2024-09-14T05:32:01.082 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 38 +222us[ +228us] +/- 73ms 2024-09-14T05:32:01.082 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 -167us[ -167us] +/- 77ms 2024-09-14T05:32:01.082 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T05:32:01.085 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-14T05:32:01.085 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-09-14T05:32:01.085 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv01.front.sepia.ceph.com 3 6 377 34 -17us[ -16us] +/- 58ms 2024-09-14T05:32:01.086 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 35 +219us[ +221us] +/- 73ms 2024-09-14T05:32:01.086 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 -190us[ -189us] +/- 77ms 2024-09-14T05:32:01.086 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T05:32:01.086 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-14T05:32:01.095 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-14T05:32:01.096 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-14T05:32:01.104 DEBUG:teuthology.orchestra.run.smithi071:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-14T05:32:01.199 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-14T05:32:01.273 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-14T05:32:01.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T05:32:01.303 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-14T05:32:01.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T05:32:01.332 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-14T05:32:01.341 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-14T05:32:01.349 INFO:teuthology.task.internal:Duration was 593.220925 seconds 2024-09-14T05:32:01.349 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-14T05:32:01.357 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-14T05:32:01.357 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-14T05:32:01.360 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-14T05:32:01.397 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-14T05:32:01.413 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-14T05:32:01.707 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-14T05:32:01.707 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2024-09-14T05:32:01.707 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-14T05:32:01.768 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-09-14T05:32:01.768 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-14T05:32:01.802 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-14T05:32:01.803 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-14T05:32:01.811 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-14T05:32:01.870 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-14T05:32:01.870 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-14T05:32:01.877 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-14T05:32:02.022 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-14T05:32:02.031 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-14T05:32:02.031 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-14T05:32:02.056 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-14T05:32:02.081 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-14T05:32:02.090 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-14T05:32:02.098 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-14T05:32:02.119 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2024-09-14T05:32:02.145 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-09-14T05:32:02.171 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-14T05:32:02.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T05:32:02.199 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-14T05:32:02.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T05:32:02.226 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-14T05:32:02.236 INFO:teuthology.task.internal:Transferring archived files... 2024-09-14T05:32:02.237 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900319/remote/smithi071 2024-09-14T05:32:02.237 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-14T05:32:02.279 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900319/remote/smithi150 2024-09-14T05:32:02.280 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-14T05:32:02.312 INFO:teuthology.task.internal:Removing archive directory... 2024-09-14T05:32:02.313 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-14T05:32:02.315 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-14T05:32:02.363 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-14T05:32:02.374 INFO:teuthology.task.internal:Not uploading archives. 2024-09-14T05:32:02.374 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-14T05:32:02.383 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-14T05:32:02.383 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-14T05:32:02.385 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-14T05:32:02.397 INFO:teuthology.orchestra.run.smithi071.stdout: 266063 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 14 05:32 /home/ubuntu/cephtest 2024-09-14T05:32:02.418 INFO:teuthology.orchestra.run.smithi150.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 14 05:32 /home/ubuntu/cephtest 2024-09-14T05:32:02.419 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-14T05:32:02.429 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-14T05:32:02.450 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} duration: 593.2209250926971 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-14T05:32:02.450 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-14T05:32:02.542 INFO:teuthology.run:pass